Microsoft CEO Satya Nadella is defending the use of the term superintelligence for advanced artificial intelligence, while warning that increasingly powerful systems could become serious security liabilities without stronger safeguards. His comments highlight a growing challenge for the industry: developing more capable models is proving easier than ensuring their behavior remains predictable and safe.
In an interview with Reed Albergotti for Semafor Technology’s Future Proof, published on October 21, Nadella addressed the terminology, AI alignment and Microsoft's strategy for securing increasingly sophisticated systems. According to The American Bazaar, he also discussed the changing economics of AI, Microsoft's security initiatives and the company's future leadership.
Why Nadella accepts the term
Nadella defended the terminology used to describe the next stage of artificial intelligence, including the shift from AI to SI, meaning superintelligence. He pointed to US President Donald Trump as the person who introduced the name, saying he was comfortable with its adoption.
His response suggests that the terminology itself matters less to him than the underlying technological developments. The central issue, in his view, is how organizations handle systems whose capabilities are expanding rapidly while their safety and governance mechanisms struggle to keep pace.
The distinction matters because the discussion is not simply about naming a technology. It concerns the industry's ability to understand, control and responsibly deploy increasingly advanced models.
Capability is outpacing alignment
Nadella said the development of more powerful AI systems has exceeded even his earlier expectations. He pointed to scaling laws, which describe how increasing computing resources, training data and model size can improve performance, as evidence that the industry's technical approach has delivered substantial results.
He compared the observed reliability of these trends to Moore's Law, the longstanding observation that transistor counts on computer chips tend to increase over time. However, he stressed that scaling laws are empirical observations rather than a complete explanation of how intelligence develops.
The more difficult problem is alignment: ensuring that an AI system's behavior remains consistent with human intentions, instructions and safety requirements. Nadella acknowledged that progress in this area has been less convincing.
The ability to build increasingly capable models has advanced faster than the industry's ability to guarantee safe behavior. He expressed hope that researchers would eventually discover more effective ways to address the problem.
The insider security problem
Nadella argued that companies cannot depend entirely on protections embedded inside AI models. Instead, security must also operate through independent systems that monitor activity, establish boundaries and provide mechanisms for intervention.
He compared advanced AI systems to employees who have access to an organization's internal information and resources. Such access can create an insider risk if it is misused, whether intentionally or through unexpected behavior.
The analogy highlights a concern that becomes more important as organizations give AI systems greater responsibilities. A model with access to sensitive data, business processes or operational tools needs oversight beyond its own internal safeguards.
"all the capability to be an insider risk"
— Satya Nadella, Microsoft CEO
Microsoft's layered security strategy
Microsoft's proposed approach combines several mechanisms designed to keep AI deployments under organizational control. Nadella described the use of external assurance and control systems, multiple models and methods for reducing the opacity of AI operations.
The principle is to avoid treating an individual model as the sole authority responsible for its own safe operation. Independent oversight can give organizations additional ways to monitor behavior and maintain control as systems become more capable.
Nadella also discussed Agent 365, part of Microsoft's security initiative. The program is designed to help businesses manage and govern AI agents, software systems that can perform tasks and interact with digital tools on behalf of users or organizations.
The initiative reflects Microsoft's broader focus on making AI deployment manageable for organizations rather than treating model performance as the only measure of progress.
AI economics and Microsoft's future
The interview also addressed whether AI models are becoming commodities as competition increases and pricing moves from individual tokens toward completed tasks. Nadella offered a qualified assessment: products and services can become commoditized, but achieving scale does not necessarily eliminate differentiation.
He used cloud computing as an example. Although cloud infrastructure can be treated as a commodity, it supports a wide range of services and business offerings. The same distinction, he suggested, applies to AI: underlying models may become more interchangeable while the services built around them retain value.
Nadella was also asked about succession planning at Microsoft. He said the company has several promising candidates with the ambition and ability to lead. The potential successors include executives developed internally as well as people recruited from outside the organization who have since performed strongly.
Why it matters
Nadella's comments place security and alignment at the center of the superintelligence debate. The industry's ability to scale models is not, by itself, evidence that those systems can be safely trusted with increasingly important responsibilities.
For businesses, the practical challenge is establishing oversight before granting AI agents access to sensitive information and operational tools. Microsoft's emphasis on independent controls, monitoring and Agent 365 points toward a model in which security extends beyond the AI system itself.
The next important question is whether researchers and technology companies can make alignment progress alongside improvements in capability. Nadella's remarks identify the gap, but offer no definitive solution. How effectively companies implement external safeguards as AI systems become more capable will be an important measure of whether the technology can be deployed responsibly.