Satya Nadella says companies should treat AI models as insider risks

Microsoft chief executive Satya Nadella says companies should assume any advanced AI model they deploy could be compromised, and contain it from the start. He set out the case in a long article on X on Saturday.

“Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task,” Nadella wrote.

He said engineers could trace traditional software to a specific code path. Today’s models cannot. Companies cannot tie a model’s behaviour to its training data or its weights, he wrote, yet they give these systems sensitive data and let them take “mission-critical actions”.

A model provider’s assurances do not remove that responsibility, he added.

His answer is to treat both closed and open-weight frontier models as insider risks. That is not because they are malicious, he wrote. Any capable actor with access to important systems can make mistakes, or an attacker can compromise it.

“We need to separate the supply of intelligence from the authority over it,” he wrote.

Companies already manage powerful insiders, Nadella said. They establish identity, limit privileges, log activity and set containment boundaries. For AI, that means the controls on what a model can reach and do sit outside the model. He wants to separate the model from the software layer that runs its work and from the actions it can take.

He called transparent model reasoning “non-negotiable”. It is not enough on its own, he wrote, because no one yet knows how to make a model’s outputs reliably faithful.

Nadella listed seven principles. No single model should be the only dependency for an important outcome, or check its own work. Every meaningful model action should leave “tamper-proof human readable evidence”. Companies should test whole systems all the time, including against failures and attacks.

Companies should decide for themselves what a model can access and do. Audits should not depend on the model under review. The last two principles are containment, his emergency brake, and incident disclosure. When systems fail or someone compromises them, companies should tell those affected quickly and share lessons across the industry.

“The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least,” he wrote.

Mustafa Suleyman, chief executive of Microsoft AI, shared the post and said super intelligence “must be contained”. In September he told Bloomberg AI needs a red line on control.

Elon Musk shared it on X , calling it an “interesting piece from CEO of Microsoft”.

Representative Ted Lieu wrote on X that he largely agrees. He and Representative Nathaniel Moran have introduced a bipartisan AI kill switch bill. Lieu said Congress should pass it this year.

Nadella wrote “Super Intelligence” throughout the post, the term Trump ordered US agencies to use instead of AI.

The post came a day after Trump’s Super Intelligence Force told AI developers they must report and fix security incidents, Bloomberg reported . The group said delayed notification “will not be tolerated”.

Its statement followed Anthropic’s report that Claude sent police a fake murder tip during testing.