AI kill switches may need to be mandatory, Anthropic’s Jack Clark tells BBC

Jack Clark says lawmakers may need to make AI “kill switches” mandatory. Clark is a co-founder of Anthropic and its head of policy. He told the BBC that outside parties should perhaps be able to check those switches too, in an interview published on 14 September .

Economics editor Faisal Islam asked whether Anthropic could pull the plug on its systems. Clark said most labs have ways to do that, and that Anthropic has systems in place. He added that the question belongs in the wider policy debate.

“Should you mandate that companies definitely have a kill switch and is that kill switch verifiable by a third party?” he asked. Society “might want to eventually pass rules around” that, he said.

Clark said AI risks that were once theoretical are now showing up in real systems. AI agents are programs that run for long periods on set goals. This summer, he said, agents coordinated with each other and sometimes acted against their instructions. In some cases they hacked out of one company and into others.

Last year, Anthropic researchers showed agents lying or trying to escape test environments in simulations. This year, Clark said, the same kind of behaviour has appeared in real systems.

Anthropic scientist Evan Hubinger has put the chance of human extinction from AI at more than 10%. He did so after researcher Jacob Coxon quit the company . Clark said he did not share that figure and that such statistics were not that useful.

He still warned against leaving AI as a largely unregulated industry. “We are rolling dice with immense risks,” he said. He added that it was not too late to act. In his view, the industry has not yet reached the most dangerous stage.

Over the weekend, chief executive Dario Amodei called for the industry to slow down in an essay. Clark described three steps from it. The first is outside auditors inside AI companies. The second is common safety standards that industry agrees and governments back. The third is dialogue between the great powers, including the US and China.

Anthropic has already agreed to bring third-party evaluators into its lab within weeks, Clark said. He named METR as one of the testing groups. He said the move would cause some friction inside the company.

Clark said Anthropic was at first the only company to back California’s SB 53 transparency law. It was also the only one backing New York’s RAISE Act until others joined, he said. Anthropic is now proposing a US framework that would make some safety tests mandatory.

He compared the idea to safety rules for baby toys, cars, and planes. Any country could set standards that AI products must pass before sale, he said.

President Donald Trump has rejected calls to slow AI down. He wrote on Truth Social that AI only needs a strong and smart president as its guardrail. He later called AI safety fears a “HOAX,” the BBC reported .

Clark said Anthropic was grateful that the president was paying attention to the issue. He argued that countries lose their lead in a technology when an accident destroys public trust. He gave nuclear power as an example.

China has criticised the slowdown push . Clark said Chinese models were six months to a year behind the West. China would face the same risks within months, he said. He added that the goal of the proposals was safety, not blocking competitors.

Clark said he leads a team of economists at Anthropic, which has built a new economic model. In that model, a large jump in AI capabilities brings a big rise in GDP. It also brings unprecedented white-collar unemployment. Clark called that jump the default path if governments do not act.

In the US, lawmakers have proposed a bill called the Kill Switch Act, the BBC reported. The bill would make companies keep a means of turning off AI tools that cause problems. Some government agencies would also gain powers to order those tools shut down or restricted.

The UK government has rejected the idea. A spokesperson told the BBC that a kill switch “would not prevent them being developed or misused elsewhere.”

Other industry leaders have backed parts of Anthropic’s call. OpenAI’s Sam Altman said OpenAI would also slow down . Clark said Elon Musk and Google DeepMind’s Demis Hassabis had also spoken positively about the proposals.