Newsom pitches an AI kill switch for California

Gavin Newsom has signed an executive order asking a working group to report within two months on strengthening California’s AI safety laws, including whether companies should build a kill switch for frontier models. He vetoed SB 1047, which required developers to be able to promptly enact a full shutdown, in September 2024.

Gavin Newsom signed an executive order on Friday telling a working group to report within two months on how to strengthen California’s AI safety laws, Bloomberg reported . Among the measures it should consider is making companies build a kill switch for frontier models.

SB 1047 would have obliged developers of covered models to be able to “ promptly enact a full shutdown “ , stopping operation and further training. Newsom vetoed it on 29 September 2024, arguing it regulated models by size and cost rather than by what they are used for. The bill was the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.

The order’s other idea is independent third parties writing safety plans for frontier companies. “ We’re not waiting to act, ” Newsom said. His office has not said who is in the working group.

Since 2 August the European Commission has been able to restrict, withdraw or recall a general-purpose model from the EU market, under a provision of the AI Act we covered this month . The power sits with the regulator rather than the developer.

The AI Act separately requires that a human overseeing a high-risk system can interrupt it or bring it to a halt through a stop button. That rule covers deployed systems rather than the models underneath them. Those obligations are not yet in application.

The gap on third parties looks similar. California’s SB 813 gives the state until January 2028 to certify organisations able to test frontier models, while Europe’s 60-member scientific panel was appointed on 1 June.

Neither side has settled who pays for the compute. On the only worked example so far, the tester’s bill was met by the company being tested.

Newsom has been on both sides of this. He encouraged the boom because it paid, and earlier this year promoted a partnership to put Anthropic’s Claude into state agencies. Anthropic and OpenAI are both based in California, and both would fall inside whatever his group recommends.