California Revives 'Kill Switch' Debate: Newsom Orders Expert Panel on AI Safety

California has reignited one of the most contentious debates in artificial intelligence policy by directing the creation of an expert advisory panel to recommend stronger AI safety measures — including the possible use of an emergency "kill switch" to disable dangerous systems.

In a reversal of his 2024 position, Gov. Gavin Newsom ordered the panel to consult "a group of world-leading experts" as the state weighs tighter rules, the Los Angeles Times reported. The advisory group is expected to evaluate proposals such as independent third-party safety planning, mandatory audits, formal emergency response plans, and technical tools that could allow companies or regulators to power down systems in an emergency.

A similar concept reached the governor last year in a bill from State Senator Scott Wiener (D–San Francisco). The measure, which passed the Legislature but was vetoed by Newsom, would have required companies to retain the ability to deactivate AI models they control should those systems cause harm. At the time, Newsom said the requirement risked slowing innovation and might not capture smaller models that could still be dangerous.

Since the veto, concern about the risks posed by rapidly improving AI has grown among researchers, industry insiders and the public. Some former employees of major AI firms have voiced alarm: Jacob Coxon, who left Anthropic over safety concerns, publicly warned that some builders believe advanced systems could present existential risk.

"People building AI earnestly believe that it could kill us all by the end of the decade," Jacob Coxon wrote in a public post.

Because many leading AI companies are headquartered in California, state rules could influence global industry practices and set precedents for other jurisdictions. Chris Cabaldon, chair of the state legislature's committee on technology, said he plans hearings on AI and expects the panel's work and recent warnings to be central to those discussions.

Lawmakers and regulators increasingly favor concrete oversight approaches — such as audits, documented emergency procedures and shutdown mechanisms — to reduce the chance that powerful AI systems cause harm. Advocates argue a state-backed framework could provide enforceable standards rather than relying solely on voluntary company commitments or industry self-regulation.

What's Next

The panel's recommendations could inform new legislation or regulatory action in California that, given the state's tech footprint, might ripple across the global AI landscape. Expect hearings, stakeholder consultations, and debate over balancing safety requirements with innovation incentives.