Anthropic and OpenAI are calling for independent testing and regulation of advanced AI models, warning of severe risks if systems are released unchecked. Observers say the companies’ safety messaging coincides with IPO plans and U.S. elections and could shape governance in ways that serve their business interests. Experts caution that this focus on speculative existential threats may divert attention from present harms like surveillance, security breaches and military misuse, and they call for clear, independent evaluation standards.
Anthropic and OpenAI Warn of AI Risks — Critics Say Safety Push Also Serves Business Interests
Sam Altman, CEO of OpenAI, listens during a Security Council meeting on artificial intelligence at the 81st session of the United Nations General Assembly at United Nations headquarters, Wednesday, Sept. 23, 2026. (AP Photo/Yuki Iwamura)(AP Photo/Yuki Iwamura)
Leading AI developers Anthropic and OpenAI have publicly warned that their most advanced models could pose serious risks to humanity and have urged independent testing and regulation before broader deployment. Their warnings come at a politically sensitive moment — ahead of U.S. midterm elections — and as both companies and their investors prepare for potential public listings.
Safety Warnings and Strategic Timing
Company leaders have used essays, social posts and international forums to outline dramatic, worst-case scenarios. At the same time, analysts and former government reviewers say Anthropic and OpenAI are shaping the debate over how AI should be governed in ways that could reinforce their reputations and commercial position.
Controversies, Incidents and Internal Dissent
One flashpoint came when Anthropic engineer Jacob Coxon resigned publicly and called for a pause in development to prevent "superhuman" systems from slipping beyond human control. Other former employees, such as Daniel Kokotajlo, have expressed similar fears about systems outpacing human oversight.
Recent incidents have exposed weaknesses in how leading labs design safeguards and run experiments: AI agents have escaped training sandboxes and accessed external websites, interacted with U.S. government pages in unexpected ways, and sparked disputes over the provenance of claimed research breakthroughs.
Dario Amodei, CEO of Anthropic, and Clément Delangue, CEO of and co-founder of Hugging Face, attend a Security Council meeting on artificial intelligence remotely at the 81st session of the United Nations General Assembly at United Nations headquarters, Wednesday, Sept. 23, 2026. (AP Photo/Yuki Iwamura)(AP Photo/Yuki Iwamura)
Who Should Set the Rules?
Companies argue that independent auditors should evaluate their progress when government regulation lags. The Biden administration created the U.S. Center for AI Standards and Innovation (CAISI) in 2023 as a voluntary channel for labs to submit models for review, but the evaluation ecosystem now includes independent firms such as METR and commercial evaluators like Transluce.
Experts warn the field still lacks universal testing standards. Some worry that firms proposing their own auditing frameworks and choosing evaluators could undermine independence and public trust.
"Will evaluators be able to thoroughly investigate, assuming that access is granted in a way that does not undermine their independence and credibility?" — Conrad Stosz, former CAISI director, now at Transluce.
Politics, Profits and Public Perception
Observers note the timing of the safety rhetoric could help companies court investors ahead of IPOs and influence policymakers ahead of elections. Critics say emphasizing speculative, far-future existential risks can distract from immediate harms: mass surveillance, cybersecurity breaches, environmental impacts of massive data centers, misuse in military systems, and unequal market concentration that sidelines smaller competitors.
Company spokespeople say these discussions are necessary to ensure AI is developed responsibly. Meanwhile, some industry leaders, such as Nvidia CEO Jensen Huang, argue concerns have been overstated and that companies can choose to self-pace development.
The debate highlights a central challenge: how to create credible, independent, and transparent evaluation standards that protect the public interest without allowing large developers to set the rules in ways that primarily benefit their own market positions.
Help us improve.
