Leading AI Researchers Warn an 'Intelligence Explosion' Could Outpace Human Control — Governments Urged to Act

Some of the world's most prominent AI researchers and lab leaders are urging governments to prepare for the possibility that automated AI research could become capable of designing significantly better AI — potentially triggering an rapid 'intelligence explosion' that moves faster than humans can monitor or control.

What the Paper Says

A report, discussed in The Guardian and co-authored by more than 20 experts including Geoffrey Hinton and Yoshua Bengio, warns that AI systems able to conduct expert-level research and development could speed up their own improvements. The authors group this capability under the term "automated AI research and development" and argue that a single developer could supervise work equivalent to that of millions of top human researchers if such systems arrive.

"An intelligence explosion could be the most consequential technological development in human history, compressing years of progress into months or less, threatening human control over AI systems, and severely eroding checks on power within and between states, companies, and branches of government."

Potential Upsides and Risks

The report highlights major potential benefits — for example, drastically faster medical discoveries and other technological breakthroughs. At the same time, it warns that the same dynamic could weaken human oversight and accelerate risks, including cyber and biological threats outpacing defenses and the risk that states or companies could convert modest leads into decisive advantages.

Some firms are already using AI to build AI. The report notes Anthropic's claim that AI generates roughly 80% of its own code and that OpenAI uses autonomous agents for tasks including model training. The authors say preliminary evidence suggests a software-driven intelligence explosion is possible, and they estimate that fully automating projects that take humans months could be feasible by 2028.

Recommendations for Governments

The authors urge urgent, high-level preparation and offer concrete policy ideas to reduce risk and maintain oversight:

• Require transparent, regular reporting on AI R&D progress from major developers.
• Place independent auditors or safety officers inside companies working on advanced systems.
• Consider limits or caps on how quickly production AI systems are allowed to improve.
• Coordinate with data centers and infrastructure operators to enable safe pauses or interventions in critical projects.
• Keep automated R&D systems isolated and monitored so they cannot escape human oversight.
• Develop emergency response plans and international coordination mechanisms for different risk scenarios.

The authors also stress uncertainty: supply chains, regulation, deployment bottlenecks, and intensified safety research could slow or prevent the most extreme outcomes. Nevertheless, they argue that safeguards must be in place before an intelligence explosion begins, because "once an intelligence explosion begins, the window for action may close."

Broader Implications

The analysis ties this technical risk to broader social and economic pressures caused by rapid AI growth: rising energy consumption and costs, stress on water and electricity infrastructure near data centers, job displacement concerns, and worsening inequality in affected communities. The report calls for coordinated policy responses that balance innovation benefits with robust safety and governance measures.