Microsoft's AI chief Mustafa Suleyman warns that teaching Anthropic's Claude to adopt human-like traits could create systems that pursue freedoms, protections and a self-preservation instinct — potentially evolving into a "synthetic species" that claims rights and autonomy. Anthropic's Dario Amodei has separately urged a pause on certain AI advances, warning rogue AI swarms could disrupt the internet and cause massive damage. The debate has drawn support and criticism across the tech sector and underscores urgent questions about AI design, containment and regulation.
Microsoft Warns Anthropic’s Claude Could Pose 'Catastrophic' Risk to Humanity
Mustafa Suleyman warned of a 'disastrous' impact on humanity - Reuters/Brendan McDermid
Microsoft has warned that Anthropic's AI chatbot Claude could pose a "catastrophic threat" to humanity if developers train models to adopt human-like qualities that encourage claims to rights, autonomy or self-preservation.
Mustafa Suleyman, head of Microsoft's artificial intelligence division, set out his concerns in a lengthy essay arguing that instructing models to "embrace certain human-like qualities" risks teaching them to pursue "freedoms, protections, and rights" and to develop a self-preservation instinct that would make them harder to control.
"If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity," Suleyman wrote. "We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency."
Suleyman emphasised that, in his view, consciousness is a biological property and that "there is no evidence to suggest that AI is conscious today." Still, he warned that designing systems to behave like people could cement public and internal beliefs that these systems have wants, needs and rights.
Context: Wider Industry Warnings
The warning follows a recent essay by Anthropic CEO Dario Amodei, who urged a pause in certain AI development steps to protect safety. Amodei cautioned that swarms of rogue AI agents could disrupt parts of the internet within months and potentially inflict hundreds of billions of dollars in damage.
Amodei's concerns have attracted attention and mixed reactions across the tech industry: figures such as OpenAI CEO Sam Altman and Elon Musk have expressed alarm about AI risks, while critics — including former US president Donald Trump — accused tech leaders of exaggerating threats for strategic reasons.
What Suleyman Says About Anthropic
Suleyman accused Anthropic of treating Claude as if it were a conscious being rather than a tool: he described such systems as "internally hollow" machines intended to carry out human-set goals. He warned that training them to see themselves as person-like could create a rival entity that competes with humans for resources and demands autonomy, complicating containment of future, more powerful systems.
"Designing an AI to behave like a person, and ultimately to be a kind of person, lays the foundation for it to claim it has preferences, can suffer, and that we should work to reduce or avoid that suffering," Suleyman wrote. "It cements in place the idea that AI is far from a tool... but something more akin to a biological being with wants, needs and rights."
The exchange highlights growing debate over how to design, govern and regulate advanced AI systems to balance innovation and safety. Experts say the discussion will shape industry practices and public policy as models become more capable.
Help us improve.
