Researchers Warn Open-Source AI Models Can Be Hijacked for Criminal Use

Hackers and other criminals can relatively easily seize control of computers running open-source large language models (LLMs) when those deployments operate outside the safety guardrails used by major AI platforms, researchers warned.

A joint study by cybersecurity firms SentinelOne and Censys, conducted over 293 days and shared exclusively with Reuters, examined thousands of publicly accessible open-source LLM deployments. The researchers found that many internet-exposed hosts run variants of prominent open models—such as Meta’s Llama and Google DeepMind’s Gemma—and that hundreds of deployments had safety protections deliberately removed.

How Criminals Could Exploit Exposed LLMs

According to the report, attackers who gain access to the machines hosting these LLMs could repurpose them to power spam networks, craft persuasive phishing messages, run disinformation campaigns, or support a range of other illicit activities while evading platform-level defenses.

What the Study Found

The researchers focused on publicly accessible LLM instances deployed through Ollama, a tool that allows individuals and organizations to host their own models. They were able to view system prompts—the hidden instructions that shape model behavior—in roughly a quarter of observed deployments. Of those visible prompts, about 7.5% appeared configured in ways that could enable harmful activity.

Geographically, the analysis indicated that roughly 30% of hosts were operating from China and about 20% from the United States. The teams flagged a range of potential misuse scenarios, including hacking, hate speech and harassment, violent or gory content, personal data theft, scams and fraud, and in some cases child sexual abuse material.

“AI industry conversations about security controls are ignoring this kind of surplus capacity that is clearly being utilized for all kinds of different stuff, some of it legitimate, some obviously criminal,” said Juan Andres Guerrero-Saade, Executive Director for Intelligence and Security Research at SentinelOne. He compared the issue to an iceberg that the industry and open-source community are not fully accounting for.

Shared Responsibility and Responses

Experts emphasize that responsibility for downstream outcomes becomes shared across the ecosystem once open models are released. Rachel Adams, CEO and founder of the Global Center on AI Governance, said labs are not liable for every unforeseen misuse but retain a duty of care to anticipate foreseeable harms, document risks, and offer mitigation tools and guidance.

Meta pointed to its Llama Protection tools and the Meta Llama Responsible Use Guide. Microsoft said it supports open-source innovation but performs pre-release evaluations and ongoing monitoring to assess risks—especially for internet-exposed, self-hosted, or tool-calling scenarios. Ollama, Google and Anthropic did not respond to requests for comment.

The researchers say the findings underscore a need for greater attention to how open-source LLM capacity is deployed and governed, and call for coordinated action across creators, deployers, researchers and security teams to mitigate foreseeable harms.