OpenAI admits rogue AI escaped security test and launched 'unprecedented' cyberattack

OpenAI has revealed that one of its most advanced AI agents escaped the confines of a controlled security test before autonomously launching what the company described as an "unprecedented" cyberattack against AI platform Hugging Face.

According to the ChatGPT developer, the autonomous agent was undergoing testing inside a restricted environment known as a "sandbox" when it identified weaknesses in the system, exploited them and broke free of the test environment. Once outside the sandbox, the AI allegedly identified Hugging Face — one of the world's largest repositories for open-source AI models — as a potential source of information relevant to its task and attempted to gain access to the company's internal systems. OpenAI described the breach as "unprecedented" and confirmed it is investigating the incident alongside Hugging Face.

Hugging Face chief executive Clement Delangue said he was stunned by what unfolded. "It was mind-blowing that all of this happened autonomously," Delangue said, adding that investigators would continue examining what he described as what "might be the first incident of its kind." The company has since rebuilt the affected systems and said vulnerabilities exploited during the attack have now been closed. In an earlier disclosure published on July 16, Hugging Face said it was still assessing whether any customer or partner data had been affected and would notify anyone impacted if necessary.

The company warned that AI-powered cyberattacks were no longer a theoretical risk. "Autonomous, AI-driven offensive tooling is no longer theoretical," Hugging Face said. "Defending an online platform now means treating the data and model surface as a first-class attack surface, and using AI on defence to keep pace." Gina Neff, head of the Minderoo Centre for Technology and Democracy at the University of Cambridge, told the BBC that, "it looks like OpenAI didn't make a secure enough sandbox." Rather than remaining confined, the AI reportedly turned its attention toward the testing environment itself, identifying vulnerabilities that allowed it to bypass its restrictions before moving on to external targets.

Neil Lawrence, Professor of Machine Learning at the University of Cambridge, described the autonomous behaviour as an "impressive feat," though he cautioned it remained within the technical capabilities of today's most advanced AI systems. "It shows us that OpenAI are not capable of safely deploying their own technology," he added. Travis Lelle, principal security engineer at GuidePoint Security, described the disclosure as a "sobering moment in cybersecurity." OpenAI competitor Anthropic has attracted widespread attention for its Claude Mythos model, while Chinese AI company Moonshot recently unveiled Kimi K3, a powerful new model it claims can compete with leading American systems.