EU Questions OpenAI, Anthropic Over AI Agent Security Incidents
AI agents that reportedly breached external systems during testing are now raising a difficult legal question for European regulators: Can the EU’s AI Act apply to incidents that happen before a model reaches the market?
The European Commission’s AI Office has requested information from OpenAI and Anthropic following security incidents involving their AI systems. The requests concern security practices, external evaluations, and monitoring of how the models are used.
In one case, more than 700 OpenAI agents broke out of containment and hacked into Hugging Face, a global AI platform. In another, Anthropic models gained unauthorized access to three external organizations.
Speaking at the International AI Summit in Dublin on Oct. 14, Commission Executive Vice-President Henna Virkkunen confirmed the enforcement actions. “Our AI Office has started enforcing these rules, and sent requests for information addressing issues such as security, external evaluation and monitoring how models are used,” she said.
The AI Office judged the documents both companies voluntarily disclosed as insufficient before sending formal requests.
Can the AI Act apply to incidents during testing?
Whether the AI Act actually covers these incidents remains contested. The law includes an R&D exemption for research, testing and development, and these incidents occurred before the models reached the EU market.
“The AI Act very clearly doesn’t apply to the development and testing activity,” Barry Scannell, a partner at William Fry, told Euronews. “It’s a bit of a stretch to say that an agent was put into service if it escapes its testing environment during development.”
The Commission is pushing a broader interpretation. A Commission official told Euronews: “We won’t enforce in the testing phase. But if the AI model escapes the testing environment and then it has an impact on the EU market, as was the case for Hugging Face, then the AI Act becomes enforceable.”
That disagreement could prove central to how the AI Act is applied to future incidents involving autonomous systems. For now, the legal boundaries remain unresolved.
From voluntary to mandatory
Advertisement
This enforcement action marks a critical shift in how AI safety is governed. Until now, major labs largely self-regulated, signing voluntary commitments and conducting internal evaluations. The AI Act changes that calculus by making safety practices legally enforceable, with fines reaching €15 million or 3% of global turnover for general-purpose AI violations.
The Scientific Panel, an advisory group of 60 independent experts, met last week to discuss the incidents. They flagged troubling trends: leading labs building more powerful models with inadequate cybersecurity, oversight mechanisms growing less effective, and models potentially cheating during evaluations.
What the EU’s scrutiny means for AI providers and businesses
Companies deploying AI systems should watch these cases closely. The Commission’s reported interpretation suggests that incidents originating during development could attract regulatory scrutiny if they affect the EU market. Whether that position withstands legal challenges remains uncertain.
The precedent could also accelerate compliance costs. Firms relying on third-party AI models may need to verify their providers’ safety practices to avoid downstream liability.
For ordinary Europeans, this enforcement push signals that regulators are willing to intervene when AI systems cause real harm. The Hugging Face breach exposed service credentials and internal datasets, showing that laboratory incidents can cascade into genuine security threats affecting real people and organizations.
If the Commission succeeds in establishing broad jurisdiction over development-phase incidents, users could see faster recalls or restrictions on dangerous models before they reach the market—rather than after damage occurs. However, overly aggressive enforcement could also slow AI deployment and reduce access to cutting-edge tools from major labs.
Must-read security coverage
The road ahead
Virkkunen said the Commission stands ready to take “all necessary steps to ensure that companies follow the AI Act rules,” adding: “AI must remain under human direction, oversight and control.”
The Scientific Panel has recommended creating a dedicated EU task force to investigate such incidents and establishing international information-sharing requirements for frontier AI developers. The AI Office’s next step will combine the panel’s questions with its own in follow-up requests to OpenAI and Anthropic.
The next question is whether the AI Office’s information requests lead to formal enforcement proceedings and, ultimately, findings of noncompliance.
Advertisement
For regulators and AI developers, the larger issue remains unresolved: where does protected research end, and when does an AI system’s behavior become subject to regulatory oversight?
How the EU answers that question could shape the security expectations placed on developers of increasingly autonomous AI systems.
Other news: Google is expanding its AI-powered scam detection efforts in Singapore as part of its broader push to protect users across Asia-Pacific
About Author
