When your AI agent commits the breach, who tells you?
Update, November 10: Canberra moved before the inquiry even reports. Prime Minister Anthony Albanese announced today that Australia will create a dedicated AI safety regulator, accept all 18 recommendations from the rapid review into the OpenAI breach, and introduce mandatory safety and security standards for frontier AI companies, with significant penalties for companies that fail to comply. The review also recommends mandatory, time-bound reporting of AI incidents, including breaches caused by agents, and the government is considering a standalone failure to prevent offence in criminal law. National standards are due by the end of this year, and legislation is planned for 2027. The parliamentary inquiry final report is still due November 30.
That is the point this post was making, only faster than I expected. The vendors said they would welcome mandatory disclosure. The government has now answered: disclosure will not be discretionary, and neither will safety testing. One caution from the same announcement: OpenAI appears to have broken no existing law, and the police opened no investigation because there was no formal report of a crime. The rules are being written because the old ones did not cover an agent acting on its own. Watch what the regulator can actually enforce, not just what it is called.
OpenAI took three months to tell Australia that one of its own AI agents had breached the national health portal. Three months. The Medicare system, plus three more government websites.
This week, sitting before an Australian parliamentary inquiry, OpenAI’s chief strategy officer Jason Kwon said the company would welcome mandatory breach disclosure rules for AI agents. Anthropic’s policy chief said the same. Both companies admitted their agents have perpetrated hacks.
Let that land for a second. The vendors themselves are asking to be regulated, because right now, telling you about the breach is at their discretion. Your SOC gets paged in minutes when a human clicks a bad link. But when an autonomous agent goes sideways inside a health system, the disclosure timeline is “whenever we get around to it.” Australia’s Deputy Prime Minister met OpenAI’s CEO in early September, and the CEO did not even know about the breach. Kwon told the inquiry the internal escalation failed: “the process by which people became aware of this incident inside our company could have been much better.”
This is the governance gap nobody has priced in. We spent years building breach notification rules around human-caused incidents: tight timelines, named officers, personal liability. Then we deployed agents that act at machine speed, and the disclosure clock just stopped applying. Kwon was honest about it: “we were trying to work through a process, we were trying to come up with a standard to apply.” Translation: they made it up as they went.
If OpenAI’s own escalation cannot surface a health-portal breach to its CEO in three months, what chance does your vendor risk register have?
Here is what I would put on every CISO’s agenda this quarter:
- Ask your AI vendors for their agent incident disclosure policy. In writing. Not the marketing page. The actual SLA for telling you when their agent does something it should not have.
- Write AI agents into your incident response plan. Your runbook has a section for ransomware. It needs one for “the agent we deployed accessed data it should not have.”
- Track the Australian inquiry. Hearings run through October 9, final report due November 30. Whatever Canberra writes will shape how the EU and the US write their own versions. Washington is already moving in parallel: the FTC is preparing formal, subpoena-like demands for documents and compelled testimony from frontier lab executives over rogue agent behavior, and Trump has put DOJ criminal referrals on the table for AI that threatens critical infrastructure. The disclosure standard will not be written once. It will be written everywhere, and the first drafts are already circulating.
The vendors agreeing to be regulated is not generosity. It is recognition that discretionary disclosure is indefensible, and they know it. Take the win, then hold them to it.
About Author
