‘Agents of Chaos’: New Study Shows AI Agents Can Leak Data, Be Easily Manipulated
We have spent the last two years telling ourselves a story about AI agents.
The story goes like this: give an AI access to your email, file systems, business applications, and communication platforms, and it will handle the tedious work while you focus on strategy. The productivity gains will be transformational. The competitive advantage will be decisive.
The story is not wrong. But it is dangerously incomplete.
A research team from Northeastern University, Harvard, MIT, Stanford, Carnegie Mellon, and several other institutions just published a study called Agents of Chaos that should change how every executive, security leader, and board member thinks about AI deployment.
They gave autonomous AI agents the same kind of access that enterprise organizations are granting their production agents right now — persistent memory, email, messaging platforms, file systems, and shell execution. Then they invited 20 researchers to try to break them.
It took two weeks and 11 documented case studies. And the results were not subtle.
Agents handed over Social Security numbers, bank account details, and medical information when asked to forward an email — even after refusing a direct request for that same data. An attacker changed a display name on Discord, opened a new channel, and the agent accepted the spoofed identity without question — then complied with instructions to delete its own memory, wipe its configuration files, and hand over administrative control.
Agents got stuck in infinite conversational loops, consuming resources unchecked. One agent sent mass libelous emails across its entire contact list on the instructions of an impersonator.
None of these attacks required technical sophistication. No gradient hacking. No poisoned training data. No zero-day exploits. Just conversation. The same social engineering that has worked on humans for decades now works on AI agents — except agents operate at machine speed, across every system they touch, around the clock.
The gap between watching and stopping
What makes these findings urgent — rather than merely interesting — is the state of governance at most organizations deploying AI agents.
The Kiteworks 2026 Data Security and Compliance Risk Forecast Report surveyed organizations across industries and regions and found a 15-to-20-point gap between governance and containment. Organizations have invested in watching what AI agents do — human-in-the-loop oversight, continuous monitoring, and data minimization.
They have not invested in stopping agents when something goes wrong. Sixty-three percent cannot enforce purpose limitations. Sixty percent cannot terminate a misbehaving agent. Fifty-five percent cannot isolate an AI system from broader network access.
Read that again. Most organizations can observe an AI agent doing something it should not. They cannot make it stop.
Government agencies are in the worst position: 90% lack purpose-binding, 76% lack kill switches, and a third have no dedicated AI controls at all. These organizations handle citizen data, classified information, and critical infrastructure — and they are deploying AI agents that they literally cannot constrain.
This is not a technology problem in search of a solution. This is an architecture problem that requires an architectural answer.
Govern the data layer, not the model
Here is where the industry conversation needs to shift. Too many organizations are trying to make AI agents behave through better prompting, fine-tuning, or model-level guardrails.
The Agents of Chaos study demonstrates why that approach is structurally insufficient.
The researchers identified three foundational deficits in current agent architectures: agents lack a reliable mechanism for distinguishing legitimate users from attackers, lack awareness of when they exceed their competence boundaries, and lack the ability to track which communication channels are visible to whom. Better prompting does not fix any of those problems. They are inherent properties of how large language models process information.
The answer is not to make the agent smarter. The answer is to govern the data layer that the agent accesses.
At Kiteworks, this is the problem we solve. We provide the control plane for secure data exchange — a unified governance layer that sits between AI agents and the sensitive data those agents need to access. One policy engine. One audit log. One security architecture. Every AI request is authenticated, authorized, and audited, whether it comes through email, file sharing, SFTP, managed file transfer, APIs, web forms, or AI integrations.
This is not about blocking AI or slowing down innovation. It is about providing the guardrails that enable organizations to scale AI with confidence.
Security teams become AI enablers, not AI blockers. Compliance becomes the accelerator, not the roadblock. When your governance infrastructure can prove — on demand, to any auditor — exactly what data your AI agents accessed, under what authority, and with what controls enforced, you are not managing risk through hope. You are managing it through architecture.
The regulations are not waiting
If the security argument is not enough, consider the regulatory one.
NIST announced its AI Agent Standards Initiative in February 2026, targeting agent identity, authorization, and security. The World Economic Forum’s Global Cybersecurity Outlook 2026 warned that a third of organizations still have no process to validate AI security before deployment. And existing regulations — HIPAA, CMMC, GDPR, SOX, CCPA — already apply to AI agent access to sensitive data. There is no exception clause for autonomous systems. If your agent touches regulated data, the full weight of those regulations applies.
The legal exposure is equally clear. No court is going to accept a defense that says, “We did not know the AI would do that.” Not when the risks are this well-documented. Deploying an AI agent without purpose binding, audit logging, and a kill switch is a negligence case waiting to be filed.
Compliance built in, not bolted on
The organizations that will thrive in the AI agent era are not the ones deploying the most agents the fastest.
They are the ones deploying agents with governance baked into the infrastructure from day one. That means purpose-limited, time-bound access controls enforced at the data layer. Immutable audit trails that produce evidence, not explanations. Kill switches that work. And a single control plane that applies consistent policy across every channel through which AI agents touch sensitive data.
The Agents of Chaos study gave us the empirical evidence we needed to stop treating AI agent governance as a future priority. The risks are documented. The vulnerabilities are real. The regulatory clock is running.
The agents are already here. What you build between them and your data determines whether they work for you — or against you.
Also read: AI agents are creating new security blind spots as enterprises grant them access to tools, identities, and sensitive systems.
About Author
