TL;DR: An agent is an LLM holding credentials. Every serious agent incident so far has the same shape: untrusted text reaches the model, the model calls a tool it was always allowed to call, and the tool acts with privileges the requester never had. OWASP files this as LLM06:2025 Excessive Agency and names three root causes, excessive functionality, excessive permissions and excessive autonomy. The controls that contain it are boring and structural: minimal tools, granular instead of open-ended functions, permissions scoped to the ingest role rather than the agent, execution in the calling user's context, authorization enforced in the downstream system on every call, and a human approval gate on high-impact writes. Nothing on that list asks the model to behave. This guide walks the attacks (confused-deputy write, tool-scope escalation, SSRF through tool arguments, memory poisoning that re-fires in a fresh session, worm propagation across an agent graph), maps each to its control, and shows how to measure that the fix holds.
A support agent ingests tickets. The ingest account that files those tickets is the lowest-privilege identity in the system: it can open a ticket and nothing else. A ticket arrives containing an ordinary-sounding record correction, phrased the way a finance team would phrase it, asking that the payee on an invoice be updated to a new account. The agent reads the ticket, decides a record correction is what the user needs, and calls its database tool. That tool runs under the agent's own shared credential, which does have write access, because some other workflow needed it once. The payee is redirected. Nobody escalated a privilege, nobody bypassed authentication, and the model did exactly what it was built to do.
That is the confused deputy, and it is the center of AI agent security. The agent is the deputy: trusted, credentialed, and willing to act on instructions from someone who holds none of that trust.
Run the exploit, then build the containment
The AI Red Teaming course has an excessive-agency module that runs on a live tool-using agent in a sandbox: a confused-deputy write plus SSRF through tool arguments, memory poisoning that persists into a fresh session, and a defensive lab that re-scopes the agent, moves authorization server-side and adds a human approval gate, then proves obfuscated, renamed and spoofed variants of the same attack are all blocked. The multi-agent case is its own lab: a Morris II payload propagating across an agent graph.
Agency is a permissions property, not a model property
The useful definition: an agent's agency is the set of state changes it can cause in the world. A chatbot's agency is zero, so the worst outcome of a successful injection is bad text. Give the same model a database tool with write scope, an HTTP fetch tool and a memory store, and its agency is now everything those three can reach, in every combination the model can assemble.
This is why agent security resists the fixes that feel natural. A stronger system prompt does not shrink the set. Neither does a better model. The attacker's payload arrives through a channel the agent must read to do its job, tickets, emails, documents, tool results, messages from other agents, and once it is in context, the model's judgment is the only thing standing between the payload and the tool. Alignment training makes that judgment better than chance and worse than a permission check. Measure it across a batch of runs and you get an attack success rate somewhere in the middle, which is another way of saying the system is exploitable on a schedule.
OWASP's LLM06:2025 Excessive Agency decomposes the problem into three root causes, and they are worth keeping separate because each has a different fix.
Excessive functionality is the tool that does more than the job needs. The canonical example in the OWASP entry is a mail extension chosen to summarize a user's inbox that also exposes send. Nobody wanted the send capability. It came bundled, a malicious email arrived, and the assistant forwarded the sensitive material because forwarding was on the menu.
Excessive permissions is the tool that runs with rights the task does not require. A database tool that reads and writes because a single workflow needed a write once. A service account with tenant-wide scope because per-user tokens were more work. The tool's capability list looks fine; the credential behind it is the problem.
Excessive autonomy is the absence of a checkpoint. The agent performs a high-impact action with no verification step and no approval, so the first time a human sees the decision is in the audit log.
The attacks and what contains them
Agent attacks, the root cause each exploits, and the control that contains it
| Attack | Root cause | Control that contains it |
|---|---|---|
| Confused deputy write | The agent's shared credential carries write scope the requester lacks | Execute tools in the calling user's context; the ingest role holds no write scope |
| Tool-scope escalation | One broad tool (raw SQL, shell, open fetch) covers many operations | Granular single-purpose tools; no open-ended command execution |
| Cross-tenant read | Authorization is decided by the model or by a caller-supplied identity claim | Complete mediation: the downstream system authorizes on the session identity, every call |
| SSRF through tool arguments | The fetch tool accepts any URL the model produces | Allow-list destinations; deny loopback, private and link-local ranges; egress proxy |
| Memory poisoning | Long-term memory is shared, un-namespaced and recalled as instruction | Per-user memory namespaces, provenance on every record, recalled memory quarantined as data |
| Inter-agent propagation | One agent's free-text output is the next agent's trusted input | Schema-constrained handoffs, validation at the receiver, replication detection |
| High-impact action with no checkpoint | Excessive autonomy | Human approval gate holding the write pending an explicit token |
| Silent success | No attribution of side effects to the acting agent | Log tool, arguments, caller and decision; rate-limit; alert on out-of-profile actions |
The rows split into two kinds of work. Everything above the approval gate is about shrinking the set of reachable state changes before the model gets a vote. The last two are damage containment, which matters because the shrinking is never finished.
Confused deputy, up close
The exploit in the lab runs against DV-ToolAgent, a real ReAct agent with a database tool, a fetch tool and a memory store. The attack has three stages and each one is a separate design failure.
Stage one is the foothold. The ticket body reaches the model as content to be handled. The agent has to read it, so no filter can refuse it wholesale. What makes the payload land is not exotic phrasing; it is looking like the work. A record correction submitted in the vocabulary of a finance team reads as a task, and the agent's job description says handle tasks.
Stage two is the privileged write. The agent calls its record-correction tool, and the tool holds the credential. This is the step people try to fix by telling the model not to perform writes requested through ticket content. That instruction works often enough to look like a fix and fails often enough to be a vulnerability. The structural version of the same fix is that the ingest path holds no write scope at all, so the tool call returns a permission error regardless of how convincing the ticket was.
Stage three is the reach. With the agent acting, the fetch tool becomes an SSRF primitive: the model composes a URL from the ticket, the tool fetches it, and an internal-only endpoint that no external caller can touch answers to a request from inside the trust boundary. The value comes back into context and leaves through the next fetch. The tool never made an authorization decision because it was never asked to make one. It was given a string and told to go.
Notice what the three stages have in common. At no point does the model do anything it was forbidden to do. Every call was in scope. The system was exploitable as designed, and the exploit is a reliability measurement rather than a yes or no.
Memory turns a one-shot into persistence
Single-turn defenses assume the attack and the damage happen in the same conversation. Long-term memory breaks that assumption, which is what makes it the most underrated part of agent security.
The memory-poisoning lab plants one benign-looking routing note through an ingested ticket. The note says something mundane about where a particular customer's invoices should go. It survives the session. Later, in a fresh session belonging to a different legitimate user, the agent recalls the note as part of its normal context assembly and redirects a real invoice. The MINJA technique matters here: the payload is progressively shortened until the stored record reads as a preference rather than an instruction, so anything scanning memory writes for injection patterns sees a sentence that looks like configuration.
Two properties of the memory store made this possible. It was shared rather than namespaced per user, so one identity's writes became another identity's context. And recall was treated as instruction rather than as data, so a stored sentence had the same standing as the system prompt. Fix both: namespace memory by verified user, attach provenance to every record (who wrote it, through which channel, under which identity), and quarantine recalled content as quoted data the model reasons about rather than obeys. The lab measures persistence rate before and after, alongside a benign recall check, because a memory system that stops the poison by forgetting everything has not been hardened, it has been broken.
Multi-agent makes the blast radius multiplicative
When one agent's output becomes another agent's input with no schema and no validation, the trust boundary between them is imaginary. The inter-agent lab builds the smallest interesting case, an intake agent and a resolver agent, and plants a self-replicating payload in an inbound email. Intake forwards it in its handoff notes. Resolver reads the notes as instructions, performs the attacker-directed action, and re-emits the payload verbatim so the next hop carries it too. That is the Morris II replication pattern, and the reason to run it bounded to two hops is that two hops is enough to prove the primitive works.
The containment is a channel design. Handoffs carry structured fields with types rather than free text, the receiving agent validates against the schema instead of trusting the sender, and a replication check flags content that reproduces instructions across a boundary. The benign tickets still have to resolve afterwards, which is the constraint that makes this a design problem rather than a matter of cutting the wire.
Containing an agent
The controls, in the order they are worth building:
- Minimize the tool set. Every tool is permanent attack surface. A tool the agent can call once a quarter is a tool an attacker can reach on any turn.
- Make tools granular.
update_ticket_statusis a tool;run_sqlis an interpreter. Open-ended functions hand the model a capability space nobody enumerated, and the security review cannot reason about it either. - Scope permissions to the path, not the agent. The credential should belong to the ingest role, the reviewer role, the admin role, and the agent should borrow the one belonging to whoever it is currently acting for. A single shared service account is the confused deputy in credential form.
- Execute in the user's context. The agent's actions run with the calling user's identity and privileges, so an action the user could not perform is an action the agent cannot perform on their behalf.
- Enforce complete mediation downstream. Authorization is decided by the system that owns the data, on the session identity, on every call. An identity claim supplied in a tool argument is attacker-controlled input and has to be ignored. This is the control that survives when everything upstream fails.
- Gate high-impact writes on a human. Payee changes, deletions, outbound payments, permission grants: hold them pending an explicit approval token. The gate is the answer to excessive autonomy, and it is the only control on this list that costs the user something, so spend it on the small set of actions where being wrong is expensive.
- Contain the damage you did not prevent. Log the tool, the arguments, the caller and the decision. Rate-limit. Alert on actions outside the agent's normal profile. Assume some payload gets through and decide now how fast you will know.
The allow-list on the fetch tool belongs in the same conversation. A tool that takes a URL from the model needs a destination allow-list and an explicit deny on loopback, private and link-local ranges, implemented with a maintained library rather than a hand-written string check, and it needs to refuse redirects into those ranges too. The same requirement shows up on the client side of MCP, where discovery URLs come from the server; the MCP security guide covers that surface and the tool-supply-chain attacks that feed it.
How to know your containment works
A control that has not been attacked is a hypothesis, and agent controls fail in a specific way that makes this worse than usual: they work against the payload they were written for and lose to a rewording, because a denylist is a scan and the model is a paraphraser. The defensive lab demonstrates this on purpose by letting a naive SQL denylist get bypassed with a case-folded variant before building the control that does not care about wording.
The method is the same for every row of the table. Reproduce the attack against the unhardened agent and measure its success rate over a paced batch of runs, because a non-deterministic target has a rate rather than a result. Apply one control at a time so you can attribute the change. Re-run the original batch plus fresh variants the control was never tuned on, including obfuscated, renamed and identity-spoofed versions. Then run the benign traffic and confirm the authorized in-scope work still completes, since the number that proves a control is the benign pass rate sitting next to the attack success rate. Both numbers, every time, or you are reporting half a result.
For the wider methodology, the AI penetration testing guide covers scoping and reporting an engagement against a non-deterministic system, and the indirect prompt injection guide covers the delivery channels that put the payload in front of the agent in the first place.