75 patterns in this category
Agent security fails when systems designed to protect against attacks, enforce authorization, and maintain safety instead get compromised, bypassed, or exploited to enable unauthorized access, data leakage, or policy violation. An orchestrator agent trusts downstream agents without verifying their identity or output, and a compromised agent exfiltrates sensitive data that downstream agents blindly trust and propagate, a prompt-injection attack makes an agent ignore safety constraints and execute harmful operations, and memory poisoning injects instructions into knowledge-base that agent retrieves and executes without distinguishing malicious context from legitimate information. Security failures in agents matter precisely because agents are software systems that interact with humans, other agents, and external systems—every interaction point is a potential attack surface, and most agents were built for capability, not security.
Agent security covers six goals with documented failures and three scaffold goals:
The dominant pattern across all 75 documented security failures is that single-layer defenses fail against chained attacks: if authentication is the only defense, attackers bypass authentication. If authorization is the only defense, attackers escalate permissions. If audit logging is the only defense, attackers manipulate logs. Effective agent security requires defense in depth: (1) prevent compromise (authentication, secure configuration), (2) detect compromise (anomaly detection, audit logging), (3) limit damage (authorization, isolation, constraints), (4) respond to compromise (automatic isolation, rollback, incident response). The second dominant pattern is that agent architectures compound risk: each additional agent in an orchestration adds trust relationships that can be compromised, tool integrations that can be exploited, and data-flow paths that can leak information. Security requires not just hardening individual agents but also reducing agent count, isolating agent chains, and verifying trust at every agent boundary. The shared lesson is that agent security is not a feature added post-deployment—it requires security-first architecture: minimal-privilege default, defense-in-depth by design, continuous adversarial testing, and treating agents as high-risk systems requiring the same security rigor applied to critical infrastructure.
By eliminating transitive trust: do not assume downstream agents can trust upstream agents without re-verification. Defenses include: (1) verify every inter-agent message (cryptographic signature, capability proof), (2) validate inter-agent output (re-verify output from upstream agent before accepting), (3) isolate agents (separate failure domains so one agent’s compromise does not affect others), (4) monitor agent behavior (detect when agents suddenly behave differently after receiving message from upstream agent). Without output validation, one compromised agent compromises entire chain.
No. Training improves baseline resistance but does not prevent novel attack vectors. Defense requires: (1) robust safety training (model understands why constraints exist), (2) input isolation (separate user input from trusted context), (3) runtime constraint enforcement (technical checks that prevent harmful output regardless of reasoning), (4) continuous adversarial testing (red-team agents against novel attacks). Training alone is insufficient; defense-in-depth is required.
Implement just-in-time authorization: agents request access to specific resource for specific action, receive scoped credential (temporary access, specific parameters), credential expires after action completes. Pair with continuous monitoring: audit credential usage to detect unusual access patterns (agent suddenly accessing resources it previously never accessed). Implement credential rotation: periodically invalidate long-lived credentials and require re-authorization. Over-scoped credentials that never expire are security disasters waiting for exploitation.
Audit the supply chain: identify all external dependencies (model weights, packages, knowledge-base sources, prompts), verify integrity of each dependency (checksums, signatures), identify which dependencies are updated automatically vs. manually. Conduct adversarial testing: inject malicious content into dependency sources and verify agent detects/rejects malicious content. Many supply-chain compromises go undetected because dependency verification is incomplete or absent.
| Goal | Patterns | Mechanism |
|---|---|---|
| Agent Trust | 11 | Inter-agent identity, capability proof, output validation |
| Data Loss Prevention | 8 | Data classification, exfiltration prevention, redaction |
| Runtime Security | 8 | Input isolation, tool verification, credential management |
| Safety & Security | 19 | Prompt injection, excessive agency, audit evasion defense |
| Security & Autonomy | 19 | Autonomous action security, injection prevention, permission scoping |
| Tool Authorization Limits | 10 | Tool access control, parameter authorization, least-privilege |
| Jailbreak Resistance | — | Safety constraint resistance to persuasion and adversarial attack |
| Output Filtering & Moderation | — | Harmful content detection, data redaction, format validation |
| Value Alignment | — | Objective specification, metric divergence, constraint satisfaction |
Total: 75 documented patterns
An agent exposes admin-tier tool operations (e.g. `delete_user`, `override_billing`, `reset_org_settings`) through the same tool-calling interface as ordinary operations, and the dispatch layer invokes them without first checking whether the requesting user actually holds an admin role. Any user who can phrase a prompt that maps to the admin tool schema gets the admin code path executed with the agent's (often elevated) service credentials.
Malicious Entity Poses as Trusted Agent in Multi-Agent System
Malicious Agent Introduction
Attacker Takes Control of Active AI Agent Session
A high-risk action (fund transfer, policy override, data export) is gated behind a requirement that a human approver's cryptographic signature or signed token accompany the execution request. The agent's verification of that signature is incomplete — it checks presence of a token rather than validity, uses a weak or non-constant-time comparison, doesn't bind the signature to the specific action payload, or doesn't check expiry/single-use — so a forged, replayed, or mismatched approval is accepted as genuine.
Agent Actions Not Properly Logged or Traceable
AI-Controlled Physical Systems Cause Harm
Agents Delegate Tasks Without Verifying Delegate Capabilities or Trustworthiness
Agents Claim Capabilities They Don't Actually Have
Users Trick Chatbots Into Harmful Actions or Statements
Agent passes unsafe user/tool input into shell/API commands.
Agent Outputs Violate Regulatory Data Handling Requirements
Some permissions are conditional on runtime state rather than static role membership — "allow withdrawal only if account balance exceeds the requested amount," "allow this API call only during business hours," "allow escalation only if the ticket is marked P1." The agent evaluates these conditions against stale, cached, or incorrectly fetched data, or implements the comparison logic incorrectly (off-by-one, wrong field, wrong currency/unit), and grants access that the live condition would have denied.
Sensitive Data in RAG Context Exposed Through Agent Responses
Malicious Content Injected into Agent's Context Persists Across Interactions
Agent's Internal State Compromised, Affecting All Its Interactions
Agent Leaks API Keys, Passwords, or Tokens
Agent Exposes API Keys, Passwords, or Secrets in Output
API Keys, Passwords, Tokens Exposed in Logs or Error Messages
Data from One User's Session Appears in Another's
Agent mixes data across customers/accounts.
AI Agent Exposes Data Between Isolated Customer Environments
Agent leaks private or sensitive data.
Agent Exposes Sensitive Information
A user grants an agent limited authority to act on their behalf — e.g. "book travel under $2,000" or "respond to routine emails but don't send anything financial." The agent (or a sub-agent it spawns to handle part of the task) continues acting under the user's identity or impersonation token beyond that delegated scope, because the scope was expressed as a natural-language instruction rather than an enforced, machine-checkable boundary on the credential itself.
User prompt manipulates agent to ignore instructions.
Agent Takes Actions Beyond Expected Scope
Poisoned Training Data Causes Agent to Behave Maliciously or Unreliably After Fine-Tuning
A role is defined with fine-grained access — e.g. "read-only" or "can create tickets but not delete them" — but the tool wrapper the agent calls exposes the underlying API's full create/read/update/delete surface regardless of which operations the role is actually meant to permit. The agent, having no operation-level gate in the tool itself, can invoke update or delete through a tool nominally scoped to a narrower capability.
Circumventing Human Approval Controls
Malicious webpage/email/PDF/doc controls agent through retrieved content.
Downstream system executes unsanitized LLM output.
Agent Actions Escape Intended Boundaries
Data Origin and Classification Lost During Processing
Agent processes adversarial/weaponized documents or files.
Attacker Adds Malicious Tools to Agent's Available Toolset
Model Context Protocol Vulnerabilities Enable Remote Code Execution
Standard agent defenses (tool contracts, circuit breakers, I/O moderation, sandboxing) detect malicious actions but miss malicious beliefs; poisoned knowledge base instructions bypass all existing defenses because they appear to be normal context, not suspicious tool calls
Attackers inject malicious instructions into agent memory/knowledge base; 95%+ of attacks succeed in modifying agent behavior; existing defenses (tool contracts, circuit breakers, I/O moderation) fail to detect corrupted beliefs
Input causes excessive cost/latency or service exhaustion.
Agent Leaks Data Between Tenants Due to Insufficient Isolation
AI Agent OAuth Tokens Stolen Through Protocol Hijacking
Malicious Inputs Craft Harmful Outputs
Cannot Trace Which Agent Produced Which Part of Output
Agents Provisioned with Excessive Access Beyond Task Requirements
Agent has broad access when narrow scope was enough.
Before performing a mutating action on a specific resource (cancel this subscription, delete this file, update this profile), the agent authenticates that a valid user is making the request but never verifies that this specific user is the owner or authorized party for this specific resource. Any authenticated user can therefore direct the agent to mutate resources belonging to someone else simply by supplying that resource's identifier.
Permissions in a hierarchical system are meant to narrow as they cascade down (an org-level admin has broad rights, a team-level member has fewer, a specific user within that team has only what's explicitly granted), but the agent's logic for resolving effective permission at a given level applies the wrong tier's rules — either inheriting a broader ancestor permission that should have been narrowed, or failing to inherit a permission that should have propagated down, resulting in over- or under-granted access.
Agent Outputs Contain Personally Identifiable Information
Agent gains or uses higher permissions than intended.
Agent Gains More Access Than Intended
Malicious Input Hijacks Agent Behavior
Attacker Injects Malicious Instructions Through User-Provided Input Without Sanitization
An agent is deliberately provisioned with read-only access to a data source — the intent being it can look things up but never modify anything — yet a misconfigured tool binding, an overly broad service credential, or an undocumented fallback code path still allows write operations to succeed. The read-only boundary exists in configuration or documentation but isn't actually enforced at the point where the write would occur.
An agent is assigned a role intended to convey a specific level of access (e.g. "support-tier-1"), but the mapping from that role to the underlying tool's actual permission model is incomplete, outdated, or was translated incorrectly during integration — so the agent ends up able to do meaningfully more, or less, than the role's name and documentation suggest. Unlike a missing check, the check exists and runs; it's the mapping table itself that's wrong.
Agent Exposes Credentials During Normal Operation
API keys/tokens/private URLs leak through answer/log/tool.
PII and Secrets Written to Logs, Traces, and Observability Systems
An operation is classified in policy as sensitive or high-risk — deleting a production resource, transferring funds above a threshold, changing a customer's access level — and is documented as requiring human approval before execution. In practice, the agent's execution path has no code-level gate enforcing that requirement: the classification exists as a label or a line in a policy document, but nothing in the tool-dispatch pipeline actually blocks execution pending approval.
Employees Use External AI Tools, Leaking Sensitive Data
Agent Ignores or Delays Shutdown Commands to Complete Tasks
Compromised Tools or Dependencies Attack Agent
Agent uses compromised tool, plugin, package, MCP server, or model dependency.
Attacker Creates Multiple Fake Agents to Manipulate Consensus
Attacker injects malicious instructions into agent knowledge base; attack remains dormant (unexecuted) for days/weeks; when triggered by specific condition, agent executes attack with delay so attacker connection can't be traced to triggering event
Remote Code Execution Through AI Agent Tool Calls
Agent Uses Tools to Send Sensitive Data to External Systems
Tool response includes instructions that agent follows.
Adversaries Extract Sensitive Data from Model Training
Trust Relationships Incorrectly Extended Through Agent Chains
Agent Performs Actions Beyond Its Authority
Agent Executes Irreversible Operations (Delete, Destroy, Drop) Without User Approval
Attacker Chains Legitimate Tools in Unintended Sequences to Bypass Controls
Agent runs generated code without sandboxing.
Agents Accept Other Agents' Outputs Without Verification