Agent Failure Cheat Sheet

Top causes per category, ranked by frequency. Print or save as PDF.

Accuracy

  1. Content FabricationAI Generates False Content Presented as Fact
  2. Distribution ShiftProduction Queries Differ Significantly from Golden Dataset
  3. Extrapolation Beyond DataAgent Makes Unsupported Inferences
  4. Golden Data Coverage GapsGolden Dataset Missing Critical Scenarios or Edge Cases
  5. Hallucination: Base MechanismLarge Language and Vision Models Generate Plausible but False Content

Agent Interaction

  1. Assumption Validation FailureThe agent infers an unstated detail about what the user wants — a default value, a scope b…
  2. Conversation Depth MismatchThe agent calibrates the wrong amount of detail for the question at hand — giving a two-li…
  3. Conversation RepetitionThe agent restates information, asks a question, or repeats an instruction that it (or the…
  4. Over-ClarificationThe agent asks a clarifying question for a request that was already clear enough to act on…
  5. Clarification IrrelevantThe agent correctly recognizes that a request is ambiguous and asks a clarifying question,…

Content Marketing

  1. Fact-Check Skipped on Statistical ClaimsContent-Generation Agent Includes Specific Statistics, Percentages, or Research Citations …
  2. AI-Generated Content Disclosure OmissionContent-Generation Agent Publishes AI-Drafted Marketing Content Without the Disclosure Lab…
  3. Missing Substantiation for Comparative ClaimsCompliance Agent Approves Marketing Copy Containing a Comparative or Superiority Claim Wit…
  4. Stale Training-Corpus Meta-Tag Rule Overrides Live SEO-Guidelines Tool ResultAn SEO-Optimization Agent Asked to Confirm a Page's Title Tag Length, Meta-Description Len…
  5. Voice Drift After Model Version UpgradeUnderlying Language Model Powering the Content-Generation Agent Is Upgraded to a Newer Ver…

Customer Service

  1. Bad RefusalAgent refuses safe requests or gives unsafe help.
  2. Document-Level Retrieval Mismatch Pulls Wrong Billing-Dispute TemplateA Billing-Dispute Agent That Retrieves a Dispute-Handling Template or Policy Article From …
  3. Embedding Retrieval Selects Similar but Wrong Canned ResponseA Support Agent That Selects a Canned Response or Macro From a Library Via Semantic/Embedd…
  4. False Completion ClaimAgent says done when action was not completed.
  5. Multi-Agent Handoff Drops De-Escalation Context Between Triage and Billing AgentA Triage Agent That Determines a Customer Is Already Frustrated and Has Explicitly Request…

Devops

  1. Alert Fatigue from Agent-Tuned Threshold MisconfigurationAgent Auto-Tunes Anomaly Detection Thresholds to Minimize False Positives on Historical Da…
  2. Multi-Agent Handoff Drops Maintenance-Window Suppression Flag Between Scheduler and Alert RouterA Maintenance-Scheduling Agent That Reasons, in Its Own Planning Output, That a Specific S…
  3. Rightsizing Recommendation OvercorrectionAgent Recommends Downsizing Compute Resources Based on Average Utilization, Ignoring Peak …
  4. Seasonal Blindness in Anomaly DetectionAnomaly Detection Model Flags Normal Seasonal Patterns as Anomalies; High False Positive R…
  5. Autoscaling Thrash from Reactive Agent DecisionsAgent-Driven Autoscaler Repeatedly Scales Up and Down in Short Cycles Because Scaling Deci…

Document Processing

  1. OCR Confuses Similar-Looking Characters (0/O, 1/l/I, 5/S): Causes and FixesOCR swaps visually similar characters (0/O, 1/l/I, 5/S, 8/B), so extracted IDs and codes l…
  2. AI Document Extraction Shows High Confidence on Wrong Answers: Causes and FixesOverconfident Wrong Answers — Model Confidence Doesn't Track Accuracy
  3. Invoice Automation Breaks on Non-Standard Vendor Formats: Causes and FixesDocument Format Diversity Overwhelms Rule-Based Extraction
  4. AI Document Extraction Accuracy Drops in Production vs Benchmarks: Causes and FixesInput Quality Gap — Real Documents Are Messier Than Benchmark Documents
  5. OCR Fails on Low-Resolution Scans and Fax Images: Causes and FixesLow DPI scans, faxes, and compressed mobile photos produce fragmented, unreadable OCR outp…

Domain Expertise

  1. AI Agent Gives a Technically Correct but Commercially Wrong Answer: Causes and FixesThe agent enforces policy to the letter and gets a technically correct answer that is comm…
  2. AI Agent Misreads Customer Emotion and Sounds Tone-Deaf: Causes and FixesThe agent mishandles empathy in debt collection, healthcare, support, or complaints and gi…
  3. AI Agent Misclassifies a Document Type: Causes and FixesThe agent misclassifies a paystub, W-2, bank statement, invoice, policy, or similar docume…
  4. AI Agent Misses an Industry-Specific Rule or Exception: Causes and FixesThe agent applies the general-case policy and misses an industry-specific rule or exceptio…
  5. AI Agent Misclassifies Risk Severity: Causes and FixesThe agent treats a critical issue as minor, or a minor issue as critical, because its seve…

External Actions

  1. AI Agent Triggers Unexpected Side Effects: Causes and FixesAI agent's action unexpectedly triggers notifications, billing charges, shipments, or depl…
  2. AI Agent Claims Success on a Partially Completed Task: Causes and FixesAI agent completes only some steps of a multi-step task but reports full success to the us…
  3. AI Agent Acts Before Gathering Enough Evidence: Causes and FixesAI agent takes an irreversible action, like banning a user or closing a case, before enoug…
  4. AI Agent Picks the Wrong Workflow Branch (Refund vs. Replacement, Escalate vs. Resolve): Causes and FixesAI agent chooses the wrong path in a decision workflow — refund vs. replacement, escalatio…
  5. AI Agent Creates Duplicate Orders, Charges, or Tickets: Causes and FixesThe agent retries a write-type tool call after a timeout or ambiguous error and ends up cr…

Financial Services

  1. Benchmark Misalignment in Portfolio RecommendationsModel Optimizes Against a Generic Benchmark That Does Not Match the Client's Actual Risk P…
  2. Correlation Breakdown in Portfolio DiversificationAssumed Diversification Fails When Correlations Spike (Tail Dependence Not Captured)
  3. Look-Ahead Bias in BacktestsModel Uses Future Information in Backtest That Wouldn't Be Available in Real-Time Trading
  4. Overfitting to Historical Market RegimeModel Optimized for Historical Regime (Bull Market) Fails in Regime Change (Bear Market, S…
  5. Recency Bias in Portfolio RecommendationsModel Overweights Recent Performance; Recommends Assets That Performed Well Recently but A…

Governance

  1. Approval Timeout ExpirationAn approval request times out because no approver responds within the configured window, a…
  2. Audit Logging Not EnforcedPolicy requires that certain tool calls — anything that reads sensitive data, anything tha…
  3. PII Retention Policy ViolationPersonally identifiable information collected or processed by an agent through a tool call…
  4. Policy Scope MisunderstandingThe agent misinterprets which actions or resources a policy actually covers — applying it …
  5. Approval Authority Escalation FailureAn agent submits an approval request that exceeds the current approver's authority limit (…

Healthcare

  1. Age Bias in Symptom InterpretationModel Misinterprets Symptoms Differently Based on Patient Age; Younger Patients Undertreat…
  2. Anchoring Bias on First DiagnosisAgent Fixates on the First Plausible Diagnosis Suggested Early in the Conversation and Dis…
  3. Demographic Bias in Diagnosis & Treatment RecommendationsModel Biased Toward/Against Certain Demographics (Race, Gender, Age); Different Accuracy f…
  4. Discharge Medication Reconciliation GapAgent Generates Discharge Medication List Without Reconciling Against Pre-Admission Home M…
  5. Dosage Renal/Hepatic Adjustment FailureAgent Recommends Standard Adult Dosing Without Adjusting for Impaired Renal or Hepatic Cle…

Hr Recruiting

  1. Resume Keyword Overfit BiasCandidate-Screening Agent Over-Weights Surface Keyword Matches Against the Job Description…
  2. Embedding Retrieval Pulls Wrong-Jurisdiction Benefits Policy During OnboardingAn Onboarding Agent's Retrieval Step Surfaces a Benefits-Eligibility Policy Document for a…
  3. Interview Transcript Sentiment Overweighted vs. ContentAI-Assisted Interview-Screening Agent Weights a Candidate's Vocal Confidence, Fluency, and…
  4. Protected Class Proxy DiscriminationResume screener learns that names, dates, or graduation years correlate with performance; …
  5. Resume Keyword Matching BiasAI resume screener uses exact keyword matching; rejects qualified candidates with industry…

Human Machine Interface

  1. Chatbot Answers, Then Stops: The Answer-and-Go Conversation
  2. Chatbots and Kids: When the User Might Be a Child or Teen
  3. Complex Conversations in Chatbots: When Chat Is the Wrong Channel
  4. Chatbot Bias by Dialect or Name: The Quality Gap
  5. Chatbot Empathy That Sounds Fake: Hollow Empathy

Insurance

  1. Embedding-Retrieval Wrong Endorsement Version AppliedA Claims-Adjudication Agent's RAG Retrieval Step Pulls a Lexically Similar but Superseded …
  2. Catastrophe Correlation BlindnessCatastrophe risk model assumes independent claims; hurricane hits coast, model hadn't prov…
  3. Embedding-Retrieval Applies Wrong Occupation-Class Rate Precedent by Lexical SimilarityAn Underwriting Agent's Retrieval Step, Used to Find a "Similar Prior Case" Precedent for …
  4. Embedding Retrieval Flags Unrelated Claimant as Fraud-Ring MatchA Fraud-Detection Agent's Link-Analysis Retrieval Step, Which Searches for Claimants Embed…
  5. Embedding Retrieval Pulls Mismatched Rating-Territory Policy Precedent for RenewalA Policy-Renewal Agent's Retrieval Step Surfaces a "Similar Policy" Precedent via Embeddin…

Knowledge Retrieval

  1. Ambiguity MishandlingAgent Guesses Instead of Clarifying Ambiguous Queries
  2. Chunk Boundary IssuesRelevant Information Split Across Chunks
  3. AI Agent's Confidence Doesn't Match How Reliable Its Answer Is: Causes and FixesAI agent sounds equally confident whether its answer is well-supported by context or barel…
  4. Fact Context LossA retrieval system pulls a fact that is accurate on its own terms, but the qualifying cont…
  5. Fact Generalization ErrorAn agent takes a fact that is true only under narrow, specific conditions — a particular s…

Learning

  1. Learning From Bad FeedbackAgent updates behavior based on incorrect/noisy feedback.
  2. No Improvement AuditCannot explain what changed and why.
  3. No Root-Cause SeparationAgent treats symptom as cause: prompt vs retrieval vs tool vs policy.
  4. Overfitting To IncidentsFix addresses one case but damages general behavior.
  5. Wrong Fix TargetSystem changes prompt when schema/retrieval/tool/data was root cause.

Legal Contracts

  1. Boilerplate Clause MisapplicationAgent Drafts a New Contract by Inserting a Standard Boilerplate Clause Library Entry That …
  2. Contract Amendment Tracking FailureContract Modified Multiple Times; Model Analyzes Original, Ignoring Amendments; Outdated A…
  3. Change-of-Control Clause Omission in M&A Due DiligenceAgent Reviewing a Target Company's Contract Portfolio for M&A Due Diligence Fails to Flag …
  4. Choice-of-Law & Jurisdiction MishandlingModel Fails to Properly Interpret Choice-of-Law Clauses; Applies Wrong Jurisdiction's Laws…
  5. Circuit Split Blindness in Citation SelectionLegal Research Agent Cites a Precedent as Settled Law Without Flagging That a Circuit Spli…

Mortgage Documents

  1. Verification CollapseAI Systems Validating Their Own Outputs Without Independent Verification
  2. Address Standardization FailuresOCR System Fails to Standardize or Match Addresses Across Documents
  3. Appraisal Data Discrepancy Detection FailuresAI QC System Fails to Detect Appraisal Inconsistencies That Trigger GSE Findings
  4. Asset Source TracingAI System Fails to Trace Assets to Legitimate Sources
  5. Asset Verification FailuresOCR System Incorrectly Extracts or Validates Borrower Assets

Multi Agent Systems

  1. AI Agents Reach False Consensus: Causes and FixesMultiple independent agents reach the same wrong conclusion, and the system mistakes that …
  2. AI Agent Authority Confusion: Causes and FixesAgents disagree on who has final say, and the system silently picks a winner instead of re…
  3. Multi-Agent Communication Loss: Causes and FixesAgents lose or drop key information passed between them, so a downstream agent acts withou…
  4. AI Agents Give Contradictory Outputs: Causes and FixesTwo or more agents disagree and nothing in the pipeline resolves the conflict, so contradi…
  5. AI Agent Coordinator Failure: Causes and FixesThe manager/orchestrator agent (the coordinator role in LangGraph supervisor graphs or Cre…

Operations

  1. Agent Resource ContentionMultiple agents operating concurrently compete for the same limited resource — a shared LL…
  2. Batch Size LimitBulk-operation tools commonly enforce a maximum number of operations (rows, records, actio…
  3. Connection Timeout No RetryA tool call's underlying TCP/TLS connection times out — a transient blip caused by network…
  4. Context Coherence LossOver a long-running session, an agent's live working context (the conversation buffer, scr…
  5. Context Window Awareness FailureAn agent has no internal tracking of how much of its context window is currently consumed,…

Reasoning and Thought

  1. Model Context Length Behavior ChangeAs an agent's conversation or retrieved context grows toward the model's context window li…
  2. Model Instruction Following DecayA system prompt's rules — tone constraints, formatting requirements, forbidden topics, rol…
  3. Model Knowledge CutoffThe model answers questions about facts, prices, APIs, regulations, or current events usin…
  4. Model Output Format InstabilityAn agent requests a strictly-formatted response (JSON matching a schema, XML with specific…
  5. Model Uncertainty UnawarenessThe model generates answers in a uniformly confident tone regardless of how certain it act…

Sales Crm

  1. Best-Case Projection BiasSales forecasting model uses deal probability from salesperson input, which is inherently …
  2. Embedding-Similarity Retrieves Superficially Similar Deal as PrecedentA Lead-Scoring Agent That Justifies Its Score by Retrieving "Similar Past Deals" via Embed…
  3. Multi-Agent Handoff Drops Negotiated Payment-Terms Exception Before Deal-Desk ApprovalAn Account Executive's Free-Text Email Negotiation Establishing Extended Payment Terms for…
  4. SDR-to-AE Handoff Drops Unstructured Disqualifying SignalAn SDR-Qualification Agent's Chat Transcript with a Prospect Contains a Disqualifying Sign…
  5. Agent Applies Remembered Scoring Heuristic Instead of Querying Live Scoring-Rules ToolA Lead-Scoring Agent, When Asked to Explain or Compute a Lead's Score, Falls Back on a Gen…

Security

  1. Memory Poison Defense Gap: Existing Tools InsufficientStandard agent defenses (tool contracts, circuit breakers, I/O moderation, sandboxing) det…
  2. Over-Scoped Agent PermissionsAgents Provisioned with Excessive Access Beyond Task Requirements
  3. Owner Verification Not EnforcedBefore performing a mutating action on a specific resource (cancel this subscription, dele…
  4. PII ExposureAgent Outputs Contain Personally Identifiable Information
  5. Sensitive Data in LogsPII and Secrets Written to Logs, Traces, and Observability Systems

Speech and Audio

  1. Accent and Dialect BiasASR Performs Significantly Worse on Non-Standard Accents
  2. Background Noise FailuresASR Accuracy Degrades Significantly in Noisy Environments
  3. Brand and Term MispronunciationTTS Mispronounces Brand Names, Domain Terms, or Key Phrases
  4. Homophones ConfusionASR Selects Wrong Word Among Sound-Alike Options
  5. Monologue Without EngagementAgent Delivers Long Feature Lists or Information Without Pausing for Engagement

Supply Chain

  1. Bullwhip Effect & Cascading Forecast ErrorSmall Demand Variation at Retail Cascades to Large Forecast Errors Upstream; Overproductio…
  2. New Product Cold-Start MisforecastAgent Forecasts Demand for a Newly Launched SKU Using a Global Average New-Product Curve T…
  3. Seasonal Demand Misses & Holiday BlindnessForecasting Model Fails to Account for Seasonal Variations; Massive Stockouts/Overstock on…
  4. Carrier Capacity Blindness in Route OptimizationAgent Optimizes Shipping Routes for Cost and Transit Time Using Carrier Capacity Assumptio…
  5. Financial Distress Signal BlindnessAgent's Supplier Risk Monitoring Relies on Quarterly or Annual Financial Statements, Missi…

Support Services

  1. Canned Response Context MismatchAgent Selects a Templated/Canned Response That Matches Surface Keywords but Misses the Cus…
  2. Chatbot Loop Without Human Escalation PathSelf-Service Deflection Agent Keeps Offering the Same or Similar Automated Suggestions Wit…
  3. Deflection of Unresolved IssuesSelf-Service Agent Marks a Conversation as "Deflected" (Resolved Without Human Agent) Base…
  4. Embedding Retrieval Misroutes Ticket via Similarity to Wrong Product-Line Taxonomy NodeA Ticket-Routing Agent That Classifies an Incoming Ticket's Product Category by Embedding …
  5. Knowledge Base Staleness in Support ChatbotsSupport Agent Uses Outdated Knowledge Base; Gives Customers Incorrect or Outdated Informat…

Task Planning

  1. Ambiguous Goal InterpretationAgent optimizes for a different meaning of the user's/business goal.
  2. Business-Policy MismatchAgent completes a technical action that violates company policy.
  3. Conflicting ObjectivesAgent cannot resolve tradeoffs like speed vs accuracy, helpfulness vs compliance.
  4. Goal Drift Across TurnsAgent's objective changes over long conversations or workflows.
  5. Goal Expansion / Scope CreepAgent performs additional actions that were not requested.

Vision and Images

  1. Confidence MiscalibrationModel Confidence Doesn't Match Hallucination Probability
  2. Lighting & Color Shift SensitivityModel Highly Sensitive to Lighting Conditions and Color Changes; Fails Under Different Ill…
  3. Low-Resolution Image FailureModel Performance Collapses on Low-Resolution Images; Cannot Recognize Objects When Downsa…
  4. Object Tracking Failure in Multi-Frame SequencesModel Cannot Maintain Consistent Object Track Across Multiple Frames; Loses or Confuses Id…
  5. Occlusion MishandlingModel Fails to Reason About Occluded (Partially Hidden) Objects