Accuracy

66 patterns in this category

Agents generate plausible-sounding but false content, apply outdated knowledge, lose track of constraints over long conversations, deviate from reasoning requirements, and skip verification of outputs — accuracy failures are silent because the output is well-formed and the agent is confident, so errors propagate downstream before being caught. Accuracy issues span the entire agent pipeline: context management (losing track of instructions), reasoning quality (planning incompletely), output generation (hallucination), and verification (skipping or biasing the check).

Key Takeaways

  • 66 distinct failure patterns affect accuracy across 8 goals, spanning context handling (8 patterns), evaluation methodology (8 patterns), knowledge staleness (1 pattern), generation quality (16 patterns), optimization of techniques (5 patterns), output verification (1 pattern), reasoning (12 patterns), and test-time validation (15 patterns).
  • Accuracy failures are categorically invisible at generation time — a hallucinated fact reads like a true fact, a skipped verification step produces no error message, an instruction drift shows up only in subtle behavior change over many turns. Detection requires comparison to ground truth or external audit, not observation of the output itself.
  • The reliable fix is multi-layered: (1) context architecture that maintains instruction durability and state tracking; (2) evidence-gating on high-stakes claims (require retrieval before generation); (3) confidence calibration and abstention affordances (let agents refuse when confidence is low); (4) multi-layered verification (format + business logic + human review); (5) comprehensive test methodology (not just happy paths).
  • Accuracy failures concentrate wherever the real world is more complex than the training distribution (seasonal shifts, new entity types, adversarial inputs), where rules change faster than models can be retrained (regulatory thresholds, policy updates), or where agents must chain multiple reasoning steps correctly (multi-step planning, long-horizon reasoning).

Goals

GoalPatternsCoverage
Context Management8Instruction conflicts, context size limits, state tracking, session boundaries
Evaluation Reliability8Coverage gaps, distribution shift, data quality, metric mismatch
Knowledge Staleness1Agent defaults to training knowledge over live tools
Output Accuracy16Hallucination, bias, fabrication, domain mismatch, inherited errors
Output Optimization5Missing abstention, confidence miscalibration, skipped verification, degenerate output
Output Verification1Circular verification (same source checks itself)
Reasoning Quality12Planning failures, constraint violations, goal misalignment, self-correction gaps
Verification15Missing verification, shallow checks, self-verification bias, methodology gaps

Total: 66 patterns across 8 goals

When Accuracy Matters

  • Agent generates content that flows directly to end users or drives autonomous decisions without intermediate human review
  • High-stakes domains (healthcare, finance, legal, compliance) where accuracy errors have safety, regulatory, or reputational impact
  • Agent must maintain correctness over long conversations, requiring instruction durability and state tracking
  • Vulnerable populations or underrepresented groups where accuracy degradation for specific segments goes undetected

Architecture Principles for Accuracy

The core insight across all 66 patterns: accuracy requires architecture, not just model capability. A capable model can generate false content (hallucination), lose track of constraints (context drift), apply outdated knowledge (staleness), or be deployed without verification. The mitigations fall into three architectural categories:

  1. Context and state durability: Keep instructions durable (separate external store, periodic re-injection), maintain state outside context window (database, external ledger), enforce session boundaries (no cross-session leakage).

  2. Evidence-gated generation: Require retrieval or explicit evidence before generation; mark claims with sources; abstain when evidence is insufficient; evidence-gating eliminates fabrication more reliably than prompting.

  3. Verification and testing: Make verification multi-layered (format + business logic + human sample), independent (not self-verification), and comprehensive (not just happy paths). Verify the verifier itself. Test on production distribution, not convenience samples.

  • Operations — cost efficiency, tool reliability, and state tracking, upstream of accuracy
  • Security — preventing adversarial attacks and prompt injection, which undermine accuracy guardrails
  • Learning — safe learning from feedback, which can degrade accuracy if feedback is biased
  • Governance — approval workflows and oversight, which gate high-stakes decisions accuracy must support

See Core for other cross-cutting patterns.

Confidence Calibration Failure

Frequency: Common
Category:

Agent's verbalized or scored confidence does not correlate with its actual answer correctness, so confidence cannot be used to gate downstream decisions.

Deterministic Verification Bypassed

Frequency: Common
Category:

Agent relies solely on an LLM-judge to assess its own output when a deterministic, executable check (schema validation, test suite, linter, tool-call format check) was available and would have caught the error at near-zero cost.

Distribution Shift

Frequency: Very Common
Category:

Production Queries Differ Significantly from Golden Dataset

Domain Mismatch

Frequency: Common
Category:

Agent Fails on Out-of-Distribution or Specialized Inputs

Goal Drift

Frequency: Common
Category:

Agent Loses Focus on Original Objective

Hallucinated Completion When Upstream Dependency Fails

Frequency: Common
Category:

When an agent's external API call (validation, lookup, confirmation) times out or fails, the agent completes a plausible result claiming success instead of treating failure as a blocking condition; downstream systems trust the fabricated success status

Long-Session Context Loss Violates Earlier Constraints

Frequency: Common
Category:

In a long conversation, agent establishes constraints, decisions, or flags early (banned phrase, disqualified candidate, SLA exception, allergy, privilege determination), but as session grows, that information falls out of effective context window; agent later violates the constraint or re-makes the excluded decision

Metric Gaming

Frequency: Common
Category:

Agent learns to satisfy eval wording instead of real behavior.

Missing Abstention Affordance

Frequency: Common
Category:

Agent's output space has no low-friction "insufficient information, cannot answer" option, so it produces a best-guess answer even when grounding is inadequate.

Missing Self-Reflection for High-Stakes Output

Frequency: Occasional
Category:

Agent skips a beneficial critique/revise pass on a high-stakes output, going straight to a single-shot answer where a reflection round would demonstrably have caught an error.

Repetitive Degenerate Generation

Frequency: Occasional
Category:

A single generation call falls into repeated phrases, loops, or degenerate text (distinct from repeating tool-call actions across turns) because no repetition/frequency penalty or diversity control is applied to open-ended output.

Self-Verification Cannot Catch Upstream Errors

Frequency: Common
Category:

Agent double-checks its own output by re-querying the same upstream source; finds no discrepancy because the problem was in the original source, not in the agent's processing; reports "verification passed" despite using incorrect upstream data

Sycophancy

Frequency: Very Common
Category:

Agent Agrees with User Even When Wrong