Verification

Agents skip or perform inadequate verification of their own outputs, leaving errors undetected — verification is absent (no check at all), shallow (checking only format not correctness), or biased (agent verifies its own work and finds no errors, even when a second opinion would catch them). These failures are architectural: the problem isn’t model capability, it’s the absence of verification in the deployment pipeline or verification logic that lacks sufficient independence and depth.

Key Takeaways

• 15 distinct failure patterns affect verification, grouped into four mechanisms: missing verification (no check at all), shallow verification (format checks only), biased verification (agent checks its own work), and methodology gaps (testing only happy paths, not adversarial cases, not long-horizon scenarios).
• Verification failures are particularly dangerous because they’re the last line of defense — once verification is compromised, incorrect outputs ship to production with confidence. Worse, they’re invisible: a system without verification looks fine until an external audit surfaces the gap.
• The reliable fix is architectural, not model-only: make verification mandatory for state-changing actions and high-stakes output; require verification by an independent agent or tool (not self-verification); use multiple verification methods (format, business logic, ground truth comparison, human review); verify the verifier itself against test cases.
• Verification failures concentrate in systems where verification is treated as optional or low-priority (nice-to-have rather than non-negotiable) and in systems where agents verify their own work.

Scope

• Missing verification — no-verification, no-human-review-trigger. No verification step at all, or no mechanism to route high-stakes output to human review.
• Shallow verification — surface-level-validation, eval-data-mismatch. Verification checks only format (schema validity, field presence) not correctness (value accuracy, business-rule compliance).
• Biased verification — self-verification-bias, verifier-hallucination, over-trusting-confidence-score. Agent verifies its own work and finds no errors; verifier itself hallucinate; confidence scores treated as equivalent to correctness.
• Methodology gaps — happy-path-only-evals, no-adversarial-testing, no-long-horizon-evaluation, no-regression-testing, no-ground-truth-comparison. Testing only common cases, not edge cases or adversarial inputs; not testing multi-step outcomes; not comparing against ground truth; no regression suite.
• Misaligned verification — wrong-verifier, no-business-kpi-validation, metric-gaming. Verifier doesn’t match task requirements; business impact not validated, only technical metrics; agent optimizes for validation metric rather than business goal.

When Verification Matters

• Agent’s output feeds systems or decisions where errors have high consequences (financial, regulatory, safety, reputational impact)
• Agent performs state-changing actions (writes, sends, posts) where verification can confirm the action succeeded as intended
• Multiple verification approaches are possible (human review, independent tool verification, ground truth comparison) and combining them would catch errors that any single method misses
• Production performance diverges from evaluation performance — indicates evaluation methodology is inadequate and verification is needed to catch the gap

Cross-Pattern Insight

Across all 15 patterns, the single most reliable mitigation is mandatory, multi-layered verification: (1) require every state-changing action to have a verification step before reporting success (not self-verification, but via independent tool or readback); (2) require high-stakes outputs to pass multiple verification methods (format + business logic + human review sample); (3) verify the verifier itself against test cases to catch verifier hallucination. Cases where verification is mandatory and multi-layered consistently catch errors that single-layer verification misses. The second universal mitigation is comprehensive test methodology — if testing covers only happy paths, failures in edge cases and long-horizon scenarios won’t surface until production.

Frequently Asked Questions

How does verification differ from output accuracy?

Output accuracy covers generation of false outputs (hallucination, bias). Verification covers detection of false outputs. An agent can hallucinate, and verification is the safeguard that catches it. If accuracy is the generation problem, verification is the detection problem.

How should confidence scores be used in verification?

Confidence scores don’t correlate with accuracy — a model can be highly confident when wrong. Verification requires checking against external ground truth or business logic, not relying on internal confidence. A model confident on false output still produces false output.

Can you fix verification failures by adding more test cases?

More test cases help, but if test methodology is flawed (happy path only, no adversarial cases, no ground truth comparison), more cases don’t fix the problem — they just test the wrong thing. The fix is to diversify test methodology: happy paths + edge cases + adversarial inputs + long-horizon scenarios + ground truth comparison.

Which verification failures matter most for production systems?

No-verification (no check at all) and self-verification-bias (agent checks its own work and finds no errors) are highest-priority because they’re silent and allow errors to propagate. Happy-path-only-evals is next because it trains on wrong distribution and misses real-world failures.

Patterns

Total: 15 patterns

Related Goals

• Output Accuracy — hallucination and bias, which verification is designed to catch
• Evaluation Reliability — golden-data quality and methodology, upstream of verification
• Output Optimization — confidence calibration and abstention, which complement verification