RAG hallucination patterns — retrieval failures that cause AI agents to fabricate answers
75 patterns
RAG hallucination patterns — retrieval failures that cause AI agents to fabricate answers. We've tracked 75 distinct RAG Hallucination patterns across production AI agent deployments — most commonly in Knowledge Retrieval, Accuracy systems. One recurring example: Agent Defaults to Stale Training Knowledge Over Available Live-Lookup Tool — Agent has access to live lookup tool that returns current reference data (policy, rules, calendar, threshold), but defaults to stale training-time knowledge instead of calling/trus…
Each pattern below documents the specific failure mode, why it happens, and how teams have mitigated it in practice — use this page as a reference when building guardrails, writing evals, or debugging an agent that's already misbehaving in this way.
Agent has access to live lookup tool that returns current reference data (policy, rules, calendar, threshold), but defaults to stale training-time knowledge instead of calling/trusting the tool; downstream decisions use outdated values
Agent Guesses Instead of Clarifying Ambiguous Queries
AI agent gives a coherent, accurate answer that doesn't actually address the question the user asked.
Correct information is retrieved but summarized incorrectly.
AI agent's citations point to documents that have been moved, deleted, or made inaccessible since they were indexed.
AI agent selectively cites supporting evidence from retrieved context while ignoring information that contradicts it.
Relevant Information Split Across Chunks
Needed fact is split across chunks and lost.
Agent cites a source that does not support the claim.
Summarizing or compressing retrieved context before it reaches the model drops the specific numbers, dates, or conditions the answer needed.
AI agent sounds equally confident whether its answer is well-supported by context or barely grounded at all.
Agent fails to reconcile contradictions between sources.
AI agent answers from its own training knowledge instead of using the specific documents retrieved for it.
Too many chunks dilute relevant evidence.
Stuffing the context window with more retrieved documents makes the AI agent's answers worse, not better — the "lost in the middle" effect.
An agent retrieves and applies information that was once the accepted best practice in a domain but has since been superseded by an evolved standard, even though the underlying fact is still technically true. The agent's knowledge source (a fine-tuned model, a static knowledge base, or a cached document set) captured the practice at a point in time and was never re-indexed against the field's current consensus, so the agent confidently recommends an approach that a current practitioner would flag as outdated. The advice isn't factually wrong in isolation — it's wrong relative to what the domain now considers correct.
An agent produces output or takes an action that violates a hard, non-negotiable constraint of the domain it's operating in — a regulatory requirement, a safety interlock, a licensing restriction — because the constraint was never surfaced by its retrieval layer. Unlike a best-practice miss, this isn't a matter of degree: the agent crosses a bright line (e.g. recommending a drug dosage that exceeds a labeled maximum, or drafting a contract clause that's unenforceable in a given jurisdiction) because its knowledge base treated the constraint as optional context rather than a gating rule.
An agent correctly establishes domain-specific context early in a session — the specialty, jurisdiction, or technical stack it should reason within — but loses track of it as the conversation grows, silently reverting to generic, domain-agnostic behavior. The regression isn't triggered by the user changing topics; it happens because the domain-framing information falls out of the effective context window or gets diluted by intervening turns, and nothing in the agent's architecture re-asserts it.
An agent correctly retrieves and applies a general domain rule, but fails to recognize that the specific case at hand falls under a documented exception that overrides or modifies the general rule. The exception exists in the knowledge base — often in a separate section, footnote, or appendix — but the agent's retrieval or reasoning never connects the specific case to it, so the general rule is applied as if it were universal.
An agent operating in a specialized domain fails to flag a risk factor that any competent domain practitioner would immediately recognize as significant, not because the underlying fact is missing from its knowledge base, but because that fact was never tagged or weighted as risk-relevant. The agent retrieves and states the fact correctly when directly asked, yet doesn't proactively surface it as a concern in a context where a domain expert would treat it as a red flag requiring attention.
An agent retrieves a correctly-stated domain rule but misapplies it because it misreads the precise conditions under which the rule holds — extending it to cases just outside its actual scope, or narrowing it to exclude cases it actually covers. The rule text itself is never altered or hallucinated; the failure is in the agent's interpretation of qualifying language like "only if," "except when," or "applies to X but not Y" that defines the rule's true boundary.
An agent interprets a domain-specific term using its common, general-language meaning instead of the narrower or entirely different meaning the term carries within the specialized domain, producing a response that's coherent but answers the wrong question. This happens most with terms that are ordinary English words repurposed with precise technical meaning (e.g. "significant" in statistics, "material" in accounting, "positive" in a lab result), where the domain meaning can even be the near-opposite of the everyday connotation.
Embedding Model Changes Break Retrieval
AI agent invents plausible-sounding citations — studies, articles, or case law — that don't actually exist.
A retrieval system pulls a fact that is accurate on its own terms, but the qualifying context that makes it correctly applicable — the condition, population, or scope it was originally stated under — is dropped somewhere between the source document and the agent's final answer. The fact survives; the sentence or clause that scoped it does not, usually because chunking, summarization, or context-window truncation separated the fact from its qualifier.
An agent takes a fact that is true only under narrow, specific conditions — a particular study population, a specific product configuration, a specific regulatory jurisdiction — and presents it as a general truth applicable broadly. The source fact isn't misquoted; the error is in stripping away the scope that made it narrowly true and applying it as if it held universally.
An agent retrieves a fact correctly in terms of subject matter but reverses its direction or polarity — reporting "increases" when the source says "decreases," "improves" when the source says "worsens," or swapping which of two entities has the higher value. The topic and the entities involved are right; the relationship between them is flipped, which is often more damaging than an unrelated error because it's confidently stated and directionally opposite to the truth.
An agent drops, adds, or misplaces a negation while processing retrieved text, asserting the opposite of what the source actually states. Unlike a full directional inversion, this failure is specifically about negation words and constructions ("not," "no longer," "except," "unless," double negatives) being mishandled during retrieval, summarization, or paraphrase, producing a claim that reads fluently but contradicts the source on the single most important word in the sentence.
An agent presents a fact that is technically accurate as stated but omits a critical qualifier that would materially change how a user should act on it — not because the qualifier is missing from the source, but because it was dropped somewhere in retrieval or generation and the resulting statement, while not false, is misleadingly incomplete. This differs from a fabrication: every word the agent says checks out against the source, but the selective omission changes the practical meaning.
An agent retrieves a fact that was expressed by its source with explicit probabilistic or statistical framing — a likelihood, a confidence interval, a rate observed in a sample — and restates it as a flat, deterministic certainty, dropping the uncertainty that was integral to the original claim's meaning. The number or direction carried over is often correct; what's lost is the epistemic status of the claim, turning "this happens in about 30% of cases" into "this happens" or "this is what will happen."
An agent retrieves facts about two distinct entities — companies, people, products, or regulations — that share a similar name, and conflates attributes from one with the other in its response. The resulting statement is internally coherent and often partially correct, but attributes a fact that belongs to Entity A to Entity B, because retrieval matched on name similarity rather than correctly disambiguating which entity the query actually concerns.
An agent retrieves a fact that was true during a specific window of time and applies it outside that window, because the fact's temporal validity period was mishandled — the agent misattributes when the fact was true, or fails to notice that the fact is now outside its valid period. Unlike a stale cache issue, this is specifically about misreading or mismanaging the time-scoping of the fact itself: applying a 2019 regulatory limit as though it were still current in 2026, or applying a "current" fact to a past scenario the user is actually asking about.
System Accepts User's Incorrect Assumptions
Agent Doesn't Maintain Context Across Conversation Turns
AI agent cites an entire document when a specific section is needed — or vice versa — leaving the user to search for the actual supporting passage.
AI agent generates information that sounds plausible but isn't actually present in the documents it retrieved.
Agent Misses Unstated but Expected Aspects of Query
Retrieved Documents Are Outdated
Agent Misunderstands What User Wants to Accomplish
Retrieved Content From Wrong Jurisdiction or Context
Malicious Content Injected into RAG Data Sources
An agent's retrieval step pulls two or more facts from different sources that directly contradict each other on the same question, and the agent proceeds to answer using one of them (often whichever appears first, scores marginally higher in relevance, or was retrieved last) without noticing, reconciling, or flagging the contradiction to the user. The user receives a confident single answer with no indication that the knowledge base itself disagrees on the point.
A knowledge base or cache stores facts indefinitely with no time-to-live (TTL) or expiration mechanism, so content that was accurate at ingestion time but has a known or implicit shelf life (prices, policies, personnel, regulatory limits) remains fully retrievable and presented with the same confidence as current content, indefinitely, unless someone manually removes or updates it. This is a systems-level gap rather than a per-fact error: the architecture itself has no concept of "this should stop being trusted after time T."
An agent applies a fact using an incorrectly assumed scope — the wrong jurisdiction, the wrong product version, the wrong organizational unit — because it never explicitly confirmed the scope the fact actually applies to versus the scope the user's situation is actually in. The fact is retrieved correctly and stated correctly for its true scope; the error is the silent assumption that the true scope matches the user's, when it may not.
A retrieval system treats every indexed source as equally trustworthy, with no mechanism to weight or rank content by the reliability of where it came from — an official, reviewed policy document is retrieved and used with exactly the same confidence as an unreviewed wiki page, a stale forum post, or a low-quality scraped page, simply because both matched the query with similar semantic relevance. When sources disagree or vary in quality, the system has no basis for preferring the more trustworthy one.
A source document explicitly scopes a fact with "as of" framing — "as of Q3 2025," "current as of the last policy revision," "prices shown are for the current promotional period" — but that framing is stripped during retrieval, summarization, or generation, leaving the agent's stated fact presented as timeless and universally current rather than tied to the specific moment the source actually anchored it to. The number or claim itself is preserved correctly; only the temporal anchor that made it interpretable is lost.
The system of record that an agent's knowledge base is supposed to reflect has been updated — a policy changed, a price changed, a product was discontinued — but the agent's indexed or cached copy has not caught up, because the ingestion/re-indexing pipeline runs on a cadence (scheduled batch job, manual trigger, event-driven pipeline with a backlog) that lags behind the actual rate of change at the source. The agent isn't wrong about what its knowledge base says; its knowledge base itself is behind reality.
An agent answers using knowledge tied to one version of a product, policy, or API — often the version most represented in its training data or knowledge base — while the user is actually working with a different version, and the two versions differ in ways that make the agent's answer wrong or actively harmful for the user's actual situation. The agent isn't confused about the fact itself; it's applying a fact that is correctly true for version N to a user who is on version N+1 or N-1, without checking or asking which version applies.
Agent retrieves irrelevant chunks and synthesizes wrong answer.
Agent misses relevant documents.
Wrong date, version, region, product, role, or policy filter.
Filters Exclude Relevant Documents
AI agent cites a real, retrievable document, but the document doesn't actually support the specific claim being made — a citation that passes a quick verification while still being wrong.
AI agent states factual claims without attaching a citation, so the user can't verify where the information came from.
Agent Only Addresses Part of Multi-Part Question
AI agent's answer quality drops when irrelevant documents get mixed into the retrieved context — the agent uses noise it should have ignored.
Agent misreads scanned, smudged, rotated, or low-quality text.
Too Many Documents Retrieved, Mostly Irrelevant
AI agent's answer reflects its training data instead of the current, retrieved context — even when the two directly conflict.
Complex Query Broken Into Wrong Subqueries
Query Expansion Adds Irrelevant Terms
Reranker Makes Retrieval Quality Worse
Relevant Documents Not Ranked Appropriately
Agent Answers at Wrong Scope or Specificity
Query Doesn't Match Document Language
When retrieving a comparable or reference item using embedding similarity over text descriptions, agent selects textually similar but structurally incompatible item; downstream operations assume structural compatibility that doesn't exist
AI agent's retrieved documents contain conflicting information, and the agent picks one arbitrarily instead of surfacing the disagreement.
Agent uses outdated policy/doc/version.
AI agent incorrectly merges facts from different documents, attributing details to the wrong entity when synthesizing an answer.
Agent misses data embedded in tables, charts, images, or PDFs.
Retrieved Content Is Outdated or Superseded
Relevant Documents Not Retrieved
AI agent attaches a real citation to a claim, but the cited source doesn't actually state what the agent says it does.
Agent searches the wrong knowledge base or tenant corpus.
AI agent confuses which retrieved document a fact came from and attaches the citation to the wrong source.