Frequency:Common
An agent hands a task to the underlying model whose combined complexity — number of constraints, depth of multi-step reasoning, size of working set held "in mind" across a long tool-calling loop — exceeds what that model can reliably handle in a single pass. Unlike a hard error, the failure is silent: the model still produces a fluent, well-formatted answer, but it drops constraints, skips reasoning steps, or produces a plausible-looking but wrong result. Nothing in the response signals that the task was too much for the model.
Frequency:Very Common
As an agent's conversation or retrieved context grows toward the model's context window limit, the model's behavior shifts in ways that are never announced: recall of early-turn facts degrades, instruction-following becomes less reliable, and the model increasingly favors recently-seen tokens over earlier ones ("recency bias"). Because the API returns a normal, well-formed response at every context length, the agent has no signal that it just crossed into a degraded-quality regime.
Frequency:Common
The underlying model exhibits systematic differences in its outputs correlated with demographic or protected attributes — name-implied ethnicity, gender-coded pronouns, geography, or dialect — that leak into agent decisions the model wasn't explicitly asked to make on that basis. Because the bias is statistical rather than an explicit rule, it survives even when the agent's prompt contains no discriminatory instruction, and it recurs consistently enough to produce a measurable disparate pattern across many decisions.
Frequency:Very Common
A system prompt's rules — tone constraints, formatting requirements, forbidden topics, role boundaries — are followed reliably in the first few turns of a conversation but are followed progressively less reliably as the conversation grows longer, even though the system prompt itself never changes and is technically still present in every call. The agent has no mechanism to notice that adherence has dropped, since each individual response still looks like a normal, fluent reply.
Frequency:Very Common
The model answers questions about facts, prices, APIs, regulations, or current events using knowledge frozen at its training cutoff, but presents the answer with the same confident, unhedged tone it would use for a fact that is still current. The agent has no built-in awareness of which of its facts have gone stale since training, so it cannot distinguish "this is still true" from "this was true as of my cutoff" without an explicit check.
Frequency:Very Common
An agent requests a strictly-formatted response (JSON matching a schema, XML with specific tags, a fixed-width table) and the model complies most of the time, but intermittently deviates — adding prose before the JSON, using a slightly different key name, wrapping output in markdown code fences one call and not the next, or emitting a subtly malformed structure. Because the deviation is intermittent rather than constant, it passes casual testing and only surfaces as parse failures at some rate in production.
Frequency:Common
The model produces different reasoning chains and different final conclusions when given logically identical inputs that differ only in superficial ways — order of options, phrasing, irrelevant surrounding text, or which call happens to sample a different token early in the chain of thought. An agent that relies on the model's reasoning to make a consistent decision (approve/deny, rank A over B, classify as X) gets a decision that isn't actually a function of the underlying facts, just of incidental surface variation.
Frequency:Common
The model refuses a request in one call and complies with a substantively identical or even more sensitive request in another, with no discernible policy logic explaining the difference — only surface phrasing, conversation framing, or incidental sampling variation. An agent that depends on the model's own judgment as its safety boundary inherits this unpredictability: the same downstream user action can be blocked or allowed depending on factors the agent's designers never intended to matter.
Frequency:Common
An agent configured with a specific persona — tone, formality level, brand voice, characteristic phrasing — maintains that persona faithfully at the start of a session but gradually drifts toward a generic, default assistant voice as the conversation lengthens, without any instruction telling it to change. The drift is slow enough that no single turn looks obviously wrong, but a comparison of turn 2 to turn 40 shows a clearly different "character" giving the responses.
Frequency:Very Common
The model generates answers in a uniformly confident tone regardless of how certain it actually is about the content, so an agent (and the end user) cannot distinguish a well-grounded answer from a guess by reading the response alone. Low-confidence, borderline, or fabricated content is phrased with the same declarative certainty as well-established facts, removing the natural signal a human expert would give ("I'm not sure, but...") that would otherwise prompt verification.
A model’s behavior degrades without any error being returned because degradation in LLMs is almost always a soft, statistical ceiling rather than a hard limit — the API still returns a fluent, well-formed response at every level of context fill, conversation length, or task complexity, so nothing in the response itself signals that quality just dropped. All 10 model-behavior-and-capabilities patterns describe a different axis along which degradation happens: task complexity exceeding effective capacity, context filling up, instructions and persona eroding over a long session, knowledge going stale, output format becoming unstable, reasoning becoming inconsistent, refusal boundaries becoming porous, and the model’s own confidence signal being decoupled from its actual accuracy.
Key Takeaways
- 10 patterns are documented here, and every one shares the same core symptom structure: a normal-looking, confident response with no built-in signal that anything degraded.
- 5 of the 10 patterns are rated “Very Common” (Context Length Behavior Change, Instruction Following Decay, Knowledge Cutoff, Output Format Instability, Uncertainty Unawareness) — the five are not edge cases but default production behavior.
- Three patterns (Context Length Behavior Change, Instruction Following Decay, Style Drift) share the identical root mechanism: a fixed-position system prompt or early-turn fact loses relative influence as the conversation transcript grows, and none of the three are announced by the model.
- Reported effect sizes are large where measured: constraint-adherence rates drop from ~95% to 60-75% past 7-8 simultaneous constraints (Capacity Limits); recall accuracy for mid-context facts runs 20-40 points below start/end-of-context facts (Context Length Behavior Change); rule adherence in 30+ turn conversations runs 20-35 points below the first 5 turns (Instruction Following Decay).
Scope
- Session-Length Decay — Model Context Length Behavior Change (GitHub), Model Instruction Following Decay (GitHub), Model Style Drift (GitHub). All three are driven by the same mechanism: a system prompt or early fact is fixed in position while the conversation grows around it, so its relative influence on the next generated token shrinks — degrading recall, rule-following, and persona/tone in that order as the transcript lengthens.
- Call-to-Call Nondeterminism — Model Capacity Limits (GitHub), Model Output Format Instability (GitHub), Model Reasoning Inconsistency (GitHub), Model Refusal Inconsistency (GitHub). The four call-to-call-nondeterminism patterns describe the same logical request producing different results across runs or under trivial surface variation (option order, phrasing, sampling draw, task density) — one call’s output isn’t a reliable function of the input’s actual logical content.
- Silent Knowledge & Judgment Gaps — Model Knowledge Cutoff (GitHub), Model Uncertainty Unawareness (GitHub), Model Fairness Bias (GitHub). The three silent-knowledge-and-judgment patterns describe a gap between what the model appears to know or judge fairly and what it actually knows or judges fairly, delivered with the same confident tone regardless of the underlying gap.
When Model Behavior & Capabilities Matters
- An agent runs long conversations or sessions (support chat, ongoing troubleshooting) where rules, persona, or early facts need to hold for the full session, not just the first few turns
- A pipeline depends on the model reliably following a strict output contract (JSON schema, a fixed set of constraints) across a high volume of calls, where even a low per-call failure rate compounds at scale
- A decision pipeline (screening, triage, risk scoring) uses model judgment on inputs where demographic proxies, stale facts, or borderline calls could silently bias or miscalibrate the outcome
Cross-Pattern Insight
Every mitigation across all 10 patterns follows the same shape: don’t trust the generating call’s own signal of success, and instrument something external to check it. For session-length decay, that means periodic re-injection of the system prompt or critical facts rather than assuming a single system-prompt statement holds for an entire session. For call-to-call nondeterminism, that means self-consistency sampling, order-randomization testing, or constrained decoding rather than trusting a single pass. For silent knowledge/judgment gaps, that means mandatory retrieval grounding for time-sensitive facts, counterfactual fairness testing, and structured confidence elicitation rather than trusting the model’s own hedging language, which the model-behavior patterns show doesn’t reliably correlate with actual accuracy.
Frequently Asked Questions
Can the model just say when it’s uncertain or when a task is too complex for it?
Per Model Uncertainty Unawareness (GitHub), confident, declarative phrasing is simply a more common surface pattern in training data than hedged phrasing, and RLHF has historically reinforced confident phrasing because human raters tend to prefer complete-sounding answers. The model’s internal token-probability distribution does encode a form of uncertainty, but that isn’t reliably linked to the surface language it chooses to use.
Is instruction-following decay the same thing as style drift?
They share a root cause — a fixed-position system prompt losing influence as the transcript grows — but affect different things. Model Instruction Following Decay (GitHub) is about rule compliance (a forbidden topic, a formatting requirement); Model Style Drift (GitHub) is about persona and tone specifically. Both patterns document that periodic re-injection of the original instruction restores compliance temporarily before it decays again.
Can prompting alone fix reasoning inconsistency or refusal inconsistency?
No — both patterns show reasoning inconsistency and refusal inconsistency are properties of autoregressive sampling and learned decision boundaries, not something a single better-worded prompt eliminates. Model Reasoning Inconsistency (GitHub) recommends self-consistency sampling (multiple runs, majority vote) for high-stakes decisions; Model Refusal Inconsistency (GitHub) recommends a deterministic policy layer independent of the model’s own judgment for well-defined categories, rather than relying on the generative model as the enforcement point.
How much does context length actually have to fill before quality drops?
Model Context Length Behavior Change (GitHub) reports instruction-following degradation becoming noticeable once transcript length exceeds roughly 60-70% of the model’s advertised context window, with recall for mid-context facts running 20-40 percentage points below facts placed at the start or end (the “lost in the middle” effect).
Patterns
Total: 10 patterns