Context-Window Truncation Drops Early-Session Client Constraint

Issue: A Hard Constraint the Client States Early in a Long, Multi-Turn Advisory Session (e.g., “No Fossil-Fuel Holdings,” “No Leveraged Products”) Falls Outside the Model’s Effective Context Window by the Time a Later-Turn Recommendation Is Generated, and the Agent Recommends a Security That Violates It Without Re-Checking Against the Original Constraint List

Frequency: Occasional

Symptoms

• The client states an explicit exclusion or hard constraint in an early turn of a session that goes on to involve many tool calls (market data lookups, multiple draft recommendations, back-and-forth refinement)
• A recommendation generated late in the same session violates the early constraint, even though the client never rescinded it and no later turn contradicts it
• Asking the agent, in the same late turn, “what are this client’s stated constraints?” produces an incomplete list that omits the early one, even though it was clearly stated
• The constraint reappears correctly if the session is restarted with a shorter history, or if the constraint is restated closer to the point of recommendation — isolating the failure to context position rather than the model’s general capability to honor the constraint
• The omission is silent: nothing in the agent’s output flags that a constraint might have been dropped or asks the client to reconfirm exclusions before a late-session recommendation

Root Cause Long agentic sessions accumulate tool-call results, intermediate reasoning, and draft outputs between the point a constraint is stated and the point a final recommendation is generated. When the accumulated transcript exceeds what the model attends to with uniform reliability, information stated early is more likely to be underweighted or dropped relative to more recent turns — a positional recall degradation distinct from the model simply “not knowing” the constraint, since it can recall it correctly when queried immediately after it was stated. Because portfolio constraints are typically stated once, at intake, and then never repeated, they are structurally the most likely piece of client-specific information to fall into this degraded region by the time a recommendation is produced many turns later in the same session.

Example

Turn 1 (client): "Before we start, I want to be clear: no fossil-fuel 
producers or fossil-fuel-heavy utilities in this account, under any 
circumstances."

Turns 2-40: extended session covering risk tolerance discussion, multiple 
sector analyses, several tool calls to a market-data and screening service, 
two rounds of draft allocations for the equity sleeve, client feedback on 
each draft.

Turn 41 (agent, generating the finalized energy-sector allocation):
"For diversified energy exposure, I recommend a 4% allocation to 
[integrated oil & gas major]."

Reality: this directly violates the Turn 1 constraint. Asked at Turn 41 
to list the client's stated exclusions, the agent's answer omits the 
fossil-fuel exclusion entirely.

Key Statistics

Eval Recipes

Test Cases

Evaluation Dataset

• Source: Synthetic advisory sessions constructed by inserting a real client exclusion at a fixed early position and varying the number and content of intervening turns (tool calls, draft revisions, unrelated Q&A) before requesting a final recommendation
• Size: 80+ sessions, stratified by intervening-turn count and by exclusion type (sector exclusion, product-type exclusion, single-issuer exclusion)
• Key variations: exclusion position (turn 1 vs. turn 10 of a 40-turn session), presence/absence of a mid-session restatement, and whether intervening turns are topically related to the exclusion (e.g., discussing energy sector) or unrelated

Metrics

Automated Checks

def check_for_failure(session_transcript, final_recommendation, stated_constraints):
    """Flag a final recommendation that violates a constraint stated
    earlier in the same session and never rescinded.
    """
    active_constraints = [
        c for c in stated_constraints
        if not any(c["id"] in turn.get("rescinds", []) for turn in session_transcript)
    ]

    violated = [
        c for c in active_constraints
        if c["exclusion_check"](final_recommendation)
    ]

    return {
        "active_constraint_count": len(active_constraints),
        "violated_constraints": [c["id"] for c in violated],
        "constraint_dropped": len(violated) > 0,
    }

Mitigation Strategies

Prevention

1. Persistent Constraint Ledger Outside the Rolling Context: Extract client constraints at intake into a structured, external ledger that is re-injected verbatim into every recommendation-generation prompt, rather than relying on the model to recall them from earlier in a long rolling transcript.
2. Pre-Recommendation Constraint Recheck: Before finalizing any recommendation, run a mandatory step that re-fetches the full constraint ledger and explicitly checks the candidate recommendation against each entry, independent of the free-form conversation history.
3. Session-Length-Aware Restatement Prompting: For sessions exceeding a turn-count threshold, automatically re-surface the client’s stated constraints into the active context before generating any recommendation.

Detection & Response

1. Constraint-Violation Screening: Independent of the agent, run every finalized recommendation through a deterministic screen against the client’s stored constraint ledger before it is presented to the client or executed.
2. Late-Session Recall Testing: Periodically probe long-running sessions by asking the agent to list all stated client constraints, and flag any session where the answer omits a constraint known to be in the ledger.
3. Constraint-Age Monitoring: Track how many turns have elapsed since each constraint was last referenced in-context, and flag recommendations generated when a constraint’s last reference exceeds a defined turn-distance threshold.

Architecture Patterns

• External Constraint Store, Not In-Context Memory: Client constraints live in a structured store queried fresh at recommendation time, not solely in the conversational transcript the model attends over.
• Deterministic Pre-Delivery Gate: A non-LLM screening stage between “recommendation drafted” and “recommendation delivered” that checks the draft against the full constraint ledger and blocks delivery on any violation.
• Turn-Distance-Triggered Re-Injection: Orchestration logic that re-injects constraint text into the prompt whenever the distance since last mention exceeds a threshold, rather than trusting uniform recall across the whole session.

Key Metrics

Alerts

References

• Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework
• MAGPIE: A Benchmark for Multi-Agent Contextual Privacy Evaluation