5 patterns for this goal
Retention-prediction agents fabricate conversational details in risk-score narratives that are not grounded in any source record, use stale industry-attrition benchmarks instead of querying live cohort-comparison tools, and anchor risk scores to mismatched historical-employee analogs selected by name or description similarity rather than by structured risk-factor alignment. The most damaging failure is self-fulfilling feedback: once an attrition-risk score is visible to managers, those managers deprioritize the flagged employee for growth opportunities, causing the predicted attrition to materialize as a consequence of the prediction itself rather than the pre-existing risk. These patterns cluster around narrative grounding, benchmark staleness, retrieval mismatches, and score visibility feedback loops.
All five retention-prediction patterns share a vulnerability in how agents bridge from structured risk signals to human-readable justification and action. The underlying risk model may correctly identify structured signals (tenure plateau, compensation percentile, manager-change history), but the narrative explanation layered on top to justify the score to HR business partners is not bound to those signals. When a narrative is generated with access to contextual information (news events, prior interactions) not actually used as model inputs, or when a risk score’s baseline benchmark is swapped from a live tool to parametric knowledge, or when a cohort analog is selected by textual similarity instead of structural risk-factor alignment, the confidence and fluency of the score’s presentation masks these divergences. Mitigation requires separating decision (structured features) from justification (narrative), grounding every narrative claim in an actual source document or model-attribution result, mandating live tools for decision-relevant baselines, and testing whether manager behavior on flagged employees actually addresses the cited risk factors or simply treats the flag as a deprioritization signal.
Require every specific factual claim in a risk narrative (quotes, dated events, manager-conversation details) to cite the source document or note ID it came from. Strip or flag any claim that cannot be traced to a retrievable source. When free-text manager notes were not actually retrieved for an employee, explicitly state the score is based on structured signals alone; do not fabricate qualitative color.
A stale benchmark is a single number (industry average attrition, 12%) that’s outdated relative to live data. A mismatched cohort is a single departed employee selected as an analog (because their name or role title resembles the current employee) who actually departed for different reasons. Both produce wrong relative-risk characterizations; different detection methods apply (numeric comparison for benchmarks, structured-attribute comparison for cohorts).
Run a score-visibility holdout experiment: withhold the risk score from managers for a randomized control group and compare actual attrition outcomes. If the visible-score group has higher attrition, the difference is attributable to manager behavior influenced by the score, not just the model’s prediction accuracy. This requires deliberate experimental design but is the only way to isolate feedback-loop effects.
Immediately trigger a manual re-score using current compensation data before any retention intervention is recommended to the employee. If a retention conversation was already scheduled, surface to HR that the compensation-related risk driver was already addressed and reframe the conversation around any remaining risk factors.
When a justification-generation step has access to broad contextual information (news events, organizational changes) not used in the underlying risk model, it constructs plausible-sounding causal links between co-occurring elements. The risk model may have flagged tenure plateau as the driver, but if a regional news event co-occurs in time and geography, the narrative can fluently connect them without any actual dependency. Fix: require narrative claims to cite specific model-attributed features, not contextual background information.
| Pattern | Mechanism |
|---|---|
| Retention Agent Fabricates Manager-Conversation Detail Not Present in Any Source Note | Narrative explanation generates specific conversation details (quotes, dated 1:1 notes) not grounded in any actual manager note or survey data retrieved |
| Stale Training-Corpus Industry-Attrition Benchmark Overrides Live Cohort Tool | Relative-risk contextualization uses generic industry average instead of calling live internal cohort-comparison tool, producing mischaracterized peer comparison |
| Embedding Retrieval Pulls Mismatched Historical-Attrition Cohort as Comparable | Retrieved departed-employee analog is selected by name/role-title similarity but differs in actual attrition drivers (relocation vs. compensation vs. management change) |
| Multi-Agent Handoff Drops Confirmed Comp Adjustment Before Retention-Risk Rescoring | Compensation-review agent confirms off-cycle pay adjustment but handoff to retention agent lacks updated compensation data; score still cites below-market pay as risk driver |
| Attrition-Risk-Score Feedback Loop Self-Fulfilling | Employees flagged high-risk are deprioritized for growth opportunities by managers, causing actual attrition as a consequence of manager behavior, not original risk factors |
Total: 5 patterns
Employees Flagged as High Attrition-Risk by the Retention-Prediction Model Are Systematically Deprioritized for Growth Opportunities, Stretch Assignments, and Promotion Consideration by Managers Aware of the Score, Causing the Flagged Employees to Actually Leave at Higher Rates as a Consequence of the Flag Itself
A Retention-Prediction Agent's Embedding Search over Past Departed-Employee Records Surfaces a Lexically Similar but Substantively Different Attrition Case as the Nearest-Neighbor Comparable, and the Agent Anchors a Current Employee's Risk Score to That Mismatched Precedent
A Compensation-Review Agent That Confirms a Manager-Approved Off-Cycle Pay Adjustment for an Employee Hands Off Its Output to a Downstream Retention-Prediction Agent Through a Structured Schema That Has No Field for a Pending or Just-Approved Comp Change, So the Retention-Prediction Agent Computes the Employee's Updated Attrition-Risk Score From Stale Compensation Data and Continues Flagging Them as High-Risk Even Though the Underlying Driver of That Risk Was Already Resolved
A Retention-Prediction Agent Asked to Produce a Narrative Justification for a High-Attrition-Risk Flag Generates a Specific, Plausible-Sounding Detail About a Recent One-on-One Conversation Between the Employee and Their Manager (e.g., "The Employee Told Their Manager They Were Frustrated With the Lack of Promotion Timeline") That Does Not Appear in Any Manager Note, Survey Response, or HRIS Record the Agent Had Access To, and HR Acts on the Fabricated Detail as if It Were Documented Evidence
A Retention-Prediction Agent, When Asked to Contextualize Whether an Employee's Computed Risk Score Is High Relative to Peers, Answers Using a General Industry Attrition-Rate Figure It Absorbed During Pretraining Rather Than Calling the Live Internal Cohort-Comparison Tool Available to It, Producing a Relative-Risk Characterization Anchored to an Outdated or Generic Benchmark Instead of the Company's Actual, Current Department-Level Attrition Baseline