Discovery Document Relevance Misclassification
Litigation-Support Agent Marks Materially Relevant Documents as Non-Responsive (or Vice Versa) During Discovery Review, Creating Spoliation or Over-Production Risk
3 patterns for this goal
Litigation-support failures concentrate on three points in the evidence-handling pipeline: classifying discovery documents for relevance when vocabulary shifts (code names, abbreviations, terminology drift over time) cause the classifier to miss genuinely responsive documents; omitting material admissions from deposition summaries due to positional bias (mid-document content is systematically under-weighted); and failing to identify privileged communications when they arrive through indirect channels (forwarded threads, in-house counsel without legal-domain email addresses, business executives summarizing legal advice). Unlike contract-analysis failures, litigation-support failures bear directly on case outcomes and ethics rules: missed responsive documents create spoliation exposure, omitted admissions undermine case preparation, inadvertent privilege waivers destroy attorney-client protection and create waiver-related sanctions. Because the outputs (discovery classifications, deposition summaries, privilege logs) are themselves litigation artifacts, errors in the agent’s work become discoverable and attackable by opposing counsel.
Litigation-support failures are particularly high-stakes because litigation artifacts themselves carry legal consequences and ethical obligations. All 3 patterns share a dependence on heuristics or single-pass classification that omit or misclassify material content: vocabulary heuristics miss terminology shifts, positional weighting misses middle-document content, metadata filters miss indirect privilege channels. Unlike contract-analysis failures (which often surface only in disputes years later), litigation failures can be audited and attacked during discovery itself. The mitigation across all three is to pair automated classification with iterative human review: active-learning vocabulary discovery for discovery classification, overlap-chunked summarization with deterministic sweep verification for depositions, and multi-pass attorney sampling with explicit false-negative rate tracking for privilege.
Before classification runs, conduct vocabulary discovery: custodian interviews, document sampling, and codename-identification to build a codename-to-standard-term map. Seed the classifier with expanded vocabulary. After classification, sample borderline cases (confidence 40-60%) plus a random 5% for attorney review, feeding corrections back to the classifier iteratively until misclassification rates stabilize — see Discovery Document Relevance Misclassification.
The documented mitigation is to split long documents into overlapping chunks smaller than the length at which positional degradation occurs, summarize each chunk independently, then merge chunk summaries — the chunked approach prevents any passage from being structurally disadvantaged by its position in the document. Additionally, run a deterministic keyword/entity sweep across the entire document in parallel, and flag any sweep hit not in the summary for attorney review — see Positional Bias Omits Mid-Document Admission in Deposition Summary.
Use purpose-driven classification, not metadata-only heuristics: scan all documents for advice-seeking language and legal-risk content regardless of sender; identify privilege-by-reference (business communications relaying legal advice), work product (litigation strategy memos), and litigation-preparation language. Then mandatory multi-pass attorney sampling: sample 10%+ of agent-classified-as-non-privileged documents, have licensed attorney review, calculate false-negative rate. If >2%, expand sample or re-classify entire set. Before production, require attorney sign-off on privilege log with specific, defensible basis statements — see Privilege Waiver Risk in AI-Assisted Document Review.
| Pattern | Mechanism |
|---|---|
| Discovery Document Relevance Misclassification | Vocabulary mismatch from code names and abbreviations causes classifier to miss responsive documents using internal terminology |
| Positional Bias Omits Mid-Document Admission in Deposition Summary | Long-document summarization systematically under-weights middle-third content, omitting material admissions despite full in-context availability |
| Privilege Waiver Risk in AI-Assisted Document Review | Metadata-heuristic privilege classification misses privilege-by-reference and work-product protected communications arriving through indirect channels |
Total: 3 patterns
Litigation-Support Agent Marks Materially Relevant Documents as Non-Responsive (or Vice Versa) During Discovery Review, Creating Spoliation or Over-Production Risk
Agent Summarizing a Long Deposition Transcript or Produced Document Systematically Under-Weights Content in the Middle of the Document, Omitting a Material Admission That Was Fully Within the Model's Context Window and Correctly Classified as Relevant
Agent Performing E-Discovery Document Review Includes a Privileged Document in a Production Set Because It Was Not Recognized as Attorney-Client Privileged or Work-Product Protected