Legal Contracts

30 patterns in this category

Legal-contracts agents fail when they reason from stale information (outdated amendments, changed regulations), when they substitute similarity matching for jurisdiction-specific or legal-effect matching (wrong template retrieved, wrong jurisdiction’s rule applied), when they fail to verify that downstream agents actually received and acted on critical information (exceptions lost between agents, overruled citations cited as good law, negotiated values never reach the final draft), and when they stop short of cross-validating across related clauses and obligations (caps without evaluating carve-outs, presence-checking without scope evaluation, multi-party obligations tracked ambiguously). Legal-contracts failures are particularly consequential because the output of contract work is the contract document itself, which becomes binding and enforceable. Errors in agent analysis do not surface as obviously as model hallucinations — the contract reads as well-drafted and internally consistent, the risk assessment looks thorough, the compliance determination appears authoritative — until a dispute arises, a counterparty redlines, or a post-closing issue forces a re-read of what the agent missed or misunderstood.

Key Takeaways

  • 8 goals and 32 total patterns are documented here, spanning contract assembly (drafting, compliance), contract review (due diligence, risk detection), contract use (IP-rights, jurisdiction handling), and litigation support (discovery, depositions, privilege).
  • The largest single failure cluster is temporal staleness and legal currency (amendments, regulatory updates, superseded case law, stale templates, outdated disclosures) — 9 patterns across 4 goals document the same root condition: an agent reasoning from a fixed snapshot in time without a mechanism to detect that the snapshot has been superseded.
  • Multi-agent handoff failures are the second-largest cluster: 8 patterns across 5 goals document how a determination or limitation identified by an upstream agent (jurisdiction-specific exception, negotiated value, field-of-use restriction, risk flag, overruled citation status) never reaches a downstream agent because the structured interchange schema has no field to carry it.
  • Retrieval-substitution failures (wrong-jurisdiction template, wrong-effect template, wrong-version clause, wrong-subsidiary entity, stale regulatory template) recur across 6 goals — the recurring root condition is similarity-based ranking applied without jurisdiction, legal-effect, or canonicality filtering as a mandatory pre-filter.
GoalCoversPatterns
ComplianceAmendment tracking, regulatory update lag, wrong-jurisdiction disclosure templates, jurisdiction-specific exceptions4
Contract DraftingBoilerplate-negotiation conflicts, wrong-version clause retrieval, negotiated value loss at redline-assembly handoff, export verification4
Due DiligenceChange-of-control omission, correlation-as-causation in risk memos, unrelated-entity matching, risk-flag loss at review-summary handoff4
IP-RightsMechanism-category-jurisdiction mismatch in assignment clauses, OSS-license vs. assignment confusion, field-of-use scope loss at handoff, unverified clearance citations4
Jurisdiction HandlingGoverning-law misapplication, wrong-jurisdiction clause retrieval, cross-border data-transfer mechanism gaps3
Litigation SupportDiscovery vocabulary mismatch, positional bias in deposition summaries, privilege misidentification3
Precedent CurrencySuperseded case-law citations, circuit-split blindness, overruled-status loss at research-drafting handoff3
Risk DetectionAmbiguity misses, indemnification cap blindness, liability exposure blindness, multi-party obligation confusion, termination-clause misinterpretation5

Total: 32 patterns

How the Goals Relate

Legal-contracts goals span the full contract lifecycle and operate at two distinct layers. At the assembly and analysis layer, contracts are written, reviewed, and analyzed for compliance and risk: Drafting creates contracts from templates and negotiated terms; Compliance verifies filings against regulatory requirements; Due Diligence assesses target companies and risk; Risk Detection identifies obligations and exposure in existing contracts. At the usage layer, assembled or analyzed contracts are put into effect: IP-Rights and Jurisdiction Handling verify that contract terms are enforced as written; Litigation Support handles evidentiary and procedural aspects when disputes arise; Precedent Currency ensures legal reasoning stands on current authority.

To troubleshoot an error by symptom: if a contract contains wrong terms or conflicting boilerplate → Contract Drafting; if regulatory filing is non-compliant or uses wrong jurisdiction’s template → Compliance; if deal risk is understated or material risks are missed → Risk Detection or Due Diligence; if a dispute surfaces an overlooked amendment or misapplied legal rule → Jurisdiction Handling; if IP rights are unclear or usage unauthorized → IP-Rights; if litigation discovery is incomplete or privilege was waived → Litigation Support; if a brief relies on bad authority → Precedent Currency.

Frequently Asked Questions

What do amendment tracking, regulatory updates, and superseded case law have in common?

All three are temporal-staleness failures where an agent reasons from a fixed snapshot in time (original contract, training-data knowledge, published case law) without a mechanism to detect that the snapshot has been superseded (by an amendment, a regulatory change, or an overruling decision). The structural fix is the same for all three: add mandatory verification against a current, dated source before treating the determination as final.

How do multi-agent handoff failures recur across so many different goals?

Handoff failures happen when a schema-constrained interchange (a checklist, a structured citation list, a findings summary) has no field to capture a nuanced determination an upstream agent identified. The fix is the same across all: add required fields to the handoff schema for any determination that is material and requires downstream action (exceptions, scope limitations, overruled status, risk flags). Validate that all schema fields are populated before handoff.

What causes retrieval-based template selection to fail so consistently?

Retrieval by similarity works well when the candidate pool is homogeneous, but legal templates differ by jurisdiction and legal effect while sharing dense overlapping vocabulary. Similarity ranking surfaces “similar to my query” not “has the intended legal effect for my controlling jurisdiction.” The fix is mandatory pre-filtering: filter candidates by jurisdiction or legal effect before similarity ranking, never after.

How many of the 32 patterns would be caught by a more powerful LLM?

Very few. Most patterns are not reasoning errors but verification gaps: the agent could reason correctly if it had the right inputs (current amendments, current regulations, current citator status, correct template version, jurisdiction-filtered candidates), but it has no mechanism to verify those inputs are current or correct. Model capacity is not the limiting factor; verification architecture is.

Patterns by Failure Mechanism

Temporal Staleness (9 patterns): Amendment Tracking Failure, Regulatory Update Lag, Superseded Case-Law Citation, Clause Version Mismatch, Disclosure Template from Wrong Jurisdiction, Jurisdiction Clause Mismatch, OSS License vs. Assignment, Unverified Clearance Citation, Rendered Export Mismatch.

Multi-Agent Handoff Loss (8 patterns): Jurisdiction Exception Drop, Negotiated Value Drop, Risk Flag Drop, Field-of-Use Scope Drop, Overruled Citation Drop, Change-of-Control Risk Miss, Vocabulary Mismatch in Discovery, Privilege-by-Reference Miss.

Cross-Clause Interaction (7 patterns): Boilerplate-Negotiation Conflict, Indemnification Cap Blindness, Liability Exposure Blindness, Termination-Renewal Interaction, Contract Ambiguity Miss, Correlation as Causation, Entity Matching Mismatch.

Reasoning Scope (8 patterns): Multi-Party Obligation Confusion, Jurisdictional Application Error, Data-Transfer Mechanism Gap, Positional Bias, Circuit Split Blindness, IP Mechanism-Category Mismatch.

  • Document Processing — the upstream problem of turning legal documents into usable text without OCR, layout, or extraction errors that feed into contract analysis
  • Knowledge Retrieval — the cross-cutting RAG and retrieval problem that recurs across legal and all other domains
  • Reasoning and Thought — model degradation, version drift, and confidence miscalibration that compound with legal-domain-specific failures

Boilerplate Clause Misapplication

Frequency: Very Common
Category:

Agent Drafts a New Contract by Inserting a Standard Boilerplate Clause Library Entry That Is Inconsistent With Deal-Specific Terms Negotiated Elsewhere in the Same Document

Circuit Split Blindness in Citation Selection

Frequency: Common
Category:

Legal Research Agent Cites a Precedent as Settled Law Without Flagging That a Circuit Split (or Equivalent Jurisdictional Disagreement) Exists, Leading to a Brief That Misrepresents the Strength of the Authority

Embedding Retrieval Applies Wrong Jurisdiction's Clause Template by Name Similarity

Frequency: Common
Category:

A Drafting Agent Asked to Insert a Jurisdiction-Specific Clause (a Non-Compete, a Consumer-Arbitration Provision, a Statutory Notice) for a Contract Governed by One State or Country's Law Retrieves the Clause From a Multi-Jurisdiction Template Library Using Semantic Similarity Over the Clause's General Subject Matter, Rather Than Matching Strictly on Governing Jurisdiction, and Pulls a Differently-Jurisdictioned Template That Is Lexically Almost Identical but Legally Ineffective or Unenforceable Under the Contract's Actual Governing Law

Embedding Retrieval Applies Wrong Jurisdiction's Disclosure Template by Name Similarity

Frequency: Occasional
Category:

A Compliance Agent Assembling a Required Regulatory Disclosure for a Filing Retrieves the Applicable Disclosure Template From a Multi-Jurisdiction Template Library Using Semantic Similarity Over the Regulation's Name and Subject Matter, Rather Than Matching on the Controlling Jurisdiction Itself, and Pulls a Template Built for a Differently Named but Substantively Different Regulatory Regime in Another Jurisdiction That Happens to Share Closely Overlapping Terminology

Embedding Retrieval Pulls Generic OSS License as IP Assignment Template

Frequency: Occasional
Category:

A Contract-Drafting Agent's RAG Step, Asked to Retrieve the Company's Standard Work-for-Hire IP-Assignment Template for a New Contractor Agreement, Retrieves a Lexically and Semantically Similar but Substantively Different Document -- An Open-Source Contributor License Agreement or Inbound-IP Template -- Because Both Documents Share Dense "Intellectual Property," "Assignment," and "License Grant" Vocabulary

Embedding Retrieval Surfaces Similarly Named, Unrelated Subsidiary in Corporate-Structure Chart

Frequency: Occasional
Category:

A Due-Diligence Agent Building a Target Company's Corporate-Structure Chart From Filings and Registry Data, Using Semantic Similarity Search to Match Entity Names Across Documents, Merges or Links a Subsidiary Into the Target's Structure Based on Name Similarity Alone, When the Matched Entity Is in Fact an Unrelated Company With a Coincidentally Similar Name

Embedding-Retrieval Pulls Wrong Clause Version from Template Library

Frequency: Common
Category:

A Contract-Drafting Agent's RAG Step, Used to Pull the Firm's or Company's Approved Boilerplate Clause for a Given Section (Limitation of Liability, Indemnification, Governing Law) from the Template Library, Retrieves a Lexically Similar but Superseded or Jurisdiction-Wrong Version of the Clause, and the Drafted Contract Is Issued with the Wrong Terms

Indemnification Cap Blindness

Frequency: Common
Category:

Agent Flags an Indemnification Clause as Present Without Evaluating Whether It Is Capped, Uncapped, or Mutual

IP Assignment Gap in Contractor Agreements

Frequency: Common
Category:

Agent Reviewing a Contractor or Consultant Agreement Confirms "IP Assignment Clause Present" Without Verifying It Actually Covers the Work Product Category at Issue

Liability Clause Blindness

Frequency: Common
Category:

Model Fails to Flag Dangerous Liability/Indemnification Clauses; Exposes Organization to Unquantified Risk

Multi-Agent Handoff Drops Field-of-Use Limitation Between Clearance Agent and Licensing Agent

Frequency: Occasional
Category:

An IP-Clearance Agent That Reviews an Inbound License and Determines, in Its Own Narrative Analysis, That the Granted Right Is Limited to a Specific Field of Use or Product Line Hands Off Its Finding to a Downstream Licensing Agent Through a Structured "Rights Cleared: Yes/No" Field That Has No Place to Carry the Field-of-Use Limitation, So the Licensing Agent Treats the Right as Cleared for Any Use

Multi-Agent Handoff Drops Flagged Risk Between Review and Summary Agent

Frequency: Occasional
Category:

A Document-Review Agent in a Multi-Stage Due-Diligence Pipeline Identifies a Material Risk (a Change-of-Control Clause, an Undisclosed Litigation Reference, a Non-Standard Indemnification Carve-Out) Only in Its Own Free-Text Annotation of the Reviewed Document, and a Downstream Summary Agent That Generates the Diligence Memo from a Structured Findings List Never Sees It

Multi-Agent Handoff Drops Jurisdiction-Specific Exception Between Compliance-Review and Filing Agent

Frequency: Occasional
Category:

A Compliance-Review Agent That Determines, in Its Own Narrative Analysis, That a Filing Qualifies for a Jurisdiction-Specific Exception to a General Disclosure Requirement Hands Off to a Filing Agent Through a Structured Checklist That Has No Field for the Exception, So the Filing Agent Applies the General Requirement the Exception Was Meant to Override

Multi-Agent Handoff Drops Negotiated Deviation Between Redline Agent and Final-Assembly Agent

Frequency: Occasional
Category:

A Redlining Agent That Negotiates a Deal-Specific Deviation From Standard Contract Terms in Its Own Turn-by-Turn Conversation With Counterparty Counsel Hands Off the Negotiated Document to a Separate Final-Assembly Agent Through a Structured "Accepted Clause Set" Record That Captures Only the Clause IDs Used, Not the Specific Negotiated Variable Within Each Clause, So the Assembly Agent Reinserts the Clause's Standard Default Variable Instead of the Negotiated One

Multi-Agent Handoff Drops Overruled-Citation Flag Between Research and Drafting Agent

Frequency: Occasional
Category:

A Legal-Research Agent That Identifies, in Its Own Analysis, That a Specific Case It Surfaced Has Been Overruled on the Exact Point It Was Going to Be Cited For Hands Off a Structured List of Candidate Citations to a Drafting Agent That Strips That Overruled-Status Context, So the Drafting Agent Cites the Case as If It Were Still Good Law

Positional Bias Omits Mid-Document Admission in Deposition Summary

Frequency: Occasional
Category:

Agent Summarizing a Long Deposition Transcript or Produced Document Systematically Under-Weights Content in the Middle of the Document, Omitting a Material Admission That Was Fully Within the Model's Context Window and Correctly Classified as Relevant

Rendered Export Not Verified Against Edited Clause Text

Frequency: Occasional
Category:

A Contract-Drafting Agent Edits a Clause Correctly in the Working Draft, Reports the Edit as Applied, and Sends the Document for Signature Without Verifying That the Final Rendered/Exported Document Actually Reflects the Edited Text Rather Than a Stale Merge Artifact

Superseded Case-Law Citation

Frequency: Common
Category:

Agent Cites a Case as Controlling Precedent Without Detecting That It Has Been Overturned, Limited, or Superseded by Later Authority

Termination Clause Misinterpretation

Frequency: Common
Category:

Agent Misreads the Interaction Between "Termination for Convenience" and "Termination for Cause" Notice Periods, Survival Clauses, and Post-Termination Obligations

Unverified Clearance Opinion Filed Without Checking Cited Clause Against Source Agreement

Frequency: Occasional
Category:

An IP-Clearance Agent Asked to Confirm a Company Holds Sufficient Rights to Use a Third-Party Asset (a Licensed Image, a Vendor-Supplied Code Library, a Co-Developed Patent Disclosure) Generates a Clearance Opinion That Quotes a Specific License Clause as Granting the Needed Right, Then Autonomously Files or Releases That Opinion to the Requesting Team Without Re-Reading the Actual Source License Text It Just Cited to Confirm the Quoted Clause Says What the Opinion Claims It Says