Missing PII Detection Framework

Issue: Team relies on ad-hoc regex or manual review for PII detection/redaction instead of adopting an established, maintained framework, missing entity types and edge cases the framework would catch by default.

Frequency: Common

Symptoms

• PII redaction logic is a small set of hand-written regexes (email, phone, SSN patterns) rather than an NER-plus-pattern-plus-checksum pipeline
• Detection misses common entity types (addresses, dates of birth, financial account numbers, medical record numbers) that a maintained NER model would flag by default
• Regex patterns show high false-negative rates on non-US formats (international phone numbers, IBANs, non-US national ID numbers) because they were only ever tested against US-centric samples
• No confidence scoring exists on matches, so redaction is all-or-nothing: a pattern either fires and masks the text, or the entity silently passes through unredacted
• Redaction logic is duplicated across services (one regex set at ingestion, a different ad-hoc filter before logging) with no shared library, so coverage silently drifts between them
• PII exposure is discovered only through customer complaints or manual spot-checks, not through proactive scanning of a representative sample of stored transcripts

Root Cause PII redaction was treated as a quick regex task handed to whichever engineer had spare time, rather than routed through security or compliance for a genuine build-vs-buy decision against established detection frameworks. Deadline pressure led to copying a handful of generic patterns rather than mapping detection coverage against the organization’s actual data classification policy, and because compliance requirements were captured only as a high-level “PII must be redacted” checkbox, they were never translated into a concrete list of entity types the detector needed to cover. With no single owner responsible for the redaction pipeline, coverage was never revisited as new document formats and locales were onboarded, so entire categories of identifiers went unchecked for months without any confidence-scoring mechanism that could have surfaced the blind spot earlier.

Example

A fintech onboarding pipeline uses an LLM agent to extract and summarize applicant
identity documents before handing structured fields to a KYC service. To keep raw
document text out of long-term logs, an engineer wrote three regexes to redact
email, US phone numbers, and 9-digit SSNs before the extracted text was persisted
to the audit log store.

Six months later a compliance audit sampled stored logs and found unredacted
UK National Insurance numbers, passport numbers, and full home addresses sitting
in plaintext across thousands of records - none of which the original regex set
was ever built to catch, since the team had only tested against US-formatted
sample documents. There was no confidence scoring or entity-type inventory to
reveal the gap earlier; the regexes either matched or didn't, with no signal
that whole categories of identifiers were never being checked at all. The
company had to notify affected users in three jurisdictions and rebuild the
redaction layer under regulatory deadline pressure, work that an established
PII detection framework's default entity library would have covered from day one.

Contributing Factors

• No evaluation of established open-source PII frameworks (analyzer + anonymizer + confidence scoring) was done before building custom regex-based detection in-house
• PII redaction was treated as “a quick regex task” and assigned to whichever engineer had spare time, rather than routed through security/compliance for a build-vs-buy decision
• Shipping deadline pressure led to copying generic regex snippets rather than mapping detection coverage against the org’s actual data classification policy
• No single owner was assigned to the redaction pipeline, so entity coverage was never revisited as new document formats and locales were onboarded downstream
• Compliance requirements were captured only as a high-level “PII must be redacted” checkbox, never translated into a concrete list of entity types the detector needed to cover

Eval Recipes

Test Cases

Metrics

Mitigation Strategies

Prevention

1. Adopt Microsoft Presidio (or equivalent maintained framework): Replace hand-written regex with an analyzer that combines NER models, pattern matching, and checksum validators (e.g., Luhn for card numbers), each returning a confidence score, plus an anonymizer for consistent masking/hashing/redaction.
2. Run a build-vs-buy evaluation before new PII work: Before extending any regex-based detector, compare its entity coverage against the org’s actual data classification/compliance entity list and against a maintained framework’s default coverage.
3. Centralize redaction into one shared library/service: Route ingestion, logging, and prompt-assembly redaction through a single canonical detector instead of letting each service maintain its own copy.

Detection & Response

1. Scheduled sampling audits: Pull a random sample of stored transcripts/logs weekly and re-run them through the canonical detector to catch entities the legacy ad-hoc method missed.
2. Confidence-score review queue: Route low-confidence detections to human review instead of silently passing them through or silently blocking them.
3. Incident postmortems: Treat any customer-reported or audit-discovered PII exposure as a trigger to identify which entity type/locale/format was missed and add a corresponding regression test case.

Architecture Patterns

1. Analyzer + anonymizer separation: Separate detection (entities + confidence scores) from the redaction action (mask/hash/redact) so redaction policy can be tuned per entity type without touching detection logic.
2. Defense-in-depth redaction points: Redact at ingestion, before storage, and again before model input/prompt assembly and logging - not just at a single chokepoint.
3. Shadow-mode migration: Run the new framework alongside the legacy regex in shadow mode, diff outputs on real traffic, and only cut over once recall parity (or improvement) is confirmed.

Metrics

1. pii_recall_rate: Target: >= 95%; Alert threshold: < 90%
2. unredacted_entity_incidents_per_month: Target: 0; Alert threshold: >= 1
3. entity_type_coverage_ratio: Target: 100% of compliance-mandated entity types; Alert threshold: < 90%

Alerts

1. Unredacted PII Detected in Stored Data (P1 - Critical): Condition - audit scan finds a compliance-listed entity type present unmasked in stored transcripts/logs. Action: page on-call security, quarantine affected records, open incident review.
2. PII Recall Regression (P2 - Warning): Condition - weekly benchmark recall drops more than 5 points from baseline. Action: block deployment of detector changes, notify pipeline owner.
3. Entity Coverage Gap on New Source (P3 - Info): Condition - a new document format or data source is onboarded without a recorded PII framework coverage check. Action: notify data platform team to run a coverage check before go-live.

Production Signals

Key Metrics

Alerts

Related Patterns

• PII Field Exposure - the downstream symptom (PII actually exposed); this pattern is the upstream root cause of not adopting a proven detection framework in the first place
• PII Field Leakage in Responses - a related downstream leakage symptom this pattern’s missing framework would help prevent

References

• Microsoft Presidio: PII Detection Guide 2026 - open-source framework combining NER, regex, and checksum validation with confidence scoring, plus an anonymizer for redaction/masking/hashing
• Preventing PII leakage when using LLMs: An introduction to Microsoft’s Presidio - recommended pipeline placement: analyze input, anonymize, retrieve, redact chunks, assemble prompt, redact output before storing the trace
• The complete guide to PII detection and redaction tools for AI pipelines in regulated industries - survey of PII tooling options for regulated-industry AI pipelines