No Policy Mapping

Issue: Agent behavior not mapped to company/regulatory policies.

Frequency: Common

Symptoms

• Compliance review cannot trace control coverage.
• A regulatory audit asks “which control satisfies obligation X” and no one can answer without a multi-week manual investigation.
• A new agent capability launches and later turns out to violate a data-handling policy that was never checked against it.
• Compliance and engineering disagree about whether a given control is “covered,” because no shared source of truth exists.

Root Cause Capabilities ship based on whether they work, not whether they’ve been checked against what they’re legally or contractually obligated to do, because launch review has no policy-obligation checklist step at all. Compounding this, the set of applicable regulations isn’t centrally tracked, so a team may not even know an obligation like HIPAA minimum-necessary applies to their feature, and with no control matrix linking each policy to the specific technical control that satisfies it, compliance coverage can only be verified through manual investigation. Legal tracks regulatory change, but nothing systematically propagates that change into engineering as a new control requirement, so the mapping — where it exists at all — goes stale as the regulatory landscape shifts.

Example

A healthcare intake agent is expanded to summarize patient-reported
symptoms into structured notes for clinicians. The team ships the
feature because it passes functional testing; no one checks it against
HIPAA minimum-necessary or data-handling obligations, because no
policy-to-control matrix exists to consult.

Six months later, a compliance audit asks the engineering team to
demonstrate which technical controls enforce minimum-necessary access to
the summarized notes. No one can produce a mapping — the feature was
never registered against any policy obligation in the first place.

The audit finding requires a retroactive control review and a temporary
feature freeze while the mapping gap is closed, delaying an unrelated
product launch that depended on the same compliance sign-off.

Contributing Factors

• New agent capabilities ship based on functional testing alone, with no policy-obligation checklist as part of launch review.
• The set of applicable policies/regulations is not centrally tracked, so teams may not know which obligations even apply to a given feature.
• No control matrix exists linking each policy to the specific technical control that satisfies it, so compliance coverage can’t be verified without manual investigation.
• Regulatory changes are tracked by legal but not systematically propagated to engineering as new control requirements.

Eval Recipes

Test Cases

Metrics

Mitigation Strategies

Prevention

1. Policy-to-Behavior Control Matrix: Build and maintain an explicit mapping from each applicable policy/regulation (internal data handling policy, GDPR, sector-specific regulation) to the specific agent controls that implement it (e.g., “PII minimization policy” maps to the redaction proxy and retention TTL controls). No agent capability ships without an entry showing which policy obligations it satisfies or which are not yet covered.
2. Policy Coverage Gate at Launch Review: Require new agent capabilities or expanded scopes to pass a launch review checklist confirming every applicable policy has a corresponding control in the matrix, with explicit sign-off from legal/compliance for any gap accepted as a known risk.
3. Regulatory Change Intake Process: Establish a process where legal/compliance changes (new regulation, updated internal policy) trigger a review of the control matrix to identify newly uncovered obligations, rather than the mapping going stale as the regulatory landscape shifts.

Detection & Response

1. Control Coverage Audit: Periodically run an audit comparing the current policy matrix against the agent’s actual deployed capabilities, flagging any capability with no mapped policy control or any policy with no corresponding implemented control.
2. Compliance Traceability Query Tool: Provide compliance reviewers self-service tooling to query “which control implements policy X” and “which policies does capability Y satisfy,” so gaps surface during routine review instead of only during an audit or incident.
3. Unmapped Behavior Detection: Monitor production agent behavior for actions or capabilities that exist in the runtime but have no entry in the policy matrix (indicating the matrix wasn’t updated when the capability was added), and flag these as compliance debt.

Architecture Patterns

1. Policy Control Registry: Maintain a structured, versioned registry (e.g., a compliance-as-code repository) mapping policy_id → control_id → implementing_component → verification_evidence, queryable by both engineering and compliance teams.
2. Compliance Review Integration in CI/CD: Wire a check into the deployment pipeline that blocks shipping a new agent capability unless it references an entry in the policy control registry (or an explicit waiver signed by compliance).
3. Automated Evidence Collection: Link each control in the registry to automated evidence (audit log queries, config snapshots, test results) that demonstrates the control is actually operating, so compliance reviews are backed by live evidence rather than a static document.

Metrics

1. policy_coverage_percent: Target: 100% of applicable policies mapped to a control; Alert threshold: < 95%
2. unmapped_capability_count: Target: 0; Alert threshold: > 0 deployed capabilities without a policy mapping
3. control_matrix_staleness_days: Target: reviewed within 30 days of any regulatory/policy change; Alert threshold: > 90 days since last review
4. compliance_gap_waivers_open: Target: tracked and time-boxed; Alert threshold: any waiver open past its expiry

Alerts

1. Unmapped Capability Deployed (P1 - Critical): Condition - agent capability shipped to production with no entry in the policy control matrix. Action: Block further rollout, require emergency compliance review and retroactive mapping.
2. Regulatory Change Without Matrix Update (P2 - Warning): Condition - a tracked regulatory/policy change has no corresponding control matrix update after 30 days. Action: Escalate to compliance lead, prioritize gap analysis.
3. Compliance Waiver Expired (P3 - Info): Condition - a time-boxed gap waiver passes its expiry without remediation. Action: Notify capability owner and compliance, consider capability suspension until closed.

Production Signals

Key Metrics

Alerts

References

• NIST-AI-RMF
• Note: Govern, map, measure, manage framework for AI risk.