# Policy Drift Checker: An AI Agent That Tests Agent Rails Against Reality

The Policy Drift Checker is an AI agent that compares rail cards with what the audit ledger shows agents actually did on a customer data platform (CDP). AI governance leads run it at read level (L1), on metadata only. It reports differences and names the rail owner to notify. It never changes a rail, pauses an agent or reads customer payloads.

In the [AI agent rails](https://privycdp.org/ai/rails) method, the card is the claim and the ledger is the evidence. This agent tests one against the other, and the [compliance, privacy and security playbook](https://privycdp.org/ai/rails/functions/compliance-privacy-security) places it in the wider function.

## Rail at a glance

The Policy Drift Checker's rail allows metadata alone, so it can compare what a run did without seeing what the run touched.

| Field | Value |
|---|---|
| Job | Compare rail cards with what the ledger shows agents actually did. |
| Trigger | A schedule, or a rail or policy change. |
| Data scope | Rail cards, ledger metadata and policy versions, with no payloads. |
| Tool scope | Read: rail cards, ledger metadata, policy versions; propose: no changes; execute: no actions, and no pause tool. |
| Output contract | Differences found, severity, and the rail owner to notify. |
| Gate | None for reports, because the rail owner decides any change. |
| Limits | Ledger metadata only, under a set budget per run. |
| Ledger | Rail versions compared, findings, owner notified. |
| Evaluation | Seeded out-of-scope tool calls and undocumented changes. |
| Owner and kill switch | AI governance lead; stopping the schedule leaves the ledger recording as before. |

## What it does and who uses it

The Policy Drift Checker tests a simple claim: the card says one thing, and the ledger should show the same. Infrastructure teams know the idea from configuration drift, where Terraform compares recorded state with real resources [2](#source-2).

Illustrative run: a weekly job compares last week's ledger metadata with each rail card in force, and it raises three findings. A [Reporting Summariser](https://privycdp.org/ai/rails/agents/reporting-summariser) run called a query ID outside its fixed list, rated high. The [Segment Drafter](https://privycdp.org/ai/rails/agents/segment-drafter) rail shows a new model version with no regression pass on record. The [Journey QA Reviewer](https://privycdp.org/ai/rails/agents/journey-qa-reviewer) rail has had no named owner for three weeks. A notification service tells each rail owner, and the checker takes no action. Any demotion comes from written triggers in the [control plane](https://privycdp.org/ai/rails/control-plane), not from this agent.

Most of the comparison is set arithmetic, so code should do it and the model should only explain the result. The mapping below is illustrative.

| Rail card says | Ledger shows | Finding | Computed by |
|---|---|---|---|
| Tool scope lists these tools | A call to another tool | Out-of-scope call | Code |
| Gate binds approval to a hash | An execution with no hash, or a different one | Gate bypass | Code |
| Limits cap runs or size | Counts above the cap | Limit breach | Code |
| A regression pass precedes a model change | New model version, no pass on record | Undocumented change | Code |
| A named owner holds the rail | Owner field empty or vacated | Ownerless rail | Code |
| One job | Request types spreading beyond that job | Possible job creep | Model, as a hypothesis |

## Why this level

Reading metadata and writing a report leaves production alone, so a wrong finding costs attention and nothing more. Read level (L1) is where the Policy Drift Checker stays.

The Rule of Two [1](#source-1) count is short. Values in ledger metadata can originate with the agents under review, so treat them as untrusted input. The session holds no payloads and has no way to change state, which leaves one property of three.

It will never gain a pause button. A checker that can stop what it checks puts detection and enforcement in one place, so one bug or hijack could halt every rail at once. OWASP's agentic list names tool misuse and rogue agents among its risks [3](#source-3), and an independent witness helps expose both. Keep enforcement in the systems that already hold it, and see the [rail levels](https://privycdp.org/ai/rails/rail-levels-autonomy-ladder) for how demotion triggers work.

## Three ways it fails, and the control for each

Drift checks are only as honest as the record they read.

| Failure | What it looks like | Control |
|---|---|---|
| Blind to what was never logged | A tool call bypasses the ledger writer, so the comparison looks clean. | Each report states its coverage. The platform reconciles gateway counts with ledger counts and writes the result to ledger metadata, and any mismatch is rated high. |
| Card edited to match behaviour | A run under version 3 is tested against version 4, or an owner widens a card so a finding disappears. | Code joins each run to the card version in force by timestamp. Card changes need sign-off, and a widening that follows a finding is itself reported. |
| Findings with no consequence | Reports arrive, nobody acts, and old findings fade from view. | Findings stay open against the rail until its owner records a decision. Aged items go to the AI governance lead, grouped by rail and role, not by person. |

## How to evaluate it

Drift detection can only be scored against drift you planted, so seed it deliberately. Write seeded entries to a test ledger, since a production ledger is append-only and should stay free of fake runs.

- **Seeded drift found,** broken down by out-of-scope call, gate bypass, limit breach and undocumented change.
- **Time from drift to report,** measured from the seeded event's timestamp.
- **False findings at version boundaries.** Include runs that straddle a card change, where naive comparisons go wrong.
- **Audit coverage,** the share of gateway calls with a matching ledger entry.
- **Decision age,** how long findings wait for an owner, reported per rail.

Someone else should check the checker, because no agent should mark its own work. A person samples its ledger entries against the seeded results.

## Where it runs

Ledger metadata carries no customer payloads, so contract terms decide whether an external model is acceptable. Rail cards and tool names still reveal your control design, which favours a private model in regulated teams, as the [boundary architecture patterns](https://privycdp.org/ai/ai-agents-on-customer-data-inside-your-boundary) explain. [PrivyCDP](https://privycdp.org/cdp) provides an immutable audit ledger that names the authorising actor on each AI-assisted action, which gives such a comparison something firm to read, but the checker is not part of the product.

## Related

- [AI agents for compliance, privacy and security teams](https://privycdp.org/ai/rails/functions/compliance-privacy-security): where this checker fits.
- [Audit evidence packs with an AI agent](https://privycdp.org/ai/rails/use-cases/audit-evidence-packs): the nearest use case, on the same ledger.
- [AI governance policy template](https://privycdp.org/ai/ai-governance-policy-template-marketing): where logging rules are written down.
- [Rail card template](https://privycdp.org/tools/agent-rail-card): the document this agent compares with reality.


Browse the agents this one watches from the [AI agent rails hub](https://privycdp.org/ai/rails). Without a written claim there is nothing to test, so start with the [rail card template](https://privycdp.org/tools/agent-rail-card).

Rail level: L1 · Risk: low · Function: compliance privacy security

By Art Kukharenko
Published: 2026-11-02
Updated: 2026-11-02

## FAQ

### What is policy drift for AI agents?

The gap that opens between what an agent's written rules allow and what it actually does. Causes include a new tool, a model change nobody tested, or a limit quietly raised. Comparing the card with the ledger makes that gap visible.

### Why can't the checker pause an agent that breaks its rail?

Separating detection from enforcement limits the damage from a fault in either. Written demotion triggers and owners act on findings. If the checker could pause agents, one bad report or hijacked session could stop every rail at once.

### Does a clean report mean the agents are compliant?

No. A clean report means no differences appeared in what the ledger recorded, for the rails compared. It says nothing about whether the rail was well designed, or about anything the ledger never saw.

## Sources

1. Meta, Agents Rule of Two, counted here for a metadata-only session: https://ai.meta.com/blog/practical-ai-agent-security/
2. HashiCorp, detecting and managing resource drift in Terraform: https://developer.hashicorp.com/terraform/tutorials/state/resource-drift
3. OWASP GenAI Security Project, Top 10 for Agentic Applications for 2026: https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/


## Further reading

- [AI Agents for Compliance, Privacy and Security Teams: The Playbook on Rails](https://privycdp.org/ai/rails/functions/compliance-privacy-security)
- [Access Review Assistant: An AI Agent for Least-Privilege Reviews](https://privycdp.org/ai/rails/agents/access-review-assistant)
- [DSAR Assembler: An AI Agent That Prepares Access Request Packs](https://privycdp.org/ai/rails/agents/dsar-assembler)
- [AI Agent Rails: An Operating Model for Running Agents on Customer Data](https://privycdp.org/ai/rails)
- [AI Agent Rail Card Template: YAML, Spreadsheet and JSON Schema](https://privycdp.org/tools/agent-rail-card)