ai
Schema Drift Sentinel: An AI Agent That Explains Schema Changes
The Schema Drift Sentinel is an AI agent that detects and explains schema changes in the sources and destinations of a customer data platform (CDP). Data-platform engineers rely on it at read level (L1): it compares schema metadata, then says what changed, what depends on it and who should act. It never alters a schema or pipeline, and it never reads row data.
Rail level: L1 · Risk: low · Function: data and analytics
By Art Kukharenko · Published · Updated
All AI agent railsIts rail is among the simplest in the AI agent rails method, because the session never sees a customer record. The response to each alert usually feeds the data quality triage workflow.
Rail at a glance
The Schema Drift Sentinel's rail limits it to metadata, which the data view enforces rather than a prompt. Code does the detecting by comparing snapshots, and the model explains what a difference means.
Scroll horizontally to see all columns.
| Field | Value |
|---|---|
| Job | Detect and explain schema changes in sources and destinations. |
| Trigger | A contract test failure or a scheduled schema comparison. |
| Data scope | Schema metadata and contract test results, with no row data. |
| Tool scope | Read: schema snapshots, data contracts, lineage; propose: no changes; execute: no actions. |
| Output contract | What changed, what depends on it, and who should act. |
| Gate | None for alerts, and the pipeline owner decides any change. |
| Limits | Metadata alone, under a budget per run. |
| Ledger | Schema versions compared, alert, owner notified. |
| Evaluation | Seeded schema changes across sources. |
| Owner and kill switch | Data platform lead; stopping the schedule leaves contract tests running. |
What it does and who uses it
The Schema Drift Sentinel turns a raw schema diff into a message that names the change, its dependants and an owner. Pipeline owners act on it, and analysts see it when their models are affected.
Illustrative run: a weekly comparison picks up three changes after a CRM vendor's release. mobile_opt_in has gone and sms_opt_in has appeared. account_id has moved from integer to string. A new date_of_birth column has arrived. The sentinel reports the first as a possible rename, not a fact, and lists the consent mapping, two audience models and one sync that read the old name. It marks the type change as a risk to joins. It routes the new column to the data-classification owner as well as the pipeline owner. The owners decide, and the sentinel changes nothing.
Schema changes differ in who they hurt, and a list of breaking changes answers only part of that. dbt's model contracts, for example, treat a removed column or a changed type as breaking 2, while a rename looks to any structural check like a removal plus an addition. The routing below is an illustrative starting point.
Scroll horizontally to see all columns.
| Change | Breaks the pipeline? | Why a person still cares | Route to |
|---|---|---|---|
| Column removed | Usually | Models and syncs fail, or load nulls | Pipeline owner |
| Column renamed | Usually | A consent or suppression mapping may stop applying | Pipeline owner and consent-field owner |
| Type changed | Often | Joins, rounding and ID formats shift | Pipeline owner |
| Column added | Usually not | It may hold personal data nobody has classified | Data-classification owner |
| Destination field retired | Later | An activation may drop the field without an error | Sync owner |
| Same name and type, new meaning | No | Pence become pounds, or a status code is relabelled | Beyond schema tools, so value tests must catch it |
Why this level
Read level (L1) fits the Schema Drift Sentinel because its whole job is reading and telling. A wrong alert wastes an engineer's morning. A missed change still meets contract tests, which run with or without the agent, so the blast radius stays small.
Under the Rule of Two 1, column names and descriptions count as untrusted text, since they arrive from vendors and source teams. Metadata is rarely private data, although a column called hiv_status reveals what a table holds. A notification service sends the alerts, and the model holds no write tool. That leaves one property, occasionally two, and never the third.
No promotion is planned, because auto-remapping a rename could point a consent mapping at the wrong column. With few sources and firm event contracts, a diff script may be all you need. Add a model when explaining impact across many vendors takes real time.
Three ways it fails, and the control for each
Schema monitoring fails mostly through false comfort, when a quiet channel gets read as proof that nothing changed.
Scroll horizontally to see all columns.
| Failure | What it looks like | Control |
|---|---|---|
| Semantic drift read as "no change" | A source starts sending pence in a pounds column, and a quiet sentinel reassures the team. | Every message says "schema only". Contracts check shape and leave content to data tests 2, and one vendor critique stresses that logic changes pass unseen 3. Value tests stay in place. |
| Rename guessed wrongly | The model calls an unrelated new column a rename, and an engineer remaps a consent field to it. | Renames are labelled as hypotheses with the evidence shown. Anything touching consent or suppression routes to that field's owner. |
| Silence from a failed comparison | A source is unreachable or a snapshot job dies, and no alert looks the same as no drift. | Each run records which sources it compared and which it could not reach. A rule alerts the data platform lead when a comparison is missing. |
How to evaluate it
Evaluating a drift agent means seeding changes the team already knows about, then checking timing, routing and honesty.
- Detection by change type, across removals, renames, type widening, new sensitive-looking columns and retired destination fields.
- Time to detect, from a change first being visible in the source to the alert.
- Missed drift found later, taken from incident reviews, which is the number that should worry you most.
- Dependency recall, comparing the listed dependants with what the lineage tool returns.
- Alert precision and routing. Noisy or misrouted alerts teach teams to mute the channel.
- Honesty on semantic seeds. A same-schema change should produce "not visible to schema comparison", never "no issue".
Where it runs
Metadata holds no customer records, so an external model can work if the contract permits it, while a private one suits regulated teams whose column names reveal too much. The boundary architecture patterns compare those options. PrivyCDP offers an MCP-ready interface, and the sentinel described here is a pattern your team would build, not part of that product.
Related
- Data quality triage with an AI agent: where failures caused by drift are sorted and fixed.
- Data Quality Triager: the agent that takes over once tests start failing.
- AI agents for data and analytics teams: the playbook covering all four data agents.
- Rail card template: the ten fields, ready to fill in for a metadata-only agent.
More agents sit in the AI agent rails hub. Before your next vendor release, use the rail card template to give each change type an owner.
Browse use casesFunction playbooksAgent roster
FAQ
What is schema drift?
A change to the structure of a table, event or API that its consumers did not agree in advance, such as a removed column or a new data type. Left unnoticed, it breaks pipelines or quietly changes what reaches downstream tools.
Should an LLM detect schema drift?
Code should detect it by comparing snapshots, because that comparison is exact and cheap. A model is useful afterwards, to explain the likely impact in plain words and to draw on lineage and vendor notes.
How is schema drift different from data drift?
Schema drift changes structure: names, types and fields. Data drift changes the values or their meaning while the structure stays put. This agent sees only the first, so tests on values must cover the second.
Sources
Further reading
- AI Agents for Data Quality
- AI Agent for Data Quality Triage: Data
- AI Identity Resolution Review Agent
- Agent rails
- AI Agent Permissions Template Download
Published: · Updated:
Editorial dates describe content changes. Confirm current sources before making legal, security or purchasing decisions.