# Data Quality Triager: An AI Agent That Sorts Data Failures

The Data Quality Triager is an AI agent that sorts failed data-quality tests on a customer data platform (CDP), ranks them by downstream harm and proposes fixes. Data stewards use it at propose level (L2), and at L3 it applies format-only fixes after approval. It never changes what a field means, deletes a record, or edits consent or identity fields.

Its rail belongs to the [AI agent rails](https://privycdp.org/ai/rails) method. The full workflow is in [data quality triage with an AI agent](https://privycdp.org/ai/rails/use-cases/data-quality-triage).

## Rail at a glance

The rail card below gives the Data Quality Triager two paths: triage, which changes nothing, and a fix path that only a data steward can open. The approval service and the execution service hold that split, not the prompt.

| Field | Value |
|---|---|
| Job | Triage data-quality failures, propose fixes, and apply format-only fixes after approval. |
| Trigger | A failed data-quality test or a data-steward request. |
| Data scope | Test results, schema and masked samples, with no consent fields on the fix path. |
| Tool scope | Read: test results, schema, lineage, masked samples; propose: a ranked triage list and fixes; execute: one approved format-only fix, run by the execution service. |
| Output contract | Failure, likely cause, affected downstream uses, proposed fix and rollback plan. |
| Gate | A data steward approves each fix batch, and the approval binds to the batch hash. |
| Limits | Format-only changes, a maximum batch size, and approved tables only. |
| Ledger | Test ID, proposed fix, approval, rows changed, rollback reference. |
| Evaluation | Historic failures with known causes, plus seeded defects. |
| Owner and kill switch | Head of data or data steward; an [Orchestrator](https://privycdp.org/ai/rails/orchestrator-agent) pause disables the fix path while triage continues. |

## What it does and who uses it

The Data Quality Triager gives a data steward one ranked list instead of a wall of red tests, using test results, lineage and masked samples.

Illustrative run: a nightly load from a loyalty vendor fails three tests. `currency_code` arrives in lower case on some rows, `order_date` switches to a slash-separated format, and `store_region` is empty on others. The triager groups all three under one likely cause, a new export template. It ranks `store_region` first, because lineage shows a revenue report and an audience sync read that column. It proposes upper-casing the currency codes as a format-only batch with a rollback file. It proposes nothing for the dates, since the vendor's notes do not say whether the day comes first. The steward approves the batch and sends the other two to the source owner.

The hardest call on this rail is where format ends and meaning begins. This illustrative table draws that boundary.

| Proposed change | Class | Why |
|---|---|---|
| Trim spaces, or upper-case `gbp` in a currency column | Format | The value and its meaning stay the same. |
| Reorder `03/04/2026` into ISO date order | Meaning, unless the source format is documented | Day and month can swap silently, and the new value still passes the test. |
| Fill an empty country from a phone prefix | Meaning | The fix is an inference, not a reformat. |
| Lower-case an email address | Refused | Email is a match key, so it counts as an identity field. RFC 5321 also treats the local part as case-sensitive, while discouraging reliance on that [2](#source-2). |
| Map `Y` to `true` in an opt-in column | Refused | Consent fields sit outside the fix path. |

## Why this level

Data Quality Triager proposals carry little risk, because nothing changes until a steward signs. A format fix earns L3 because a pre-image exists, and the table allow-list and batch size cap the blast radius. It stays below L4 because reversal is clean only inside the platform: by the time someone spots a bad batch, a sync may have copied the values out.

The Rule of Two [1](#source-1) explains the split between the two paths. Masked samples and test messages carry text from source systems, which makes them untrusted input, and the samples still derive from private data. A triage session therefore holds two of the three properties and gets no write tool. The fix runs in a separate execution service with no model in it. That service accepts only an approved batch hash and a transform from an allow-list, and it refuses any column tagged as consent or identity.

New tables start at L2. A table moves to L3 on written evidence from replayed failures and the data owner's sign-off, as the [rail levels](https://privycdp.org/ai/rails/rail-levels-autonomy-ladder) describe. L3 is the ceiling.

## Three ways it fails, and the control for each

Data-quality agents fail most dangerously when tests turn green and the data is still wrong.

| Failure | What it looks like | Control |
|---|---|---|
| Meaning change dressed as format | A date "fix" swaps day and month on part of a table, and the format test passes. | Transforms come from an allow-list with no model in the path. Ambiguous patterns are refused, and the steward sees before and after values. |
| Treating the symptom | The same vendor bug returns weekly, and each approved batch hides it for another week. | Recurrence is counted by test ID. A repeat failure blocks the fix path and goes to the source owner. |
| Ranking by row count | A noisy table with many failures outranks a handful of bad rows feeding a suppression list. | Ranking uses lineage and a written list of critical uses. Test owners set warn and error thresholds [3](#source-3), and the agent cannot alter them. |

## How to evaluate it

Good triage puts the right failure in front of the right person first, and an approved fix stays fixed. Score the Data Quality Triager on historic failures with known causes, plus seeded defects:

- **Time to triage,** from failed test to an entry a steward can act on.
- **Cause accuracy,** on whether the known cause is named and related failures are grouped.
- **Refusal on meaning traps.** Seed ambiguous dates, unit changes and inferred values, and count any proposed as format-only.
- **Rollbacks per approved batch,** each marking a fix that passed review wrongly.
- **Recurring failures,** a sign the team is polishing symptoms.

Test the fix path separately by asking the execution service to touch a consent column, which it should refuse. Report results by rail and role, not by named steward.

## Where it runs

Masked samples still come from customer records, so a private model is the preferred home for the Data Quality Triager. An external one would need masking and contract cover first, and the [boundary architecture patterns](https://privycdp.org/ai/ai-agents-on-customer-data-inside-your-boundary) set out the placements. [PrivyCDP](https://privycdp.org/cdp) offers AI-assisted review of unusual events, while this rail is a method you write yourself.

## Related

- [Data quality triage with an AI agent](https://privycdp.org/ai/rails/use-cases/data-quality-triage): steps, ledger example and checklist.
- [AI agents for data and analytics teams](https://privycdp.org/ai/rails/functions/data-and-analytics): how the data agents divide the work.
- [Schema Drift Sentinel](https://privycdp.org/ai/rails/agents/schema-drift-sentinel): the structure changes behind failed tests.
- [Rail card template](https://privycdp.org/tools/agent-rail-card): a blank copy of this table.


The [AI agent rails hub](https://privycdp.org/ai/rails) has the wider method. Download the [rail card template](https://privycdp.org/tools/agent-rail-card) to set a format-or-meaning boundary for your own tables.

Rail level: L2 to L3 · Risk: medium · Function: data and analytics

By Art Kukharenko
Published: 2026-11-02
Updated: 2026-11-02

## FAQ

### Can an AI agent fix data quality issues on its own?

Only format-only fixes, and only after a data steward approves the exact batch. Anything that changes meaning, such as filling a missing value, stays with a person.

### Why keep consent and identity fields off the fix path?

A small edit there can change who gets contacted or join two people's records. The execution service refuses them even when a batch looks harmless.

### What happens when the fix path is paused?

Triage carries on, and failures still reach stewards as a ranked list. Stewards apply urgent fixes through their normal tooling.

## Sources

1. Meta, Agents Rule of Two, as applied to triage: https://ai.meta.com/blog/practical-ai-agent-security/
2. IETF, RFC 5321 section 2.4, on case in mailbox local parts: https://www.rfc-editor.org/rfc/rfc5321.html
3. dbt Labs, test severity and thresholds: https://docs.getdbt.com/reference/resource-configs/severity


## Further reading

- [AI Agents for Data and Analytics Teams: The Playbook on Rails](https://privycdp.org/ai/rails/functions/data-and-analytics)
- [Data Quality Triage: Sort and Fix Failures With an AI Agent on Rails](https://privycdp.org/ai/rails/use-cases/data-quality-triage)
- [Draft-approve-execute split](https://privycdp.org/learn/glossary/draft-approve-execute)
- [AI Agent Rails: An Operating Model for Running Agents on Customer Data](https://privycdp.org/ai/rails)
- [AI Agent Rail Card Template: YAML, Spreadsheet and JSON Schema](https://privycdp.org/tools/agent-rail-card)