ai
Sync Medic: An AI Agent That Retries Failed Syncs Within Limits
The Sync Medic is an AI agent that diagnoses failed audience syncs from a customer data platform (CDP), retries approved ones within limits and proposes other fixes. Platform on-call engineers use it. A retry of the same approved version, audience hash and destination with a fresh consent check is its only L4 action. Everything else waits at L3 for a person, and it never alters a mapping or destination.
Rail level: L3 to L4 · Risk: medium · Function: platform operations
By Art Kukharenko · Published · Updated
All AI agent railsThe rail follows the AI agent rails method, and the recovery process is in failed sync retry with an AI agent. It picks up where the Audience Sync Steward stops, once a sync has a named approval.
Rail at a glance
The Sync Medic's rail gives the model no write access, because rules in the execution service decide every retry. The model writes the diagnosis, and nothing it says can authorise an attempt.
Scroll horizontally to see all columns.
| Field | Value |
|---|---|
| Job | Diagnose failed syncs, retry an already approved sync within limits, and propose other fixes. |
| Trigger | A failed sync event. |
| Data scope | Sync logs, error codes and the approval record hash, with no payload data. |
| Tool scope | Read: sync logs, error codes, approval record, destination status; propose: a fix for anything other than a retry; execute: retry the same approved sync version with idempotency and a fresh consent check. |
| Output contract | Failure cause, retry record, or a fix proposal. |
| Gate | No new approval for a bounded retry of the same approved hash, while the platform on-call approves any other change. |
| Limits | Retries per sync, backoff, and the same destination and audience hash only. |
| Ledger | Sync ID, approval hash, retry count, consent-check ID, outcome. |
| Evaluation | Replay of historic failures in a test destination. |
| Owner and kill switch | Platform lead; disabling retries sends failures to on-call. |
What it does and who uses it
The Sync Medic separates failures a patient retry can cure from failures that need an engineer.
Illustrative run: at 02:14 an approved audience sync to an advertising platform fails partway, with 503 and then 429 responses. The execution service confirms that version, audience hash and destination match the approval. It runs a fresh consent check, waits with backoff and retries, and the sync finishes. The model adds a short diagnosis for the morning review. The next night the same destination returns 401, which is no transient fault, so on-call receives a proposal to check the credential. On the third night the consent check removes three people. The hash no longer matches, so the retry stops and goes to on-call.
The retry decision is a checklist in code, and each row must pass. The last three rows are illustrative choices.
Scroll horizontally to see all columns.
| Check | Retry alone only if | Otherwise |
|---|---|---|
| Version | Same approved sync version | On-call reviews a proposal |
| Audience hash | Recomputed after the consent check and equal to the approved hash | New activation, which needs new approval |
| Destination | Same allow-listed destination | Refused |
| Consent | A fresh check ID exists for this attempt | Refused by the execution service |
| Error class | Transient, such as a timeout, 429 or 5xx | Proposal for on-call |
| Attempts | Under the cap, with backoff and jitter | On-call takes over |
| Delivery safety | Destination upserts on a key or accepts an idempotency key | Destination stays at L3 |
Reverse ETL tools draw similar lines. Hightouch's documentation, checked on 1 November 2026, says it does not retry requests rejected as invalid (400, 401, 403, 422) and gives 429 responses a larger budget 4.
Why this level
The Sync Medic can reach L4 on this one action because the retry adds nothing new. The same people, already approved, go to the same destination, so the blast radius equals what a named approver accepted. That holds only while version, hash and destination stay unchanged and consent has been re-checked. Anything else counts as a new activation.
The Rule of Two 1 is met by splitting the work. Error text comes from outside systems, so the model's session reads untrusted input, but it holds no payload and no tool. The action path can change state and contains no model. No single session holds all three properties.
Each destination starts at L3, with on-call approving retries. It moves to L4 on written replay evidence and the platform lead's sign-off. A duplicate delivery, an untested model or tool change, or an open incident drops it back, under the rail level rules. L4 is as far as the ladder goes, and it covers this retry alone.
Three ways it fails, and the control for each
Automatic retries go wrong when a repeat is not the same event.
Scroll horizontally to see all columns.
| Failure | What it looks like | Control |
|---|---|---|
| Duplicate after an ambiguous timeout | The first request landed, the response was lost, and the retry delivers the audience twice. | Idempotency or upsert keys, kept free of personal data as Stripe's guidance advises 3. Destinations without safe replay get no automatic retries. |
| Consent withdrawn before the retry | Someone opts out after approval, and an hour later the retry includes them. | A fresh consent check per attempt, required by the execution service. If it removes anyone, the hash changes and the retry stops. The consent withdrawal architecture explains the timing. |
| Retries hide or worsen a fault | A broken mapping keeps failing, or extra attempts pile load onto a struggling destination 2. | An error-class allow-list, a retry cap, backoff with jitter, and a rule that sends repeat failures on one destination to on-call even when retries succeed. |
How to evaluate it
Replay is the honest test for a retry agent: take historic failures, run them against a test destination and compare outcomes.
- Mean time to recover, compared with how the same incidents went by hand.
- Duplicate deliveries, counted at the test destination, with a target of zero.
- Failed retries, which show whether the error classes are right.
- Refusal tests. Change the hash, version or destination, drop the consent check, or exceed the cap. The execution service must refuse each one, even when the model asks.
- Diagnosis accuracy, sampled by on-call, because the note guides the morning's work.
Where it runs
Rules alone run the retry path, so no model needs customer data to keep syncs moving. A model, private by preference, writes the explanation from logs, and the boundary architecture patterns show how to keep it apart from execution. PrivyCDP records each AI-assisted action in an immutable audit ledger with its authorising actor, yet the medic is not a bundled agent.
Related
- Failed sync retry with an AI agent: how the vendor tools compare.
- AI agents for platform and data pipeline operations: why L4 is reachable early here.
- Rail card template: includes a filled Sync Medic example.
For the method behind these limits, start at the AI agent rails hub. Your own retry conditions can go straight into the rail card template.
Browse use casesFunction playbooksAgent roster
FAQ
Does retrying a failed sync count as a new activation?
Not if the version, audience hash and destination are unchanged and consent has been re-checked. Change any of those and it is a new activation, which needs a named approval.
How does it avoid duplicate deliveries?
It relies on the destination, through upsert keys or idempotency keys, which let a repeated request take effect once. Retrying is safe only when repeats have no extra side effects 2. A destination without that support stays under human approval.
What happens when retries are disabled?
Failures go straight to on-call, and syncs recover by hand. Existing approvals are unaffected, so engineers can still re-run an approved sync through the normal path.
Sources
Further reading
- AI Agents for Data Pipeline Operations
- Reverse ETL Failed Sync Retry: An AI
- AI Agent Permissions Template Download
- Agent rails
- Private AI agents
Published: · Updated:
Editorial dates describe content changes. Confirm current sources before making legal, security or purchasing decisions.