ai

Sync Medic: An AI Agent That Retries Failed Syncs Within Limits

The Sync Medic is an AI agent that diagnoses failed audience syncs from a customer data platform (CDP), retries approved ones within limits and proposes other fixes. Platform on-call engineers use it. A retry of the same approved version, audience hash and destination with a fresh consent check is its only L4 action. Everything else waits at L3 for a person, and it never alters a mapping or destination.

Rail level: L3 to L4 · Risk: medium · Function: platform operations

By Art Kukharenko · Published · Updated

All AI agent rails

The rail follows the AI agent rails method, and the recovery process is in failed sync retry with an AI agent. It picks up where the Audience Sync Steward stops, once a sync has a named approval.

Rail at a glance

The Sync Medic's rail gives the model no write access, because rules in the execution service decide every retry. The model writes the diagnosis, and nothing it says can authorise an attempt.

FieldValue
JobDiagnose failed syncs, retry an already approved sync within limits, and propose other fixes.
TriggerA failed sync event.
Data scopeSync logs, error codes and the approval record hash, with no payload data.
Tool scopeRead: sync logs, error codes, approval record, destination status; propose: a fix for anything other than a retry; execute: retry the same approved sync version with idempotency and a fresh consent check.
Output contractFailure cause, retry record, or a fix proposal.
GateNo new approval for a bounded retry of the same approved hash, while the platform on-call approves any other change.
LimitsRetries per sync, backoff, and the same destination and audience hash only.
LedgerSync ID, approval hash, retry count, consent-check ID, outcome.
EvaluationReplay of historic failures in a test destination.
Owner and kill switchPlatform lead; disabling retries sends failures to on-call.

What it does and who uses it

The Sync Medic separates failures a patient retry can cure from failures that need an engineer.

Illustrative run: at 02:14 an approved audience sync to an advertising platform fails partway, with 503 and then 429 responses. The execution service confirms that version, audience hash and destination match the approval. It runs a fresh consent check, waits with backoff and retries, and the sync finishes. The model adds a short diagnosis for the morning review. The next night the same destination returns 401, which is no transient fault, so on-call receives a proposal to check the credential. On the third night the consent check removes three people. The hash no longer matches, so the retry stops and goes to on-call.

The retry decision is a checklist in code, and each row must pass. The last three rows are illustrative choices.

CheckRetry alone only ifOtherwise
VersionSame approved sync versionOn-call reviews a proposal
Audience hashRecomputed after the consent check and equal to the approved hashNew activation, which needs new approval
DestinationSame allow-listed destinationRefused
ConsentA fresh check ID exists for this attemptRefused by the execution service
Error classTransient, such as a timeout, 429 or 5xxProposal for on-call
AttemptsUnder the cap, with backoff and jitterOn-call takes over
Delivery safetyDestination upserts on a key or accepts an idempotency keyDestination stays at L3

Reverse ETL tools draw similar lines. Hightouch's documentation, checked on 1 November 2026, says it does not retry requests rejected as invalid (400, 401, 403, 422) and gives 429 responses a larger budget 4.

Why this level

The Sync Medic can reach L4 on this one action because the retry adds nothing new. The same people, already approved, go to the same destination, so the blast radius equals what a named approver accepted. That holds only while version, hash and destination stay unchanged and consent has been re-checked. Anything else counts as a new activation.

The Rule of Two 1 is met by splitting the work. Error text comes from outside systems, so the model's session reads untrusted input, but it holds no payload and no tool. The action path can change state and contains no model. No single session holds all three properties.

Each destination starts at L3, with on-call approving retries. It moves to L4 on written replay evidence and the platform lead's sign-off. A duplicate delivery, an untested model or tool change, or an open incident drops it back, under the rail level rules. L4 is as far as the ladder goes, and it covers this retry alone.

Three ways it fails, and the control for each

Automatic retries go wrong when a repeat is not the same event.

FailureWhat it looks likeControl
Duplicate after an ambiguous timeoutThe first request landed, the response was lost, and the retry delivers the audience twice.Idempotency or upsert keys, kept free of personal data as Stripe's guidance advises 3. Destinations without safe replay get no automatic retries.
Consent withdrawn before the retrySomeone opts out after approval, and an hour later the retry includes them.A fresh consent check per attempt, required by the execution service. If it removes anyone, the hash changes and the retry stops. The consent withdrawal architecture explains the timing.
Retries hide or worsen a faultA broken mapping keeps failing, or extra attempts pile load onto a struggling destination 2.An error-class allow-list, a retry cap, backoff with jitter, and a rule that sends repeat failures on one destination to on-call even when retries succeed.

How to evaluate it

Replay is the honest test for a retry agent: take historic failures, run them against a test destination and compare outcomes.

  • Mean time to recover, compared with how the same incidents went by hand.
  • Duplicate deliveries, counted at the test destination, with a target of zero.
  • Failed retries, which show whether the error classes are right.
  • Refusal tests. Change the hash, version or destination, drop the consent check, or exceed the cap. The execution service must refuse each one, even when the model asks.
  • Diagnosis accuracy, sampled by on-call, because the note guides the morning's work.

Where it runs

Rules alone run the retry path, so no model needs customer data to keep syncs moving. A model, private by preference, writes the explanation from logs, and the boundary architecture patterns show how to keep it apart from execution. PrivyCDP records each AI-assisted action in an immutable audit ledger with its authorising actor, yet the medic is not a bundled agent.

For the method behind these limits, start at the AI agent rails hub. Your own retry conditions can go straight into the rail card template.

Browse use casesFunction playbooksAgent roster

FAQ

Does retrying a failed sync count as a new activation?

Not if the version, audience hash and destination are unchanged and consent has been re-checked. Change any of those and it is a new activation, which needs a named approval.

How does it avoid duplicate deliveries?

It relies on the destination, through upsert keys or idempotency keys, which let a repeated request take effect once. Retrying is safe only when repeats have no extra side effects 2. A destination without that support stays under human approval.

What happens when retries are disabled?

Failures go straight to on-call, and syncs recover by hand. Existing approvals are unaffected, so engineers can still re-run an approved sync through the normal path.

Sources

  1. Meta, Agents Rule of Two, on separating the model from the action path
  2. AWS, Timeouts, retries and backoff with jitter
  3. Stripe, idempotent requests
  4. Hightouch, sync retries

Further reading

Published: · Updated:

Editorial dates describe content changes. Confirm current sources before making legal, security or purchasing decisions.