Customer data glossary

Evaluation set

An evaluation set is a maintained collection of test cases with known good outcomes, used to show whether an agent on a rail does its job and still does after any change.

By Art Kukharenko · Published 2026-11-02 · Updated 2026-11-02

All glossary terms

Part of the Learn hub and the customer data glossary.

What it means in practice

An evaluation set gives each rail a repeatable test. Anthropic describes the parts of an eval as tasks with defined success criteria and graders, which can be code, a model or a human 1. Rules and code should grade whatever they can, because they repeat exactly.

Good sets mix four kinds of case: typical work, edge cases, seeded defects and adversarial records, such as injection attempts. Anthropic separates capability evals, which may start with a low pass rate, from regression evals, which should stay near 100% 1. It suggests starting with 20 to 50 tasks drawn from real failures and grading the outcome over the exact path taken 1. No size suits every job.

Rerun the set before any change to the model, prompt or tools. On a rail, a model or tool change without a regression pass is a demotion trigger. The Trainer can keep sets current, and the rail owner accepts each addition.

A set built from real records holds personal data. Prefer synthetic or masked examples, state how long the set is kept, and include it in any impact assessment. Otherwise the test harness becomes a second, ungoverned copy of your customers.

PrivyCDP offers a bring-your-own-model interface for private models in the customer environment, and a model swap there is the kind of change a set should catch.

Example (wealth manager, illustrative): the Journey QA Reviewer is tested on journeys with seeded audience, trigger, branch, link, consent and timing errors. The team counts catches per error type, so a strong overall result cannot hide a blind spot in consent logic.

Trainer agent: keeps sets and drills current.
Rail card: the Evaluation field.
Rail level: promotion rests on this evidence.
Data minimisation: applies to test data too.

Where it matters

The agent rail card: what the Evaluation field must say.
Marketing automation QA with an AI agent: seeded defects at work.

Back to the customer data glossary or the Learn hub.

Browse use cases Function playbooks Agent roster

FAQ

Should an evaluation set include real customer records?

Avoid it where you can. Synthetic or masked cases usually test the same behaviour with less risk. If real records are needed, govern the set like any other copy of personal data.

Who owns an evaluation set?

The rail's owner accepts what goes in. A trainer agent, a script or an analyst can maintain it, but it should not grade its own changes.

Sources

Anthropic, "Demystifying evals for AI agents" (9 January 2026)

This entry is general information, not legal advice.