Write the evals before you write the prompt

The usual loop is: write a prompt, try it, notice a bad output, adjust the prompt, try again. It feels productive. It is a random walk, because you are optimising against whichever failure you saw most recently and you have no way to know what the change broke.

The missing artefact is a scored set

Fifty to two hundred real inputs with an expected outcome. Not synthetic ones — real ones, taken from the actual traffic or from the domain expert who will judge the system.

evals
  id
  input               -- the actual user turn or document
  expected            -- the outcome, or the assertion
  kind                -- exact | contains | tool_called
                      -- | schema_valid | judged
  tags                -- edge case, regression, adversarial

Now a prompt change has a number attached. 84% to 91% is progress. 84% to 79% is a revert, and you know immediately rather than in three weeks when a customer notices.

Most assertions are not fuzzy

There is a reflex to reach for a model-graded judge for everything. Most useful checks are deterministic and free: did it call the right tool, is the JSON valid against the schema, does the answer contain the account number, did it refuse when it should have refused.

Save the judged evals for genuine quality questions — tone, completeness, whether a summary is faithful — and keep them a minority. Deterministic checks do not drift and do not cost per run.

Every incident becomes a case

The discipline that compounds: when something fails in production, the fix is not just a prompt tweak. The input goes into the eval set with the correct expected outcome, tagged as a regression. It runs forever afterwards.

Six months in you have a set that encodes every way the system has actually failed, which is worth far more than any benchmark.

Run it in CI, gate on it

The set runs on every change to a prompt, a tool definition, a retrieval config, or a model version. A drop below threshold fails the build.

That last one matters most. Providers update models underneath you, and a silent quality regression from a version bump is invisible without a scored set — which is precisely when you find out from a customer instead of a pipeline.