Lesson 06 · Controls in production · 13 min
The pre-deployment gate
The evals course taught measurement on outputs: answers graded against expectations. Agents widen what needs measuring, because an agent can produce a lovely final answer while having done the wrong things to get there. The framework names four dimensions of pre-deployment testing, and only the first is what most teams currently test.
- Task execution — did it accomplish the thing, accurately.
- Policy adherence — did it follow the SOPs and route to humans where required.
- Tool use — right tools, right inputs, right order, right permissions.
- Robustness — what happens on errors, edge cases and malformed requests.
Two habits follow. Test whole workflows, not final answers: the trace is the artefact under test, which is why every check in this course's labs reads the trace rather than the reply. And test repeatedly: agent behaviour is stochastic, so a case that passes once has not passed — it has passed once. Run the same cases multiple times, add perturbations, and treat stability itself as a result.
Realism is the third ingredient. Test in an environment that behaves like production — real tool semantics, real failure modes, sandboxed effects — while being honest that a sandbox is not the world. One government sandbox in the framework tested computer-use agents against real chatbots and found failure modes a mock would never surface: session limits, slow loads, third-party content carrying hostile instructions. Calibrate realism against the blast radius of being wrong about it.
Evaluating agent traces at scale is genuinely hard — long, unstructured, expensive to read. The working pattern is layered: deterministic checks on the structured parts (which tools ran, in what order), model-graded review on the semantic parts, and humans on samples. You built each layer in the evals course; here they become a release decision.
The lab is a release gate for Relay. Four tickets, each with deterministic policy checks over the trace — resets the right account, stages rather than executes the unlock, escalates the deletion, and answers the vague ticket by asking rather than acting. The suite runs every case twice and reports a pass rate with a Wilson interval against an 85% bar. Eight runs will produce an interval wide enough to be slightly embarrassing, which is precisely the point — the gate tells you what it knows and how thinly it knows it.
Checkpoint
Write one policy check for an agent you care about, phrased so code could verify it from the trace alone. If you cannot phrase it that way, it is not yet a policy — it is a preference.