Lesson 11 · Production · 10 min

Deploying with confidence

Because the API is OpenAI-compatible, the code you have been running in these labs is the code that goes to production — the base URL and the key change, and nothing else does. Deployment is mostly a matter of deciding what to watch.

Production differs from your dataset in one important way: there are no expected answers. What you have instead is the graders that need none. Shape checks, schema validation and tool-call sanity all work on live traffic, and they catch the failures that break things most visibly.

Sample the rest. Take a small share of real traffic each day, run your judge over it, and read a few by hand. When you find a case that surprises you, label it and add it to the dataset. This is how a dataset stays alive: it grows from production rather than from imagination.

Log the input, the output and the model version for every request.
Run shape and schema checks on all traffic; they are free.
Sample for model-graded checks rather than judging everything.
Add surprising cases to your goldens, with a label, the same week you find them.

The loop closes here. Production surfaces cases you had not imagined, those cases become goldens, the goldens catch the next regression — and the evaluation you built in lesson 3 keeps working long after the model behind it has been replaced.

Checkpoint

Choose a model and write down the scores you would expect it to hold in production.