Lesson 07 · Practice · 12 min
The harness as the product
Models get replaced. Prompts get rewritten for the new model. The harness — the sensors, the preconditions, the dataset of caught failures — is the part that carries over, and it is usually the part with the most engineering value in it. It is worth treating it as the product.
- Version it. A sensor that changed last Tuesday should be diffable, and a trace from last Monday should be readable against the harness that governed it.
- Own it. A harness with no owner drifts into a pile of checks nobody remembers the reason for. Each sensor should have a reason recorded beside it — the failure it was added to catch.
- Test it. A sensor is code; it can be wrong. The dataset of caught failures is the test suite: every sensor should fire on the case it was added for, and the suite should run when the harness changes.
- Measure it. Which sensors fire, how often, and whether the rate is moving. A sensor that never fires is either a guide that works or a check that is broken, and you should know which.
The skills deep dive makes the same argument about knowledge: the value is not in the text, it is in the versioning, the ownership and the test. A harness is the same shape. The mechanism of any one check is trivial; the discipline around the set is what compounds.
This also changes what a model upgrade means. Without a harness, swapping the model is a leap — you re-read outputs and hope. With one, it is a run: the dataset goes through the new model, the sensors report, and you know before shipping what got better and what regressed. The harness is what makes the model a replaceable part.
Ask once a quarter: are we steering the harness, or just steering prompts? Prompt edits feel like progress and do not accumulate. A harness accumulates — every sensor added keeps working, every precondition keeps holding, and the dataset of caught failures only grows.
Checkpoint
Run the same task on two maker models and say what the unchanged rubric caught on each.