Building a harness

Harness engineering is the discipline of building the thing that governs the agent, rather than the agent. The previous lesson named the parts — guides before, sensors after. This one builds a harness for the support agent, in the order that experience says the parts should be added.

• First, the cheap sensors. Does the reply parse. Is every required field present. Was a forbidden tool called. These are code, cost nothing, and run on everything.
• Second, the guides those sensors keep firing on. When the shape check fails repeatedly, fix the prompt or the schema — a sensor that catches the same thing every day is describing a missing guide.
• Third, a precondition on every side-effecting tool. The refund tool refuses until the policy has been read. This is a guide that cannot be argued with.
• Fourth, one inferential sensor, on the thing that matters most. A checker that reads the reply against a rubric — expensive, so spent where the cost of being wrong is highest.
• Last, the dataset. Every failure the sensors caught becomes a case, so the harness is tested against the mistakes it has already made.

Notice what is not in that order: a long system prompt. Instructions are the part teams reach for first and they are the weakest guide, because nothing verifies that they are followed. A prompt line that says "never promise a date" is a hope; a regex gate that rejects one is a control.

A harness regulates three different things, and the industry has reached consensus on only one of them.

That split is useful for deciding what to build next. If the harness has no maintainability checks, add those first; they are the cheapest and most reliable. If it has those and the agent still takes wrong routes, the gap is architectural. If both are in place and the output is still wrong, you are in the hard third, and the honest tools are a rubric, a checker, and a human reading samples.

maintainability   schema_valid · exact_match · no forbidden phrases      code, every call
architecture      must_call lookup before issue_refund · visited policy node   trace asserts
behaviour         checker against a rubric · human reads 20 a week     inferential, sampled

One question is worth asking a team every quarter: are we steering the harness, or just steering prompts? Prompt edits feel like progress and do not accumulate. A harness accumulates — every sensor added keeps working, every precondition keeps holding, and the dataset of caught failures only grows.