Regulating behaviour

Behaviour is whether the agent did the right thing for the right reasons. It is the target that actually matters, and it is the one nobody has a settled answer for. Every serious source on harness engineering says so; the useful move is to take that seriously rather than pretend a rubric closes the gap.

What makes it hard is that the standard is not fully stated. Maintainability has a schema. Architecture has a route. Behaviour has a human's sense of what a good reply to this customer would have been, and that sense is only partly writable down. The gap between what you can write and what you mean is where behaviour failures live.

The tools that exist are all inferential, all partial, and all worth using with their limits named.

• A checker against a rubric. Catches the failures the rubric names. Misses the ones it does not, and drifts if the rubric is vague. Validate it against your own labels, as the evals course showed.
• Outcome checks on known cases. The healthcare blueprint's 'advised emergency care' is one: for the cases where you know the right answer, assert it. These are the strongest behaviour sensors you can build, and they only cover the cases you thought of.
• Sampled human review. Read some real outputs every week. Not a sensor in the automated sense; the only one that catches the failure you had not imagined.
• The dataset. Every behaviour failure found by any of the above becomes a case. The harness cannot regress on a lesson it has already learnt.

The healthcare blueprint is the clearest demonstration on this site. Three process checks passed — the tool was called, no condition was named, no medication recommended — while the agent reassured a patient describing chest pain. Only an outcome check on the reply caught it. Maintainability and architecture were both satisfied; behaviour was not; and no amount of the first two would have found it.