Lesson 06 · Workflows · 11 min

Evaluator-optimiser

One model generates; another grades it against criteria and, if it fails, sends back a critique; the generator tries again. This is the verification loop from the loop-engineering course, named as a pattern.

The guide is specific about when it works: when you have clear evaluation criteria, and when iteration measurably improves the result. Both conditions matter. A vague evaluator drifts between runs; a task where the first draft is usually fine pays for grading it cannot use.

One model generates, another grades against criteria and feeds back a critique. Worth its cost only when the criteria are clear and a first draft is often not good enough.

Give the evaluator a criterion it can actually check. 'Makes no promises' is checkable; 'is good' is not.
Feed back the fix, not only the verdict. A generator told only that it failed will produce nearly the same draft again.
Bound the rounds. Two is usually enough; a loop still failing after three is telling you the criterion or the generator is wrong, not that it needs a fourth go.
Compare against the chain's code gate. If a regex can catch the failure, a second model call is the expensive way to do it.

Against the lab's single call, the evaluator-optimiser typically costs three to five calls and takes several times as long. On the task that invites a promise it earns that; on the one-question task it usually accepts the first draft and has paid one extra call to learn nothing.

Run it on all three tasks and read the accepted-after notes. The pattern's value is entirely in how often the first draft fails — which is a property of your task, and only a run can tell you.

Checkpoint

On one task, say whether the evaluator-optimiser earned its extra calls over the chain.