Loops in production

A loop in production is a system with a variable cost per request and no fixed upper bound on time. Both need watching, and neither is visible if you only log the final answer.

• Record the whole trajectory: every call, every observation, the exit taken. A final answer alone cannot be debugged.
• Track the distribution of iterations, not the average. A rising tail means tasks are getting harder or something has broken.
• Alert on exit mix. A jump in budget-exhausted or no-progress exits is the earliest signal that something changed upstream.
• Log side-effecting calls separately, with the iteration that produced them.

Evaluating a loop needs more than grading its final answer, because a right answer reached by an unreliable route will not stay right. The evals course covers the graders; the loop-specific ones are: was the required tool called, was a forbidden tool avoided, and did the trajectory stay within a sane length.

There is also a limit worth recognising. Loops are implicit control flow — the model decides what happens next, every turn. That flexibility is exactly what you want for open-ended tasks, and exactly what you do not want when a process has to follow a known path with approvals in specific places. When you find yourself writing prompt instructions that are really a flowchart, the loop has stopped being the right shape.