AssayAssay
04 / 08

Lesson 04 · Control · 11 min

Iteration budgets

Every iteration is a model call, and every call carries the entire conversation so far. This means cost does not grow linearly with iterations — it grows roughly with the square of them, because each turn re-sends everything before it plus the new observation.

That is the real argument for a budget. A loop with no ceiling that gets stuck does not fail quickly and cheaply; it fails slowly and expensively, and the last iterations are the most expensive of all.

  • Set the ceiling from observed need, not intuition. Run your tasks, see how many iterations the successful ones use, then allow some headroom.
  • Count tokens as well as iterations. A loop with long observations can blow a budget in three turns.
  • Treat hitting the ceiling as a signal worth logging, not just a stop condition. A rising rate of budget exhaustion means something changed.
  • Return partial work when you can. An agent that stopped halfway usually has something worth showing.

The lab reports tokens and elapsed time for every run. Run the same task at different budgets and watch the token count grow faster than the iteration count — that gap is the cost of carrying the conversation forward.

Long observations are the usual culprit. A tool that returns a whole document rather than the relevant field will consume the budget on its own. Trimming what a tool returns is often a bigger win than changing the model.

Checkpoint

Show that tokens grow faster than iterations across three runs of the same task.

“Where is my order 41902? It hasn't arrived yet.”

The shortest useful loop: one tool, one answer. A model that replies without calling anything is inventing the status.

Run the same task at budgets of 3, 6 and 10. Compare the token totals, not just whether it finished.

Nothing has run yet. Pick a task and press Run — every iteration appears below.