The agent loop

A tool list describes everything an agent may do and says nothing about when it stops doing it. The agent loop is the control structure that answers that question. A model holding tools produces either an answer or a request for a call, so something has to decide what happens next, how many times that repeats and on what condition the cycle ends.

Two arrangements of the cycle have names and a published record behind them, and they differ in when the model decides. Reasoning and acting in turn, where each observation feeds the next decision, was set out in a paper posted in October 2022. Planning the whole sequence first and executing it afterwards followed in May 2023. Almost every framework shipped since is one of the two with a different vocabulary.

The sections below give the cycle itself and take the two arrangements in the order they arrived. They then price both on the same four step task, and close on the part most descriptions leave out, which is the set of conditions that make the cycle end.

The cycle an agent runs

One pass through the loop is four operations, and only the second of them belongs to the model.

Three properties of that picture decide most of what follows. The window is reassembled from scratch on every pass, so the whole transcript is resent and repaid each time. The model never executes anything, because it emits a request and the application runs the guard from the previous page before honouring it. And the count of passes is unknown when the request arrives, which is precisely what a chain gave up.

ReAct and the interleaving of thought and action

The first named arrangement is ReAct, published by Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan and Yuan Cao, posted in October 2022 and presented at the International Conference on Learning Representations in 2023. The idea in it is a single interleaving. The model writes a short piece of reasoning, takes one action, reads the observation that comes back, and then reasons again with that observation in hand.

The argument for interleaving runs in both directions. Reasoning between actions lets the model revise a plan that has just met a surprise. Acting between pieces of reasoning gives the reasoning facts it would otherwise have invented. A run in this style is legible by reading, which is most of why the pattern spread.

Goal   The customer says they were charged twice in August. Find out what
       happened and explain it.

Thought       I need the account id before I can read any invoice.
Action        identify_account({"email": "r.okafor@northwind.example"})
Observation   {"account_id": "acct_4b71c9e20d8af315"}

Thought       Now list the August invoices on that account.
Action        find_invoices({"account_id": "acct_4b71c9e20d8af315",
                             "from_date": "2026-08-01",
                             "to_date": "2026-08-31"})
Observation   {"count": 2, "truncated": false, "invoices": [
                {"id": "in_9f21", "date": "2026-08-01",
                 "total_minor": 8240, "status": "paid"},
                {"id": "in_8c04", "date": "2026-08-14",
                 "total_minor": 3600, "status": "paid"}]}

Thought       Two charges with different amounts on different dates, so this
              is not a duplicate. The second looks like a mid cycle change.
              Check the plan history for the same window.
Action        plan_changes({"account_id": "acct_4b71c9e20d8af315",
                            "from_date": "2026-08-01",
                            "to_date": "2026-08-31"})
Observation   {"changes": [{"on": "2026-08-14", "from": "team",
                            "to": "business", "prorated_minor": 3600}]}

Final         Two separate charges. The monthly subscription of 82.40 on 1
              August, and 36.00 on 14 August covering the prorated upgrade
              from Team to Business.

The third thought is the whole point of the arrangement. Nothing in the original goal said anything about plan changes. The model reached that question by reading two invoice amounts that did not match, which is a decision no plan written in advance would have contained.

Plan then execute as the other arrangement

The alternative separates deciding from doing. A planner produces the whole sequence in one call, with placeholders standing in for results that do not exist yet. Workers then run the steps and collect the evidence. A solver receives the goal and the evidence together and writes the answer.

The published form of this is ReWOO, from Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu and Dongkuan Xu, posted on 23 May 2023. They reported five times the token efficiency of the interleaved arrangement along with a 4 per cent accuracy gain on HotpotQA, a benchmark built from questions needing several hops to answer.

Plan, produced in one call before any tool runs

  #E1 = identify_account(email = "r.okafor@northwind.example")
  #E2 = find_invoices(account_id = #E1.account_id,
                      from_date   = "2026-08-01",
                      to_date     = "2026-08-31")
  #E3 = plan_changes(account_id = #E1.account_id,
                     from_date   = "2026-08-01",
                     to_date     = "2026-08-31")

Workers run #E1 first, then #E2 and #E3 together, since neither depends on
the other. Three results come back.

Solver receives the goal and the three results in one call and writes the
answer. The transcript is never resent, so each result is read once.

The gain and the loss are both visible in that block. Two of the three tool calls run at the same time, which a strictly interleaved loop cannot do, because each of its steps waits for the one before. The loss is that a planner committed to plan_changes before seeing the invoices. On this task the guess happened to be right. On the task where the two invoices are genuinely identical, the third step is wasted and the plan has no way to notice.

A useful reading is that the two arrangements differ in when the model is allowed to change its mind. Interleaving allows it at every step and pays for the privilege. Planning first forbids it and banks the saving.

The token cost of each arrangement

The same four step task priced both ways makes the difference concrete. The system prompt and tool definitions come to 1,600 tokens and are resent on every call.

The interleaved run reads 7,930 input tokens against 3,690, which is a little over twice as many for the same answer, and the gap widens with every extra step because each one carries the whole transcript again. The five times figure reported for ReWOO came from a benchmark with longer chains than four steps, which is the same effect further along.

Prompt caching changes the arithmetic in favour of interleaving, since the 1,600 token prefix is identical across every step. It does nothing for the growing transcript, which is different on every pass by construction.

The stop condition and the four budgets behind it

An agent needs an exit that belongs to the application, because the exit belonging to the model is the one that fails. Four budgets and one detector cover the cases that occur.

MAX_STEPS    = 12
MAX_TOKENS   = 60_000
MAX_SECONDS  = 90
PER_TOOL     = {"search_docs": 6, "lookup_order": 4, "issue_refund": 1}
DEFAULT_LIMIT = 3

def run(goal: str, tools: ToolSet, ctx: Session) -> Outcome:
    transcript = [system(SYSTEM_PROMPT), user(goal)]
    spent = Budget(tokens=0, calls=Counter())
    started = time.monotonic()
    recent = deque(maxlen=3)

    for step in range(MAX_STEPS):
        reply = model(transcript, tools=tools.visible_for(ctx))
        spent.tokens += reply.usage.total
        elapsed = time.monotonic() - started

        if reply.final_text is not None:
            return Outcome.answered(reply.final_text, spent, step)

        call = reply.tool_call
        signature = (call.name, canonical(call.arguments))

        if spent.tokens > MAX_TOKENS:
            return Outcome.stopped("token_budget", spent, step)
        if elapsed > MAX_SECONDS:
            return Outcome.stopped("wall_clock", spent, step)
        if spent.calls[call.name] >= PER_TOOL.get(call.name, DEFAULT_LIMIT):
            return Outcome.stopped("per_tool_limit", spent, step)
        if recent.count(signature) >= 2:
            # The same tool with the same arguments for a third time.
            return Outcome.stopped("no_progress", spent, step)

        spent.calls[call.name] += 1
        recent.append(signature)

        result = execute(call, ctx)          # the guard from the previous page
        transcript += [assistant(reply), tool_result(call.id, result)]

    return Outcome.stopped("step_cap", spent, MAX_STEPS)

Each budget catches something the others miss.

The step cap bounds the count of passes and is the one most teams have. On its own it lets twelve very expensive steps through.

The token budget bounds what the run can cost. It matters because the transcript grows, so the twelfth step is several times the price of the first and a step cap says nothing about money.

The wall clock bounds what the customer waits. A run at step four with a slow tool has already lost, whatever the other two budgets say.

The per tool limit bounds each tool separately, and it is the one that prevents damage. Six searches is a model working. Six refunds is an incident. The default limit exists so a tool added next month is bounded before anybody remembers to bound it.

The detector alongside them counts a repeated call. An agent that issues the same tool with the same arguments three times running has stopped making progress, because the third result will be identical to the first two. That check costs one comparison and catches the commonest runaway, which is a model retrying a failing call with cosmetic changes to the arguments.

One property of the code matters more than any single limit. Every stop returns a reason. A run that ends at per_tool_limit and a run that ends at wall_clock are different faults with different remedies, and an agent that returns the same generic failure for both leaves the team guessing. The reason belongs in the metrics, in the trace and in whatever a person sees.

Nothing in the loop above was designed to remember anything. The transcript accumulates because the code appends to it, growing with every observation until a budget stops the run or the window fills. That is a data structure nobody chose, and deciding what an agent keeps within a run and between runs is the next question.