Agent failure modes

Tools, a loop, memory and isolation each give an agent a capability, and each of them arrives with a way to fail attached. Acting produces a wrong action, a loop produces a run that will not end, memory produces a false fact that outlives the conversation which made it, and isolation produces two agents disagreeing with nobody reading. Those failures recur across every agent anybody has shipped, they have names, and each one has a control that bounds it.

What makes them a subject of their own is that none of them exists in a single call. One request either returns something usable or it does not, and the application sees which. An agent produces a sequence of decisions, so a fault at step three arrives as a plausible answer at step eleven with nine steps of reasonable looking work between the two. The industry ranking has moved to match. The OWASP GenAI Security Project published the 2026 edition of its top ten for language model applications in August 2026, and excessive agency rose from sixth place in the 2025 edition to third, with unbounded consumption also on the list.

The sections below take the five modes in turn, give the mechanism behind each and show the control. The last two sections cover approval gates on anything that cannot be undone, and the trace without which none of the rest can be investigated at all.

Five failure modes and the control for each

The five are separated by the mechanism underneath them, because a control follows from the mechanism and never from the symptom.

The table is worth reading twice for what it does not contain. Not one of the five controls is a prompt. Every one of them is code the application runs, and that is the property that makes them hold when the model behaves in a way nobody anticipated.

The runaway loop and the shared budget that bounds it

The simplest runaway is a loop with no cap, which the page on the agent loop settled with four budgets and a repeat detector. Two variants get past all of them.

The first is nesting. A parent with a 60,000 token budget that may spawn three workers, each with a 60,000 token budget, has a run budget of 240,000 tokens and a design that thinks it has 60,000. A worker that may spawn its own workers turns that into a number nobody can state.

class RunLedger:
    """One ledger for a run and every worker inside it. A nested worker spends
    from the same pool, so spawning cannot multiply the budget."""

    def __init__(self, tokens: int, seconds: float, max_depth: int):
        self.pool = Pool(tokens=tokens,
                         deadline=time.monotonic() + seconds,
                         lock=threading.Lock())
        self.depth = 0
        self.max_depth = max_depth

    def spend(self, tokens: int) -> None:
        with self.pool.lock:
            self.pool.tokens -= tokens
            if self.pool.tokens < 0:
                raise BudgetExhausted("run token budget spent")
        if time.monotonic() > self.pool.deadline:
            raise BudgetExhausted("run wall clock spent")

    def child(self) -> "RunLedger":
        if self.depth >= self.max_depth:
            raise DepthExceeded("a worker at this depth may not spawn another")
        nested = RunLedger.__new__(RunLedger)
        nested.pool = self.pool          # the same pool, never a copy of it
        nested.depth = self.depth + 1
        nested.max_depth = self.max_depth
        return nested

One line carries the whole idea. The child holds a reference to the parent's pool, so every token a worker spends is a token the parent no longer has, and the depth limit stops the structure growing sideways for ever.

The second variant is two agents handing work back and forth. Agent A asks agent B to check something, B replies with a question, A answers it and asks again, and neither has a counter because each one is inside its own first iteration. A per agent step cap never sees this. The shared ledger does see it, because the pair spends real tokens on every exchange.

Hallucinated tool calls and the closed registry

A model asked to choose among tools sometimes emits one that does not exist, and the failure has four shapes worth separating.

1  A tool name nothing declares
     {"name": "cancel_subscription", "arguments": {...}}
     There is no such tool. The model produced a name that fits the pattern of
     the other names and matches the task.

2  An argument key the schema never declared
     {"name": "find_invoices",
      "arguments": {"account_id": "acct_...", "include_drafts": true}}
     find_invoices has no include_drafts parameter.

3  A value outside the declared type or enumeration
     {"arguments": {"status": "pending"}}
     The enum lists paid, open, void and uncollectible.

4  An identifier for a record that does not exist
     {"arguments": {"account_id": "acct_0000000000000000"}}
     Well formed, correctly patterned, and not a real account.

The first three are closed by a registry lookup and a schema parse, which is the guard shown on the page about tool use, and they are cheap to catch because each one fails deterministically. The fourth is the dangerous one. The identifier is well formed, passes every pattern, and only the database knows it is wrong.

Three things reduce the fourth. The tool description states that an identifier comes from a lookup tool and is never constructed. The lookup tool is the only producer of identifiers, so a well designed set has no path to a fabricated one that the model has not been told to avoid. And the authorisation check runs on the session, so even a real identifier belonging to somebody else is refused.

Counting the first three is worth the small effort. A rising rate of unknown tool names usually means the tool set has grown past what its descriptions keep distinct, and the remedy is a better description or a smaller exposed set.

An action taken twice because the call was not atomic

A tool call is not one operation. It is a request, some work at the far end and a response, and each of the three can fail independently. An agent that sends a refund request and receives a timeout has no information about whether the refund happened, and a model reading a timeout will usually try again.

def execute_once(call: ToolCall, run_id: str, ctx: Session) -> ToolResult:
    # The key comes from the run and the call, so the same intent produces the
    # same key across a retry, a process restart and a resumed run.
    key = sha256(
        f"{run_id}:{call.name}:{canonical(call.arguments)}".encode()
    ).hexdigest()

    seen = idempotency.get(key)
    if seen is not None:
        metrics.increment("tool.replayed", name=call.name)
        return seen.result               # the first outcome, handed back again

    # Reserve before acting. A crash between the reservation and the write
    # leaves a record a sweeper can resolve against the downstream system.
    idempotency.reserve(key, call, ctx)

    # The key travels downstream as well, so the payment provider refuses a
    # duplicate even if this process never learns the first one succeeded.
    result = payments.issue_refund(**call.arguments, idempotency_key=key)

    idempotency.complete(key, result)
    return result

Two details decide whether this actually works. The application derives the key from the intent and never generates a fresh one, since a new key on a retry is a new refund. It also sends the key downstream, because a store this process owns helps nobody once the process dies between the reservation and the write.

The same reasoning applies to every tool that changes something. Sending an email, opening a ticket, posting a message and updating a record are all operations a well meaning retry will perform twice.

A plan that drifts away from the goal

At step fourteen the goal sits at the front of a 40,000 token window and the last twelve steps all concern a subproblem the agent invented at step four. Attention is finite and recency is strong, so the agent optimises the subproblem and answers a question nobody asked. The answer is coherent, which is what makes drift hard to spot in review.

Two controls hold it. The first restates the goal on every pass, at the front, with an explicit completion test.

<goal priority="highest">
Work out why account acct_4b71c9e20d8af315 was charged twice in August 2026
and write one paragraph explaining it to the customer.

This is complete when the paragraph names every charge, its date, its amount
and its cause, and every amount in it came from a tool result in this run.
</goal>

The completion test is the part that earns its tokens. A goal with no test leaves the model to decide when it has finished, and a model twelve steps into a subproblem decides that far too easily.

The second control asks a small model, every few steps, whether the recent work serves the goal.

if step >= 6 and step % 3 == 0:
    verdict = call(DRIFT_PROMPT, goal=goal, recent=transcript[-6:],
                   model=SMALL, schema=DriftSchema)
    if verdict.serves_goal is False:
        metrics.increment("agent.drift", reason=verdict.reason)
        return Outcome.stopped("drift", spent, step, note=verdict.reason)

Below is a goal and the last three steps an agent took.

Answer two questions.

  serves_goal   true if the last three steps move towards the completion test
                in the goal. false if they are working on something the goal
                does not ask for.
  reason        one sentence naming what the steps are doing, in the agent's
                own terms.

Judge only the steps shown. Do not attempt the task yourself.

<goal>{{goal}}</goal>
<steps>{{recent}}</steps>

The check costs one small model call every third step, which at the prices of the cost page is a rounding error against the run it stops.

A tool that fails quietly

The last mode produces the most convincing wrong answers, because nothing anywhere reports an error. A search returns an empty list when its filter was wrong. An index returns results four hours out of date. A downstream service returns a 200 response carrying an error message in its body.

A model receiving an empty array reads it as a finding. It then writes that the customer has no invoices in August, which is a sentence somebody will act on.

Four results a search tool can return, and the model can tell them apart

  {"status": "ok", "count": 3, "invoices": [...]}

  {"status": "no_match", "count": 0, "invoices": [],
   "note": "The filter excluded every row. status=void was requested and this
            account has no void invoice in the range."}

  {"status": "degraded", "count": 3, "invoices": [...],
   "note": "The index is 4 hours behind. Anything raised today may be
            missing. Treat this list as incomplete."}

  {"status": "failed", "error": "upstream_timeout", "retryable": true,
   "next_step": "Call again once. If it fails twice, report that invoices
                 could not be read and stop."}

A bare empty array means all four of these, and a model will choose whichever reading lets it continue. A status field costs nothing and removes the choice.

The same separation belongs in the metrics. The rate at which each tool returns no_match and degraded is the earliest signal that something upstream has changed, and neither of those shows up in an error rate or an alert on exceptions.

Approval gates on the calls that cannot be undone

A control that stops an agent taking an action is a gate, and its quality is decided entirely by what the approver is shown.

The agent is asking to run a tool whose effect cannot be undone.

  tool        issue_refund
  account     acct_4b71c9e20d8af315      Northwind Ltd
  amount      GBP 82.40
  reason      "Duplicate charge on 2026-08-01"

  evidence    invoice in_9f21   2026-08-01   GBP 82.40   paid
              invoice in_8c04   2026-08-14   GBP 36.00   paid
              plan change       2026-08-14   team to business, prorated 36.00

  requested   step 6 of run r_7c19ae, started 14:07:22

  [ Approve ]   [ Deny ]   [ Deny and stop the run ]

The evidence block is what makes this a review rather than a formality. An approver reading it can see that the second charge is a prorated upgrade and that the refund should therefore be refused, which is a judgement no summary from the agent would support. A gate showing only that the agent would like to issue a refund produces a button people press.

Denying and stopping the run earns its place as a separate choice. When an approver denies one call and lets the run continue, the agent usually asks again two steps later with a slightly different argument.

The trace that makes any of this visible

Every control above produces a signal, and a signal nobody can reconstruct is not worth much. One run therefore writes one trace, keyed by a run identifier, holding the whole of what happened.

1. The goal, the system prompt version and the model version.
2. Every model call, with its full input, its output and its token counts.
3. Every tool call, with its arguments, its result and its elapsed time.
4. Every guard decision, meaning allowed, denied, pending approval or replayed.
5. Every budget reading at every step.
6. The stop reason, which is the field that turns a failed run into a category.

A trace of that shape answers the question a team actually asks after an incident, which is what the agent knew at the moment it made the wrong choice. It also makes a failed run into a test case, since replaying the same inputs against a changed prompt shows whether the change fixed anything.

The stop reason deserves its own dashboard. A week in which per_tool_limit stops rose and step_cap stops fell describes a different problem from the reverse, and an application returning a single generic failure for both hides the distinction that would have named the fault.

None of this can be tested the way ordinary software is tested. The same input produces a different path through the loop on two consecutive runs, so an assertion on one exact sequence of calls fails for a run that was perfectly good. A test suite here needs cases with known acceptable answers, a way to grade an answer that is never byte identical, and a threshold agreed before anybody looks at the result. That is a different discipline, and it is the one the next module builds.