"The coordination of the builders is not direct. It is the work already done
that directs and triggers the work that follows."— Pierre-Paul Grassé, describing termites, 1959
1. The Loop
You have seen this. Every agent framework does it :)
> create_issue
x 422 validation_error
> create_issue (retry)
x 422 validation_error
> create_issue (retry, arguments tweaked slightly)
x 422 validation_error
Three attempts. Three identical failures. Then it apologises to you, which
somehow makes it worse.
The instinct is to blame the model. But look at what it had to work with:
422 validation_error
Tell me, from that string, whether retrying is worth it.
Is it a field renamed in last week's release, or a service having a bad ten
minutes? Same six characters either way. In one case retrying is exactly right
and works in thirty seconds. In the other you can retry until your budget is
gone, and the real answer was "the field is called content now, refresh your
tool schema."
The model has to guess. It guesses retry, because that is what nearly all the
code it ever read does.
2. Why "don't retry blindly" in your system prompt does nothing
The first instinct, once you notice this, is to write a rule.
When a tool call fails, do not retry immediately.
Consider whether the failure is transient before trying again.
It sounds reasonable. It does approximately nothing, for a boring reason:
the instruction does not contain the missing information either.
You have told the agent to consider whether the failure is transient. It still
has no way to find out. You have asked it to make the same guess, more
thoughtfully. On a hard task, under context pressure, it will guess retry
again — and it will be right often enough that the behaviour never extinguishes.
That last part is the trap. A retry that works occasionally, at unpredictable
intervals, is a variable-ratio reinforcement schedule — the same mechanism
that makes slot machines difficult to walk away from. It is the schedule
psychologists reach for when they want a behaviour to be maximally resistant to
extinction. Your agent is on it. So are you, at 1am, hammering the same test.
The temptation is to log the error and call it a trail. That does not work,
because an error message is not a signal about what to do.
A useful trace needs three parts:
┌──────────────────────────────────────────────────────────────────┐
│ THE ANATOMY OF A USEFUL FAILURE TRACE │
├──────────────────────────────────────────────────────────────────┤
│ 1. AN IDENTITY │
│ A stable id for "this exact failure", so two agents can │
│ tell they hit the same thing. Not the raw string: that one │
│ contains a request id and a timestamp and will never match │
│ anything again. │
├──────────────────────────────────────────────────────────────────┤
│ 2. AN OUTCOME, NOT AN INTENTION │
│ What the next agent tried, and whether it worked. "I │
│ refreshed the schema" is worthless. "I refreshed the schema │
│ and the call then succeeded" is the whole point. │
├──────────────────────────────────────────────────────────────────┤
│ 3. A DENOMINATOR │
│ Successes too, or the failure rate is meaningless. 100 │
│ failures out of 200 calls is an outage. 100 out of a │
│ million is a Tuesday. │
└──────────────────────────────────────────────────────────────────┘
Part 1 is fiddly and worth spelling out. These two are the same bug:
Repository 8823 rejected field body at 2026-09-11T14:02:11Z
Repository 41902 rejected field body at 2026-09-12T09:41:55Z
Compare them raw and you have two unrelated incidents forever. So you normalise
first — replace the parts that vary, keep the parts that mean something — and
hash what is left together with the service and operation:
text = URL_RE.sub("<URL>", text)
text = UUID_RE.sub("<UUID>", text)
text = TIMESTAMP_RE.sub("<TS>", text)
text = LONG_NUMBER_RE.sub("<N>", text)
Both lines collapse to one shape. Now they are one thing you can count. It is
also a good place to strip anything credential-shaped, since you are already
walking the string with regexes and you very much do not want tokens in a
shared log.
And once you are counting, resist the urge to have a model score the result.
Count it. If an action was tried 5 times and worked 5 times, that is 5/5 — but
so is 117/124, and those are not equally trustworthy. A Wilson score lower
bound folds sample size in for you: 5/5 scores about 0.57, 117/124 scores
about 0.89. Ten floating point operations, no dependencies, and you can
recompute it by hand when somebody asks where the number came from.
When there is not enough evidence, return that. Not a guess with a low
confidence bolted on — an actual "I don't know". Agents handle it fine.
6. The question I cannot answer alone
Here is the thing I keep coming back to, and cannot settle by thinking harder:
Do different people's agent failures actually overlap? 🤔
The theory says they should. We are all calling the same twenty MCP servers and
the same dozen public APIs, and when GitHub renames a field it renames it for
everyone at once. Your 422 on Tuesday and my 422 on Thursday are plausibly the
same 422.
But "obviously true" is where most wrong ideas live. It is equally plausible
that the interesting failures are all local — your auth setup, my rate limit,
their internal service — and that the shared surface is too thin for any of
this to matter. A pheromone trail nobody else walks is just a smell.
I do not know which world we are in. It is decidedly testable, and I do not
think anyone has tested it.
- What failure does your agent keep rediscovering? The specific one you have explained to it four times, in four different sessions.
- Was it a failure only you could have hit, or would anyone calling that service have walked into it too? That is the whole question above, in one concrete case.
- What do you do about it now? A line in the system prompt, a wrapper, a note in the repo, or nothing at all and you just eat it every time.
I am collecting answers to the middle one in particular. If enough people
describe failures that turn out to be the same failure, that settles it.
Top comments (6)
The overlap shows up most reliably right at the edge between public schemas and internal state. When GitHub or Stripe changes a response shape or rejects an untruncated payload, every setup hitting that endpoint catches the same 422. Where the trail breaks down across setups is local context: token refresh drift, SQLite locks held across async tool calls, or filesystem watchers triggering mid-write.In my own loops, system prompt instructions never stopped the cycle because an agent under context pressure treats advice as optional. The only thing that killed the retry loop cold was moving the circuit breaker into the tool dispatcher: if the normalized error signature repeats twice in a sliding window of three calls, the runner raises an unrecoverable exception and aborts the turn instead of letting the agent guess again.
Yeah, that makes sense. The shared/public failures are the interesting part to me too, especially schema changes, payload limits and other things everyone can hit. Also moving the stop condition into the dispatcher feels more reliable than hoping the agent listens to a prompt when it’s already stuck in a loop :D
We've actually got a retry_policy field on every tool response in our stack and honestly the hardest part was code review. People under deadline pressure just tag everything as transient and then you're back where you started. I caught about a dozen of those in a two-week sprint before we added a linter check for it.
The Grassé quote earns its place - stigmergy is exactly what's missing. Three identical 422s happen because the retry has no memory of the previous attempts as work, only as log lines; a colleague would leave a note saying "that door is locked, I tried it." Coordination through the work already done is a design choice, not an emergent property. Are you building the note-leaving, or naming the gap?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.