The agent used to take eleven turns to close a ticket. On the new model it takes four and misses a check, or it takes forty and trips the guard. The system prompt, the tool definitions and the task are unchanged. Nothing is broken; the loop is counting something that stopped meaning what it used to mean.
The symptom, and the error you hit first
The error is almost always your own. Every agent harness has a guard, and it surfaces as some variant of Agent exceeded maximum iterations (max_steps=12) or a silently truncated run that returns a partial answer with no exception at all. Because the string comes from your code rather than from the provider, the instinct is to look at the prompt — and the prompt is the one thing that did not change.
Two other presentations of the same underlying shift:
- The run gets shorter and the quality drops. Fewer turns, plausible answer, a verification step quietly skipped. This one does not raise anything and is found by users.
- Per-turn accounting stops adding up. Cost per run rises while turn count falls, or the other way around. That combination is diagnostic and is covered under causes below.
Why this is not a prompt bug
A prompt bug is a defect in what you asked for. This is a change in how the request is decomposed, and decomposition is a post-training property. The relevant fact is that models differ in how much planning they externalise as separate assistant turns versus how much they do inside a single response before acting.
That distinction is invisible to a harness. A loop that counts assistant messages is measuring how often the model chose to hand control back, which is a proxy for work done that holds only while the model’s habits hold. Swap the model and the proxy breaks even though the work is identical. Your budget was never a budget on work; it was a budget on a formatting habit, in the same way the parser in chain output format breakage was parsing a habit.
Four causes, and how to tell them apart
- Internal planning. A model that plans within one response emits fewer turns for the same work. Signature: turn count down, total output tokens flat or up, tool calls roughly unchanged. The work is happening; you are seeing less of it.
- Parallel tool calling. Several tool calls issued in one assistant turn collapse what was N turns into one. Signature: turn count down sharply, tool calls per turn above one, total tool calls unchanged. This also breaks any per-turn cost accounting and any logic that assumed one pending call at a time — see testing parallel tool calls.
- Eagerness. The model does more than asked: extra verification, extra searches, extra file reads. Signature: turn count up, tool calls up, cost up, task completed correctly. This one has a documented cause worth quoting — Anthropic advises that if your prompts previously encouraged the model to be more thorough or to use tools more aggressively, you should dial that guidance back, because current models are more proactive and may overtrigger on instructions earlier models needed (Anthropic, prompting best practices). Your anti-laziness stanza is now the bug.
- Suppressed narration. The model stops emitting the preamble your harness was reading. If anything downstream parses a
Plan:block or a post-tool summary to drive routing, it now sees nothing and takes a default branch. Signature: turn count unchanged, a specific branch of your own code stops firing. The same documentation notes current models may skip the verbal summaries after tool calls that earlier ones produced, and gives the prompt to ask for them back.
The four have distinct signatures, which is the point of listing them this way: turn count alone cannot distinguish any of them, and turn count alone is what most harnesses log.
Instrument before you re-tune
Log per run, not per turn, and log all of it before changing anything:
{
"run_id": "...",
"model": "...",
"prompt_version": "triage/2026-08-11",
"turns": 7,
"tool_calls_total": 14,
"tool_calls_per_turn_max": 4,
"tool_calls_by_name": {"search": 6, "read_ticket": 5, "close": 1},
"input_tokens": 48210,
"output_tokens": 3907,
"wall_ms": 21400,
"terminated_by": "model",
"outcome": "resolved"
}
terminated_by is the field people leave out and need most. It distinguishes a run the model ended from one your guard cut off, and without it the distribution of turn counts is censored at your limit — which makes the new model look tidier than it is and hides the runs that were truncated mid-task.
Collect this over a representative sample on both models with identical inputs. Do not compare against historical logs from a different prompt version; the whole point is holding everything except the model fixed, which is the same discipline the test dataset migration page applies to prompts.
Re-tuning the budget
- Remove anti-laziness and thoroughness prompting first, before touching any limit. If eagerness is the cause, this fixes it and every other change you were about to make would have been compensating for a self-inflicted instruction.
- Change the budget’s unit. Turns are not a stable measure of work across models. Budget in total tool calls and total tokens, with a turn limit left only as a very loose infinite-loop guard well above the observed distribution.
- Set the limit from the new distribution, not from the old constant: take the p99 of successful runs on the candidate and add headroom. A limit that clips real work is worse than no limit, because it fails silently in the middle of a task.
- Add a cost ceiling per run as the real guard. It is the constraint you actually care about and it is invariant to how the model chooses to decompose — the approach in tool loop cost budget stops.
- Make truncation loud. A run that hits any ceiling should emit a distinct outcome, not a partial answer that looks like a completed one, and it should be recoverable — the ground covered by workflow error recovery.
- Re-run the sample and confirm the truncation rate is near zero on successful cases before shifting traffic.
The durable lesson is that a step limit is a safety mechanism and not a specification. Once it is expressed in tokens and money rather than in turns, the next model swap moves the distribution underneath it without tripping it, and the detailed re-tuning in step limit re-tuning becomes a periodic adjustment rather than an incident.
Top comments (1)
Planning behavior is the migration surface people miss. Output quality can look similar while the model changes how it budgets steps, asks for tools, or abandons a branch. I would snapshot plans as well as final answers when testing a model switch.