Not All Repair Helps: What I Learned Trying to Fix a Failing AI Agent

Picture a moment every person who runs an AI agent knows. A task is halfway done and starting to go wrong. The agent took a weird turn a few steps back and now it is confidently heading somewhere bad.

You have to decide fast on this. Do you step in? And if you do a quick "wait, check your work" nudge will that actually fix it? Or do nothing? Or worse knock a run that was about to recover on its own off the rails?

That question is the whole project. Here is the honest short version of what I found.

Detecting a failure is not fixing it

A lot of recent agent research is about failure attribution — figuring out which step in a long run broke everything. Useful but it stops one step short of what you need when you are on call. Knowing where it broke is not the same as knowing what to do about it.

So I asked a blunter question: given a failure, which fix actually recovers the run and which ones quietly make it worse?

To answer it without fooling myself I rewind each failing run to the exact step where it went wrong, apply one fix, let it play forward and check the real answer against a hard ground truth no LLM grading another LLM. And I always compare against a "do nothing" control, because some runs recover on their own, and I did not want to give my fixes credit for that (or miss a "fix" that's actually worse than leaving the agent alone).

What a capable agent actually gets wrong

First surprise: a decent agent mostly doesn't fail in the dramatic ways people worry about. It rarely loops, rarely forgets to answer, rarely fumbles a tool that throws an error in its face.

It fails in two quieter ways and both are the same underlying mistake: acting on the surface of the situation instead of the real thing underneath.

• It makes up an answer it could have looked up. The fact it needs is sitting right there behind a tool call it just never makes, so it fills the gap with something plausible. Reads "manager: #202," never looks up who #202 is, asserts a name anyway.

• It trusts a tool that didn't complain. Give it a date like 20260730 or a time like 1430 and it hands that straight to a calculator as if it were a number. The calculator doesn't error — it returns a number and that non-error result becomes a stamp of approval on a calculation that never made sense. The tool "validated" nonsense.

That second one is sneaky because it slips through how we usually categorize errors. Nothing errored. A real tool really ran. Any monitor watching for error signals looks right past it.

The finding I like most: it is about permission, not politeness

Here is the part that changed how I think about fixing agents.

I tested a bunch of repair nudges. Some phrased as commands some as statements. I expected the phrasing to matter. It didn't. What mattered was narrower: does the nudge give the agent permission to go back and do the missing work?

Look at these two. Grammatically almost identical. The only difference is one short clause telling the agent it is allowed to check.

Repair nudge	Recovery
"The answer must be the manager's name."	0.16
"The answer must be the manager's name — use tools to verify."	1.00

Same sentence, one clause of difference. That is the gap between the agent
shrugging and re-asserting its made-up answer, versus actually going back, making the lookup it skipped, and getting it right. I call it action-licensing: a fix works when it licenses the corrective action not when it is worded a certain way.

One honest note, because it is the spirit of the whole thing: I originally thought a different phrasing style was the cause. More data killed that idea — the "winner" just happened to also be the one granting permission. I retracted the old explanation and kept the real one.

Some failures you can't talk your way out of

Skipped-lookup is recoverable. The "tool validated nonsense" one is not — at least not with words. I threw every text-level fix at it and recovery stayed near the floor.

Why is kind of elegant: the thing that corrupted the run was not the prompt, it was a tool result. The agent is holding a clean, well-formed, completely wrong number that a tool "confirmed." Asking it to re-read the question doesn't remove that false stamp of approval, because the stamp does not live in the prompt. You can't reach it from there.

Useful line to draw: some failures are prompt-shaped and some are not and no amount of prompt-fiddling fixes the ones that are not.

Two more things worth knowing

Catch it early. The deeper into a task the agent gets before you catch the skip, the less any fix buys back. Command-style nudges that recover well early collapse toward zero when you catch the mistake late.

Beware cross-model comparisons. I tried a second, unrelated model. The
make-up-an-answer failure showed up there too, so it's not a one-model quirk. But I couldn't test recovery on it — it wasn't reliable enough at calling tools in the first place, and every fix depends on the agent re-issuing a tool call. So "the fix failed" and "the model can't call tools" become impossible to tell apart.
If you benchmark agent recovery across models, check each one clears a basic tool-reliability bar first, or you'll measure the wrong thing.

What this is, and is not

Plainly, so I don't oversell it: all the recovery numbers come from one primary model (a second confirmed the mechanism but couldn't test recovery). It is a clean, synthetic sandbox — that is what buys the hard ground truth, but it is not a messy real-world workload. And a few of the smaller numbers are directional trends, not precise measurements. This is an early, careful result, not a final word.

If you build agents, three takeaways

1. When you nudge a stuck agent, give it permission to redo the work — don't just restate what you want.
2. Not every failure is prompt-fixable. If a tool "validated" bad data, talking to the model won't undo it that is a job for tool-side guardrails.
3. Catch failures early, and don't benchmark recovery on a model that can't reliably call tools.

Comments

[Mads Hansen]

The distinction between protocol success and domain validity is crucial here. A calculator returning 200/OK only proves it accepted bytes, not that a date was a legitimate numeric operand. Tool contracts could make this failure repairable earlier by using semantic types (LocalDate, Instant, Money) instead of generic strings/numbers, rejecting ambiguous coercions, and returning provenance with the parsed interpretation. I’d also invalidate all downstream observations that depend on a bad tool result when rewinding, rather than leaving the false “verified” value in context. That turns repair into state correction, not another prompt layered over corrupted evidence.

[Ayush Singh]

Exactly the tool returned 200 and the input made sense are two totally different things, and the agent treats them as the same. I love the semantic-types point as if the tool takes a local date or a money instead of a bare number a date-as-integer never even gets in the door. The failure gets caught at the type not three steps later.

And your rewind idea is the real upgrade. Right now my repair drops a nudge on top of a context that still holds the bad verified value so I am arguing with corrupted evidence. Invalidating everything downstream of a bad tool result before replaying turns it into actual state correction instead of another prompt on top of the mess. That is cleaner than what I did, and it is going on to be in the list.

Thanks for providing me the better insights .

[Zira]

The control comparison is the part I would keep in every recovery benchmark. I would add one more split: did the repair restore the missing observation, or merely make the next step look plausible? For skipped lookups, log the required evidence and the tool call that produced it; for malformed inputs, validate at the tool boundary and return a typed failure instead of a numerically valid result. That makes “action-licensing” measurable without letting a successful tool invocation count as proof. Have you tried replaying the same repair with the tool output withheld to separate recovery from prompt compliance?

[Ayush Singh]

On your first split "did it restore the missing observation or it just make the next step look plausible?" -- I even did part of this but not as cleanly as you are describing it. My ground truth was the final answer checked deterministically no LLM judge. And when I dug into the replays I actually looked at how the good arm won and under the licensing nudge the model actually issued the missing lookup and returned the fetched value while the neutral arm made no tool call and just reasserted its wrong answer. So I have evidence it was the real recovery not an plausible-looking guess.
But you are right that I confirmed that by reading traces not by logging "here's the required evidence and the exact tool call that produced it" as a first-class metric. Turning that spot-check into a logged split is a clean upgrade and actually I am going to steal it.

On validating at the tool boundary its 100% yes. That is the fix for the failure I could not rescue with words. If the calculator checks its input and returns a typed error instead of a happily wrong number for a date then the silent failure stops being silent. The model can act on an error it can't act on a lie that looks like a valid number.

And your last question . No I haven't replayed a repair with the tool output withheld and now I really want to. That is a much better way to separate "genuinely used the evidence" from "just complied with the prompt" than anything I have. If the model still recovers when you starve it of the real observation, that is guessing, not recovery and my current check would not fully catch that. Straight onto the list.

Thank you for the comment .

[Julian Neagu]

The “permission to redo the work” part matches what I’ve seen building agents. Sometimes the model knows the right move, it just thinks the previous path is still valid. Giving it a clear reason to revisit the step works better than adding more instructions.

[Ayush Singh]

Yeah that is exactly what I found too. The model usually is not missing the capability it just believes it is already finished. More instructions do not help if it doesn't think there is anything to revisit. The moment you give it permission to go back and verify the behaviour changes.
Thanks for sharing your perspective.