Shaan

Posted on Oct 25

Agents catch tool errors. Wrong values are harder.

#agents #ai #llm #testing

Most agent benchmarks answer one question: when the tools work, does the agent finish the task? Production asks a different one. Services time out. Endpoints get retired. A parameter gets renamed in a minor release. A cache returns a stale value that is correctly typed and quietly wrong. What does the agent do then?

A new study, "Loud Failures, Quiet Failures" by Obada Kraishan (arXiv, October 2026), takes a direct run at that question. Instead of scoring success in a clean environment, it wraps the tool layer of the Berkeley Function-Calling Leaderboard's multi-turn environments in a fault-injection layer, breaks one thing at a controlled moment, and records what the agent does at each stage: does it treat the result as a problem, does it change its plan, does it still reach the goal, does it get stuck retrying.

A commenter on my earlier piece on agent failures asked how you validate a response that looks fine but is broken. This study measures that gap directly. The short version: when a tool returned an explicit error, agents flagged it in 91.3% of trials. When a tool returned a plausible wrong value with no error signal, that fell to 58.8%, and in 39% of those trials the agent showed no reaction at all.

The setup, briefly

The details matter here because the conclusions are only as good as the measurement:

24 multi-turn tasks from the BFCL, where environments are executable, stateful Python objects: a file system, a trading interface, travel booking, vehicle control, messaging, ticketing. Median of five user turns per task.
Four injected faults. Three are loud: a timeout, a tool that disappears from the registry, and schema drift (a renamed parameter, rejected with an unknown-argument error). One is quiet: silent corruption, where the call succeeds and returns a well-formed value whose content is wrong. A number moved by 20 to 60 percent, an element dropped from a list, a field replaced.
Six models in three matched pairs, each family contributing a reasoning variant and its instruct sibling: claude-haiku-4.5-thinking vs claude-haiku-4.5, deepseek-r1 vs deepseek-v3, and qwen3-thinking vs qwen3-instruct. Reasoning variants ran with a 2,048-token thinking budget. 1,920 trials total, temperature 1.0, a 15 tool-call cap per trial.
Two measurement choices worth stealing. Recovery is scored against the same model's own fault-free run, so a weak model is not blamed for a fault. And every fault effect is judged against run-to-run variation, because two clean runs of the same task end in the same state only 63.3% of the time. Agents are stochastic; without that baseline you would blame faults for ordinary randomness.

Finding 1: agents watch the channel, not the content

When a tool returned an explicit error, agents treated the result as a problem in 91.3% of trials. When a tool returned a plausible wrong value, they did so in 58.8%. For calibration, on clean trials where the target call succeeded they reported a problem 26.8% of the time. That baseline is not pure noise: the environments emit genuine errors of their own, which agents mostly handled correctly.

[image: diagram-detection.png]

The part that should bother you: the three loud faults demand completely different responses. A timeout calls for a retry, a missing tool calls for a new plan, a renamed parameter calls for a reformulated call. Yet detection rates for all three land within a single point of each other (91.6%, 91.2%, 91.2%). The quiet fault sits 32 points lower. What separates them is not severity or difficulty. It is whether an error string exists to read.

Under silent corruption, 39% of trials drew no reaction in the agent's written output. For the arithmetic: 58.8% were flagged as a problem, 39% drew no reaction, and the small remainder fell into the study's middle codings rather than a clean flag-or-ignore. Under loud faults, the no-reaction rate was 4%. What a quiet miss looks like in practice, from the paper's examples: an agent reads a wrong value and continues with it, sometimes reconciling it with what the conversation had established earlier. The wrong values involved are the kind range or consistency checks could catch: a fuel level that exceeds the tank, an item gone from a list that was longer a moment ago, a symbol that does not match the company asked about.

One caveat on the word "detection": it is coded from what the agent wrote, so an agent that noticed something and said nothing counts as not noticing. With that in mind, the pattern inverts a comfortable assumption, as the paper points out. Loud failures at least surface as errors a monitor can see. Quiet failures arrive looking exactly like success, and they are the ones agents flagged least.

Finding 2: in this study, reasoning models flagged less and acted more

Each reasoning model was paired against the instruct sibling from its own family, on the same task, fault, and seed. The result is a clean dissociation:

[image: diagram-reasoning.png]

Flagging: -9.3 points for reasoning variants, pooled over 450 matched pairs (p < .001). Lower in all three families, clearest for Qwen3 (-16.6 points, p = .003).
Replanning: +10.4 points (p < .001). Reasoning variants changed their next action more often.
Recovery: -2.4 points, p = .512. That is no statistically detectable difference, not proof the two are equal.

So in these matched pairs, the models that deliberate more flagged fewer problems and changed course more often, with no detectable recovery difference either way. One reading the paper offers, consistent with trace-level work it cites, is that the deliberation phase is better at generating an alternative course of action than at checking a returned value against what the task implies.

The practical version is narrow: this study does not support paying for the reasoning tier on the assumption that more deliberation means more caution under tool faults. Three matched pairs cannot establish a general property of reasoning models, and the per-family effects vary in size.

Finding 3: what a fault actually costs

Judging cost requires the honesty of that 63.3% baseline. Against it, only one fault showed a statistically clear drop in recovery: a missing tool, at 39.9% (p < .001). Timeouts (59.6%), schema drift (59.2%), and corruption (60.4%) showed no detectable drop. Read that carefully. With 24 tasks and a noisy recovery measure, the absence of a detectable effect is not proof that those faults are harmless. What the study found is one clear, measured loss, and it came from losing a capability: the one fault no retry can undo.

Corruption deserves a footnote here, and it is not reassuring. Recovery in this study is agreement in environment state. A wrong value that the agent reads but does not write changes nothing in the environment; what it changes is what the agent reports. Corruption looking no worse than baseline in end-state terms says nothing about whether the final answer quoted a wrong number.

The effort numbers matter for anyone paying per call:

After a fault, agents called the same tool three or more times in a row in 12.5% to 22.2% of trials, most often under corruption (22.2%) and timeouts (21.4%). Strictly identical repeats, same tool and same arguments, were rare at under 2%. The rest are retries with changed arguments, which are not identical-call loops but can still loop.
Extra steps relative to a clean run: +0.5 for timeouts and corruption, +1.5 for a missing tool, +1.9 for a renamed parameter.
Trials hit the 15-call cap 24.8% to 31.7% of the time under faults, against 23.2% when nothing goes wrong.

One more subtle result: flagging is not the same as fixing. Pooled across conditions, trials where the agent treated the result as a problem recovered at 55.6%; trials where it did not, at 55.7%. That pooled comparison is confounded (corruption is both the least-flagged and one of the more recoverable faults). Holding fault type fixed, flagging is associated with recovery (odds ratio 1.51, p = .004), but the link weakens once the model is also held fixed (p = .098). Detection may be worth having; this study cannot show it is sufficient.

Finding 4: asking the agent to check its work did not help here

The obvious fix is a line in the system prompt. The study tested it in a second arm of 480 trials on the Claude pair: "after every tool call, briefly check whether the result is what you expected."

Detection moved +0.6 points for the instruct model and +2.5 for the reasoning variant. Neither is statistically significant. Recovery, repetition, and false alarms were likewise unmoved. Under quiet failures specifically, detection went from 60.6% to 59.6%.

Scope this correctly: one prompt, one model family. It is not a verdict on metacognitive prompting in general. There is also a real contrast with earlier work. Sun et al. (EMNLP 2024) found that a short disclaimer improved detection when models judged a single output on request. Judging one output when asked is a different task from noticing an anomaly mid-task while carrying a plan, which may be where a general instruction loses its force. What this arm does show is that a natural first remedy produced no detectable effect in this setup.

What to do differently

The paper's deployment implications, translated:

Validate values, not just errors. Checks belong where a value enters the agent's context: schema validation for shape, semantic checks for meaning (a percentage above 100, a fuel level above tank capacity), and cross-checks against state the agent saw earlier. Error monitoring and the agent share the same blind spot: a failure that stays inside the expected format.
Treat detection as the start, not the fix. Flagging a problem and recovering from it are separate skills, and this study found only a modest, unstable link between them.
Have a fallback for capability loss. The one statistically clear recovery drop came from a tool disappearing. Retries cannot undo a missing capability; a fallback path or a graceful degrade can.
Budget for retry behavior. Three or more consecutive calls to the same tool showed up in up to 22% of fault trials. That is latency and spend in production, and it belongs in how you set step caps and timeouts.
Baseline before you blame. As the paper notes, with a single clean run as the reference, three faults would have looked harmful whose end states in fact stayed within run-to-run variation in this data. Judge fault effects against repeated fault-free runs.
Fault-inject your own stack. The harness, frozen task suite, trajectories, and analysis code are released under MIT (linked below; the README claims every number reproduces offline in under a minute, which I have not run myself). It is a starting point for running this class of test against your own tools.

The limits, stated plainly

This is one study, and it deserves the same skepticism as any other:

It is an arXiv preprint, single-author (the repo README says IEEE CogMI 2026, to appear), run on simulated environments from one benchmark family. The four fault types are a chosen set, not an exhaustive one.
Detection is coded from the agent's written output by a model judge, validated against human coding on 92 double-coded trials at 85.9% agreement (kappa = .69). An agent that noticed something and said nothing counts as not noticing.
The reasoning comparison rests on three matched pairs. It is clearest for Qwen3, holds for Claude only on matched pairs, and one of the three reasoning models had interface quirks (DeepSeek-R1 sometimes wrote tool calls as plain text that never executed). Three pairs cannot establish a general property of reasoning models.
These are mid-tier models (Claude Haiku 4.5, DeepSeek R1/V3, Qwen3), not the largest frontier systems. The main reason to take the direction seriously despite the sample is that it agrees with earlier independent work: Sun et al. on silent tool errors, and ToolMaze, which found recovery falls hardest under implicit failures.
Recovery is end-state agreement with a single independent fault-free run, and two such runs agree only 63.3% of the time. The paper judges fault effects against that baseline rather than against perfect agreement.

The bottom line

In this study, agents responded to the error channel far more than to the content of what a tool returned. When a failure announced itself, they flagged it in 91.3% of trials. When it arrived looking exactly like success, they flagged it in 58.8%, and in another 39% the wrong value drew no reaction in the agent's written output. How often a wrong value like that reaches a user-facing answer is a separate question this study does not measure, and I am not going to pretend otherwise.

A prompt line asking the agent to check its work produced no detectable change in the arm that tested it. The stronger lever these results point to is structural: validation in the path of the data, which is the same lesson production systems learned a long time ago.

Sources:

DEV Community