Catching the AI Agent Failure Mode Your Loop Guard Can't See

Faizan Raza — Tue, 28 Jul 2026 01:11:27 +0000

Your agent's loop guard almost certainly has a blind spot

Most agent frameworks that bother to guard against runaway loops do it the same
way: hash each tool call's name and arguments, keep a sliding window of recent
hashes, and flag it if the same hash shows up twice. It's cheap, it's simple,
and it catches the most obvious case.

It also has three specific blind spots that let an agent burn real money while
looking perfectly fine on a dashboard that only checks for exact duplicates.

Fuzzy loops. An agent retrying the same failing action with a slightly
different argument each time (an incrementing attempt counter, a fresh
timestamp) never hashes equal to its own prior attempt, even though it's the
same unproductive retry every time.

Semantic oscillation. An agent alternating between two strategies,
check_status → retry_action → check_status → retry_action, with each
individual call looking "new" relative to the one immediately before it.

Productive-looking stalls. An agent whose every tool call is technically
unique (different tool, different arguments, every single time) but whose
actual results are boilerplate and whose context keeps growing regardless. An
exact-match detector sees zero duplicates and concludes everything is fine.

I built Wattage, an open-source
token-spend profiler and cost-regression gate for AI agents, because I kept
hitting all three of these in practice, and none of them showed up until the
bill did.

Measuring progress instead of counting duplicates

Rather than asking "have I seen this exact call before," Wattage's
nonconvergence detector measures progress directly, per iteration, using
five signals:

Evidence gain (E): how novel this iteration's tool/retrieval results are against everything seen so far in the loop (cosine novelty over a lightweight embedding).
State delta (S): how much the agent's chosen action actually changed from just the immediately prior iteration (a local, not cumulative, comparison, which is what keeps this a distinct signal from E).
Oscillation (O): periodic repetition in the sequence of actions taken (period 2 or more only), robust to argument noise because it keys on tool names, not exact arguments.
Growth-vs-information (G): how many new context tokens this iteration cost relative to how much genuinely new evidence arrived, so a loop can have low evidence gain yet still be judged on whether that repetition was cheap or expensive.
Goal proximity (P): only meaningful when an explicit goal signal is supplied; defaults to a neutral value otherwise, never a guess.

These combine into a per-iteration progress score, and a loop gets classified
as productive, thrashing (the same unproductive action repeated),
oscillating (a detected cycle), or stalled (context keeps growing,
evidence and state both stay flat: the exact-match blind spot). A loop that
genuinely recovers only gets credit for that if there's real, substantive,
different content in its later iterations, ending isn't the same as
succeeding.

The evidence, not a marketing claim

I didn't want to just assert this works better. So I built a hand-reviewed set
of 10 labeled synthetic loops (three productive, three thrashing, two
oscillating, two stalled), each specifically constructed so its label is
defensible on inspection, and benchmarked Wattage's classifier against a real
reference implementation of the SHA-256 exact-match baseline:

Classifier	Precision	Recall	F1
Wattage	1.00	1.00	1.00
SHA-256 exact-match	1.00	0.14	0.25

The baseline catches exactly one of the seven genuinely non-convergent cases:
the one where the arguments happened to be byte-identical every time. The
concrete case it misses entirely: a loop where every tool call is technically
unique, yet Wattage correctly flags it as stalled because the context kept
growing while the results stayed boilerplate.

Reproduce it yourself, no cherry-picking:

uv run python -m benchmarks.harness

It's not just loop detection, it prices everything

The convergence engine is the part I'm proudest of, but it's one of eight
detectors. Point Wattage at an OpenTelemetry GenAI trace (OTLP JSON, no
vendor lock-in, no proprietary SDK) and it prices every call against a real,
dated pricing snapshot and runs detectors for:

prefix_churn: a stable prompt re-sent instead of cached
cache_gap: caching attempted but under-redeemed by later reads
verbosity: output far beyond what the step needed
redundant_tool_calls: the same tool call repeated, exact or fuzzy
nonconvergence: the loop detector above
retrieval_thrash: repeated retrieval that never yields relevant results
model_mismatch: a pricier model doing work a cheaper one could handle
reasoning_overspend: heavy reasoning-token spend on a simple step

On a real captured agent trace (not synthetic), Wattage's prefix_churn fix
simulation shows a 44.7% cost reduction ($0.000199 to $0.000110) from
enabling prompt caching on a stable prefix, a small dollar figure because
it's a 3-turn demo trace, but the mechanism is identical at production scale.

Wattage never fabricates a number: an unpriced model leaves that call's cost
at zero and fails loudly rather than guessing, and an unmeasured quality
signal is reported as unmeasured, never assumed fine.

Try it

Fully offline, no API key:

uvx wattage report trace.json

And if you want this enforced automatically rather than checked by hand,
wattage ci is a cost-regression gate. It fails your build when a change
makes your agent measurably more expensive, the same idea as a
perf-regression gate, with a per-detector delta posted as a PR comment.

GitHub: https://github.com/faizannraza/wattage
Docs: https://faizannraza.github.io/wattage/