Same Call, Same Result: A No-Progress Detector for Agent Tool Loops

An agent sat on step 7 for forty minutes. It was not hung. It was repeating.

Every turn it re-ran the same search. Every turn the tool returned the same twelve lines. Then it re-planned the same fix.

The trace above is illustrative. Step counts and timings are placeholders, not measurements.

A diff between two runs will not catch this. A loop lives inside a single run.

Why run-level diffing misses a loop

Diffing answers "what changed between attempt A and attempt B". A stuck agent answers a different question: "what did not change inside attempt A".

• The diff shows two identical branches. That looks like stability, not failure.
• Token and latency metrics rise smoothly. No spike marks the turn where progress stopped.
• The final answer often looks plausible. The loop burned budget, then produced something.

You need a second signal: progress per tool call. Progress is measurable without understanding the task.

Give every tool call a signature

A tool call is a tuple: tool name, normalised arguments, result digest, status. Hash the tuple and you get a signature you can count.

Normalise before you hash

Raw arguments are full of noise. Drop volatile fields or every call looks unique.

• Remove timestamps, request IDs, span IDs, nonces, and temp paths.
• Sort object keys so key order cannot create false variety.
• Canonicalise paths and whitespace before hashing.
• Hash the result too. Same call plus different result is progress, not a loop.

That last rule matters most. A polling call repeats by design and still makes progress.

Build the progress ledger

# progress_ledger.py
import json, hashlib
from dataclasses import dataclass

VOLATILE = {"ts", "timestamp", "request_id", "trace_id", "span_id", "nonce"}

def stable(obj):
    """Deterministic JSON: sorted keys, volatile fields removed."""
    if isinstance(obj, dict):
        return {k: stable(v) for k, v in sorted(obj.items()) if k not in VOLATILE}
    if isinstance(obj, list):
        return [stable(v) for v in obj]
    return obj

def digest(obj, n=12):
    blob = json.dumps(stable(obj), sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(blob.encode()).hexdigest()[:n]

@dataclass
class Step:
    seq: int
    tool: str
    args: str
    result: str
    status: str

def load(path):
    steps = []
    with open(path) as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            ev = json.loads(line)
            if ev.get("type") != "tool_call":
                continue
            steps.append(Step(
                seq=ev["seq"],
                tool=ev["tool"],
                args=digest(ev.get("args", {})),
                result=digest(ev.get("result", {})),
                status=ev.get("status", "ok"),
            ))
    return steps

def ledger(steps):
    seen, rows = {}, []
    for s in steps:
        sig = f"{s.tool}:{s.args}:{s.result}:{s.status}"
        history = seen.setdefault(sig, [])
        history.append(s.seq)
        rows.append({
            "seq": s.seq,
            "tool": s.tool,
            "signature": sig,
            "repeat_of": history[0] if len(history) > 1 else None,
            "count": len(history),
        })
    return rows

def stall_score(rows):
    if not rows:
        return 0.0
    repeats = sum(1 for r in rows if r["repeat_of"] is not None)
    return repeats / len(rows)

The ledger is append-only and cheap. One pass over the trace file, no model calls.

Run it on a trace

Your trace needs three fields per tool event: seq, tool, and args/result. If you already log JSONL spans, you have them.

python - <<'PY'
from progress_ledger import load, ledger, stall_score
rows = ledger(load("traces/run_7f2a.jsonl"))
for r in rows:
    print(r["seq"], r["tool"], r["count"], r["repeat_of"])
print("stall_score", round(stall_score(rows), 3))
PY

Output shape for a loop-shaped trace:

seq	tool	count	repeat_of
5	search	1	None
6	read_file	1	None
7	search	2	5
9	search	3	5
11	search	4	5

stall_score near 0.4 with four repeats of one signature is a loop. A healthy run trends toward 0.0.

Break the loop without killing the run

Detection is postmortem. You also want a live gate. Keep it tiny.

class StallDetected(Exception):
    pass

class NoProgressBreaker:
    def __init__(self, limit=3):
        self.limit, self.last, self.hits = limit, None, 0

    def observe(self, tool, args, result):
        sig = f"{tool}:{digest(args)}"
        cur = (sig, digest(result))
        if cur == self.last:
            self.hits += 1
        else:
            self.last, self.hits = cur, 1
        if self.hits > self.limit:
            raise StallDetected(f"{sig} repeated {self.hits}x with identical result")

Then pick a policy. Do not hard-fail on the first repeat.

Repeats	Result digest	Action
1-2	changed	Continue silently
3	unchanged	Inject a stall report into the context, allow one re-plan
5	unchanged	Break the loop, return partial work plus the ledger
any	changed every time	Not a loop. Cap cost instead.
any	status=error	Escalate as a tool failure, not a stall

A stall report is more useful than a stack trace. It names the signature, the repeat count, and the first seq where it appeared. The model often changes strategy when it sees that.

Test the detector with three fixture traces

Write fixtures by hand. Do not wait for a production loop.

import pytest
from progress_ledger import load, ledger, stall_score

@pytest.mark.parametrize("fixture", [
    "fixtures/loop.jsonl",
    "fixtures/progress.jsonl",
    "fixtures/poll_idempotent.jsonl",
])
def test_shapes(fixture):
    score = stall_score(ledger(load(fixture)))
    if fixture.endswith("loop.jsonl"):
        assert score > 0.5
    elif fixture.endswith("progress.jsonl"):
        assert score == 0.0
    else:
        assert score > 0.0  # known false positive, see limitations

Three files, three behaviours. The third one is the important one: it keeps you honest about what the detector cannot do.

Where a free tier fits into this loop

The detector is model-agnostic. It only reads tool calls. That makes it a good fit for testing the same task against a second model without paying twice for the trace.

This is where MonkeyCode is relevant. The open-source project offers free model access and a free server option, and the operator states a 10M-token allowance on the free tier. Terms change, so confirm the current numbers in the repo before you plan capacity around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A practical use: run your stalled task on the free tier, capture JSONL spans, and feed both traces to the same ledger. If the loop signature matches across two models, your prompt or tool schema is the cause. If only one model loops, it is a model-behaviour issue. That distinction saves hours.

If you want a second trace without a second invoice, the free server option is a reasonable starting point.

Limitations and known false positives

This is a narrow detector. It has real failure modes.

• Idempotent polling. Cache checks and status polls repeat with unchanged results by design. Allowlist them by tool name.
• Nondeterministic results. If the tool returns a fresh UUID or timestamp each call, the result digest never matches. Normalise or exclude that tool.
• Aggressive normalisation. Drop too many fields and two genuinely different calls collapse into one signature. You get false loops.
• No causal answer. The ledger says progress stopped. It never says why the plan was wrong.
• Non-tool reasoning loops. An agent can circle in pure text with zero tool calls. This detector is blind to that.

Set the repeat limit per tool class, not globally. search and deploy deserve different thresholds.

Who should not use this

Skip it if your agent makes one model call per request. There is no loop to detect.

Skip it if you have no structured tool log yet. Fix logging first, then come back.

Skip it if your runs are fully deterministic and short. A repeat count under three is noise, not signal.

The reusable part

Four lines of policy, one hash, one pass over a file. That is the whole mechanism.

The ledger survives model swaps, prompt rewrites, and vendor changes. Tool calls are the stable interface. Count them, hash their results, and stop paying for turns that change nothing.