I've been reading BEACON from Zhejiang University's ZJU-REAL lab (arXiv:2605.06078), and the more I read it the more directly applicable it felt to the long-horizon agent work we're doing. So I'm turning that into a series.
The framing
To build a long-horizon agent that actually works, I think two things have to be right:
1. The agent keeps making progress toward the goal
- Context compaction — what is safe to forget
- Milestone design — how do you know a step is really done
- Spin prevention — when it gets stuck, something has to notice
2. The agent can reflect and improve
This post covers spin prevention, from the first group. It goes first because it's the one that hurt us most.
The symptom
We had user reports, and saw it internally too: some very long tasks just stop moving forward partway through.
Look at the logs and the agent is busy — reading files, searching, running commands, never idle. But half an hour goes by and nothing has advanced.
My first explanation was context loss: compaction dropped the record that "this path was already tried," so the agent went and tried it again.
Digging through the code and the logs, that turned out to be only half the story.
We already had guards. Three tiers of them.
| Tier | Criterion | Thresholds |
|---|---|---|
| Exact repeat | Tool name + canonicalized args, byte-identical |
LOOP_WARN=3 warn / LOOP_HARD=5 force-stop |
| Near-duplicate | Identical except volatile id / timestamp fields |
NEAR_DUP_LOOP_WARN=6 / NEAR_DUP_LOOP_HARD=12
|
| Spin convergence | ≥2 compactions and ≥75% of the tool-loop budget burned |
SPIN_CONVERGENCE_MIN_COMPACTIONS=2SPIN_CONVERGENCE_TOOL_LOOP_RATIO=0.75
|
So the problem wasn't that nobody built anything. It's that what we built couldn't catch it.
The blind spot: all three are input-side detectors
toolCallSignature = tool name + canonicalized args
That signature answers exactly one question: are you doing the same thing twice?
But a capable model that's stuck doesn't repeat calls. It does this:
read file A → grep X → re-read A with a different line range
→ run a slightly different command → read file B → grep again …
Every single call has a distinct signature. All three tiers stay silent.
And during that entire stretch, the number of verifiable state changes is zero. No file actually rewritten. No command producing a new result. Nothing irreversible happening at all.
One line:
Loop detection ≠ stall detection. The first looks at inputs. The second has to look at outputs.
The paper gives exactly that output-side definition
BEACON's in-segment reward:
r_t = R_ms · γ^(t_k − t) if the segment ends in a milestone
= 0 otherwise
A segment that doesn't terminate in a milestone earns exactly zero — regardless of how many actions it contained. Action count doesn't enter the formula at all.
That's the cleanest formalization of busy ≠ progressing I've seen.
The baseline layer is harsher still. The baseline is the group-average per-step return, so if your segment took 8 steps while the group averaged 5, the entire segment's advantage skews negative, and the penalty scales with how far over you went.
Spinning isn't merely unrewarded. It's actively penalized, proportionally.
Figure 8 in the paper has a failed trajectory where the last two actions receive an identical −2.20. That tail — the stretch after the last milestone that never reaches another one — is the mathematical shape of spinning.
How common is it? Their data: trajectories that complete at least one subgoal but fail the task hold steady at 39–47% of samples.
One ablation that kills the "split by step count" idea
This is the number I found most valuable:
| Partitioning | Score | vs. baseline (72.8) |
|---|---|---|
| Random 5-way split | 74.2 | +1.4 |
| Real milestones | 91.4 | +17.2 |
Translated into product terms: splitting by arbitrary step counts is worth almost nothing. Splitting by real structure is worth a lot.
Now look back at our third-tier criterion — "75% of the tool-loop budget consumed." That's an arbitrary-step-count trigger. It asks how much you've burned, not what you've achieved.
The right shape is:
❌ if steps > N → intervene
✅ if steps > N AND zero verified milestones → intervene
The second one won't misfire on a legitimately long task that's progressing (it hits milestones along the way). The first one will.
There's also a positive feedback loop
Put a few constants side by side:
context at 82% → trigger compaction
2 compactions + 75% of loop budget → only then flag spinning
Spin → context fills up → compaction fires → durable state gets summarized away → re-derive what was lost → spin more.
The spin detector infers spinning by observing that compaction happened repeatedly — but compaction is the very step that causes the amnesia. It's detecting a downstream symptom, and it has to wait for the loop to go around twice.
Worse, its intervention is to nudge the model to re-anchor on its durable state. If the durable state is precisely what got compacted away, there's nothing left to re-anchor on.
This is a prompt-layer patch for a state-layer problem.
What we plan to change
Add a fourth tier: output-side stall detection. Two counters, both derived mechanically from tool observations. No model judgment required.
1. Steps since the last verified milestone
2. Deduplicated new state changes ← the more useful one
- File read with an identical hash → not new information (re-reading the same file at a different line range leaves the hash unchanged)
- Command + exit code + stdout hash identical → not new information
- Write where
afterHashdidn't change → nothing was actually written
This counter targets exactly the case signature matching misses: every action different, information gain zero. The criterion is mechanical — no semantic understanding needed.
Then make the pressure continuous instead of a one-shot nudge:
surface the counter in context so the model can see it
→ force a plan revision (admit this path is dead)
→ ask the user
→ abort, but preserve the milestones already achieved
That last rung matters: if you're going to stop, stop holding what you earned — which echoes the 39–47% number above.
One honest discount
BEACON is a training method. It shapes gradients so the trained policy is less prone to wandering, but it has no runtime detection or intervention mechanism of its own.
It gives you a definition of progress, not a controller. Thresholds, escalation ladders, when to abort — all of that you design yourself. The paper doesn't help there.
Also, its per-step baseline needs a group-average segment length as reference. In production a given user task usually runs exactly once, so there's no group. You can only approximate with historical similar tasks, which is considerably noisier.
But the first thing to do isn't to fix it — it's to measure it
Instrument only. Change no decision logic. Answer a question we currently can't:
Of the spins happening in production, how many are amnesia-type vs. no-gradient-type?
- Mostly accompanied by information gain at zero while call signatures all differ → no-gradient-type. The existing three tiers structurally cannot catch it; we need output-side detection.
- Mostly accompanied by re-reading content that was already compacted away → amnesia-type. What needs fixing is what compaction preserves.
Those two directions call for completely different investments. Measuring first is cheaper than designing first.
Next post: why AI self-reflection always seems to conclude with "be more careful."
If you're building long-running agents — what does yours look like when it gets stuck? Repeating the same action, or looking busy while nothing changes? Curious what others are seeing.
Top comments (1)
The part I like here is treating “busy” as separate from “making progress.” A lot of agent loops look healthy if the only metric is tool calls per minute.For spin detection, I’d probably want an external progress ledger rather than asking the agent to summarize whether it is stuck. Same reason trace summaries get risky. The worker can make a repeated failure look like a sequence of reasonable local steps.The hard bit is defining a milestone that can be checked outside the agent’s own narration. File changed, test became greener, source verified, artifact moved. Something boring and inspectable.