When should an agent stop?

Field note 009. A replication of a published agent-loop benchmark, and what it hides.

Abstract

Every agent loop has to decide when to stop. The usual options are a fixed max_iterations or letting the agent declare itself done. In June I published a small benchmark with CDV, an open-source judge for coding agents, showing that a Bayesian adaptive stop rule used 41% fewer steps than a fixed six-step budget while reaching the quality bar on 99.7% of tasks. This note replicates that result on today's code (it reproduces to the digit), then extends it. Three findings. (1) The adaptive policy ties a plain threshold rule ("stop at the first score ≥ 0.80") everywhere, and it does so by construction: the threshold guard runs first in the guard stack. (2) Both collapse as soon as the judge's score is noisy. At a per-step noise of σ = 0.10, the loop believes it reached the bar in 99.5% of runs and actually reached it in 71%. A fixed budget of six, which never reads the score, holds 93% at every noise level. (3) A one-line fix, requiring two consecutive scores over the bar, restores 97% true reach at σ = 0.10 for 1.4 extra steps. The published headline was true and beside the point: the decision is only as good as the score it trusts.

The problem

An agent loop is a repeat-until: act, evaluate, decide whether to go again. The deciding part is where things go wrong. A fixed max_iterations is either too small for hard tasks or wasteful for easy ones. Letting the agent grade itself means the entity deciding to stop is the entity being judged, and agents optimise reported progress.

CDV's answer was to separate the two. Each step the agent claims is scored by a judge, and a policy decides from the score history whether to continue. The policy is a stack of guards evaluated in order, first stop wins: a score-threshold guard, a plateau guard (last three scores within 0.01 of each other), a Bayesian guard (stop when the learned expected improvement over the remaining steps can't close the gap to the bar), then budget, wall-clock, token and repeated-output guards.¹

The benchmark I shipped with it compared four strategies on a synthetic task set. This note asks two questions the benchmark didn't: what is the adaptive policy actually adding over the simplest rule? and what happens when the judge is wrong?

Prior work, briefly

The published table (June 2026, commit b8a219f in the public repo):²

Strategy	Mean steps	Mean final score	% reaching 0.80
fixed (budget = 2)	2.00	0.698	34.3%
fixed (budget = 6)	6.00	0.939	94.0%
threshold (reactive)	3.56	0.852	100.0%
adaptive (CDV)	3.56	0.852	99.7%

Read the third and fourth rows again. They are the same row. The headline compared row 4 to row 2 and said "41% fewer steps". It did not say that the plain threshold rule was already there, with nothing learned and nothing Bayesian about it. The benchmark script was later removed in a repository cleanup; it's still in the public history, and this note restores it.

Method

The simulation is the original one, unchanged. Each task has a hidden quality curve with diminishing returns, q(t) = 1 − (1 − s₀)·e^(−r(t−1)), with s₀ and r drawn per task from three task types (easy, medium, hard) that converge at different depths. The bar is 0.80; the cap is 8 steps. What the loop sees is q(t) plus Gaussian noise. The original used σ = 0.02.

Replication. I ran the restored script against the current package (cdv at 356d496, Python 3.12.3). It reproduces the table above exactly.³

Extension. Three sweeps, each over 10 seeds × 300 test tasks, reported as mean ± 1 sd:

1. Noise. σ ∈ {0.02, 0.05, 0.10, 0.15, 0.20}. The key metric changes: true reach is whether the hidden quality was ≥ 0.80 at the step the loop stopped, as opposed to observed reach, whether the noisy score was. The original benchmark only measured the second.
2. Cold start. At σ = 0.10, the adaptive policy warmed on 0, 10, 30, 100 or 300 prior tasks before the test split.
3. Cheap fixes. Three rules that need no learning: confirm-2 (stop at the first pair of consecutive scores ≥ 0.80), smooth-2 (stop when the mean of the last two scores ≥ 0.80), and margin (stop at the first score ≥ 0.85).

No LLM is called anywhere. This is a study of the decision policy given a score, not of any model's quality. The scripts, raw results and figures are in the artifact folder linked at the end.

Results

1. Adaptive equals threshold, by construction

Across all five noise levels the adaptive policy and the threshold rule are within one standard deviation of each other on every metric (Figure 1, the two overlapping lines). The reason is in the code, not the statistics: the guard stack evaluates ScoreThresholdGuard first.¹ The adaptive policy can therefore stop earlier than the threshold rule (via plateau or Bayesian guards) but never later, and in this simulation it almost never fires earlier once the priors are warm. Whatever the Bayesian machinery is worth, this benchmark cannot show it.

2. Trusting one score collapses under noise

Table 1 is the noise sweep. The observed reach for the threshold rule is ~99.5% at every σ: the loop always saw a score over the bar before it stopped. True reach falls from 93.4% at σ = 0.02 to 54.8% at σ = 0.20.

σ	fixed 6, true	threshold, steps	threshold, observed	threshold, true	adaptive, true
0.02	92.8 ± 1.1	3.66	99.9	93.4 ± 1.4	92.8 ± 1.6
0.05	92.8 ± 1.1	3.66	99.7	84.6 ± 2.0	84.1 ± 1.8
0.10	92.8 ± 1.1	3.54	99.5	71.2 ± 2.7	70.9 ± 2.5
0.15	92.8 ± 1.1	3.38	99.5	61.4 ± 3.3	61.0 ± 3.2
0.20	92.8 ± 1.1	3.23	99.6	54.8 ± 3.6	54.5 ± 3.5

Table 1. Percent of tasks reaching the bar, mean ± sd over 10 seeds. "Observed" is what the loop believed; "true" is what was the case.

Notice the step counts fall as noise rises. The loop isn't just wrong more often; it's wrong faster. A noisy judge produces a lucky score sooner, and the rule takes the first one it sees.

The fixed six-step budget, the strategy the original headline was beating, holds 92.8% true reach at every noise level, because it never reads the score. Above σ ≈ 0.03, the dumb budget delivers more real quality than either learned rule. It costs 2.3 more steps per task to do it.

3. The Bayesian guard is harmful before it is warm

At σ = 0.10 with no warm-up history, the adaptive policy stops after 2.79 steps on average and reaches the bar truly on 50% of tasks, worse than the threshold rule (71%) and far worse than the fixed budget (93%). It needs about 30 prior tasks to catch up (Figure 3). The should_continue rule falls back to a plain threshold check for the first three observations, then starts trusting expected-improvement estimates that are still wide.⁴ In a real deployment that is the first sprint on every new task type.

4. A one-line fix

Confirm-2, stopping only when two consecutive scores clear the bar, is the best rule tested at every noise level (Table 2). At σ = 0.10 it reaches 97.2% true reach in 4.91 steps: more real quality than the fixed budget, for one step less. It costs 1.4 steps more than the threshold rule and buys 26 points of true reach.

σ	threshold	confirm-2	smooth-2	margin +0.05	fixed 6
0.02	93.4 (3.66)	100.0 (4.65)	100.0 (4.28)	99.9 (4.19)	92.8 (6)
0.05	84.6 (3.66)	99.6 (4.73)	97.8 (4.27)	96.5 (4.19)	92.8 (6)
0.10	71.2 (3.54)	97.2 (4.91)	92.8 (4.25)	85.1 (4.03)	92.8 (6)
0.15	61.4 (3.38)	95.4 (5.06)	88.7 (4.28)	73.9 (3.81)	92.8 (6)
0.20	54.8 (3.23)	93.7 (5.16)	85.3 (4.28)	64.7 (3.59)	92.8 (6)

Table 2. True reach % (mean steps). Margin, the intuitive fix, is the weakest: a higher bar is still one noisy sample.

Findings

1. The published comparison was against the wrong baseline. Adaptive beat a fixed budget; it tied the simplest reactive rule, and the guard order guarantees it can't do otherwise here.
2. The decision is only as good as the score. A loop that stops on one crossing of the bar is fooled at a rate set by the judge's noise, and it is fooled faster as the noise grows. Observed success stays flat while real success falls. Any harness that reports "reached the bar" from a single judge score is reporting the judge's variance.
3. Learning has a cold-start cost that a fixed rule doesn't. Before ~30 observations per task type, the Bayesian guard is worse than not having it.
4. Redundancy beats sophistication. Asking the judge twice (confirm-2) does more for real quality than either a learned prior or a higher bar. That is the same argument CDV makes at the scoring layer with min(A, B), the stricter of two judges wins, applied at the stopping layer.

Limitations

• This is a simulation of the decision policy, not of models. The quality curve is a hand-made exponential; real agent progress isn't smooth, isn't monotone, and can regress. Noise is Gaussian and independent per step; a real judge's errors are correlated with the task, which is worse for confirm-2 than this study shows.
• Noise levels are assumed, not measured. I don't know what σ a real LLM critic has on real coding steps. If it is under 0.03, the threshold rule is fine and this note is moot. If it is over 0.05, the fixed budget wins on quality. Measuring that number is the next study.
• The adaptive policy was tested as shipped, with default guards. Reordering the stack (Bayesian before threshold) or tuning the plateau delta could change result 1. I did not tune anything; that would be a different claim.
• Same author, same code. I wrote the benchmark and the policy it flatters. The replication and the extension are both mine. Treat the artifact as the check, not my reading of it.
• The cold-start result depends on total_calls < 3 fallback logic in the priors and on the warm-up being drawn from the same distribution as the test set. A mismatched warm-up would be worse.

What would change my mind

A live run, with a real judge scoring real agent steps, where the single-crossing rule's true reach (measured by a held-out oracle: the test suite, a human, a stricter judge) tracks its observed reach within a few points. That would mean judge noise is low enough that none of this matters. I'd also update if a reordered guard stack let the Bayesian guard beat confirm-2 on the noise sweep at equal step cost.

Artifact

• Original benchmark, restored from public history: git show b8a219f:benchmarks/adaptive_vs_fixed.py in azank1/cdv.
• Extension scripts, raw results (JSON), and the figure code: github.com/azank1/code-desk-cli → research/009-stop-rule.
• Environment: cdv at 356d496, Python 3.12.3, no network, no model calls. Runtime under two minutes.

Notes

1. src/cdv/guards.py, default_guard_stack(): ScoreThresholdGuard, PlateauGuard, BayesianGuard, then budget, max-steps, timeout, token and output-repeat guards; GuardStack.evaluate returns the first stop reason. CONVERGENCE_DELTA = 0.01.
2. benchmarks/results/adaptive_vs_fixed.md at commit b8a219f (2026-06-25); the same table is in the README at README.md §Evidence.
3. Output of the restored script on 2026-09-25: identical rows, 41% step saving, 99.7% goal-reach.
4. src/cdv/priors.py, AdaptivePriors.should_continue: if prior.total_calls < 3: return current_score < quality_threshold, then expected-improvement summation over the remaining steps.