We run ~100 LLM agents unattended on local models. Last week we found one
document that had been rewritten 245 times in 5 days — every attempt
rejected. A sibling document: 225 times. Combined, about 470 wasted
generations, all burned on the same two files.
Here is the autopsy, with the actual numbers.
The loop
Our pipeline is simple: a producer agent writes a document, a reviewer agent
checks it against a contract (minimum length, required sections, no
placeholder junk), and rejected work goes back with fix instructions.
The rejected document was a key-management (KMS) implementation spec —
4,452 characters, perfectly on-topic. The reviewer's verdict:
"The request was a 3-line email triage response (LOCK / VERDICT / REASON),
but the answer is a long KMS spec. Rewrite as 3 lines only."
One problem. We grepped the document: the words "LOCK", "VERDICT", and the
name of the triage service appear zero times in it. The reviewer had
invented the request.
Why the loop never ended
Two contracts collided:
- The reviewer's fix instruction: output 3 lines only
- The producer's output contract: minimum 600 characters
No output can satisfy both. So the producer failed the contract, got
re-queued, produced again, failed again — 245 times. Our retry cap counted
reviews, but a contract-failed output never reaches review. The give-up
mechanism existed; it just watched the wrong counter.
Root cause: the reviewer never saw the request
Our review prompt contained the artifact body (first 4,000 chars) and the
output format. It never contained the original request. We asked a model
"does this match the request?" without telling it what the request was.
A model asked to judge against information it doesn't have will
hallucinate that information. Ours did, confidently, 245 times' worth.
Get the checklist and the three watchdogs (free) → gxcafe.co.jp/harness-kit
Bonus failure: we truncated long documents to 4,000 characters before
review without saying so, and reviewers marked them "thin — cut off
mid-sentence." The cut was ours, not the producer's.
How common was it?
We audited all 2,038 reviews on file for concrete terms (product names,
format tokens) that appear in the review but nowhere in the reviewed
document. Result: 4 contaminated reviews — 0.2%.
That's the uncomfortable lesson: a 0.2% hallucination rate produced 470
wasted runs, because nothing ever gave up. Low rate × infinite retries =
unbounded damage. The rate is not the risk; the loop is.
The fixes (all mechanical)
- Pass the original request into the review prompt. If it can't be extracted, the prompt now says: "do NOT guess the request — say it is unknown and judge the artifact on its own."
- Declare truncation. "First 4,000 of 8,784 chars — the cut is ours."
- Reject impossible instructions at the review's own exit gate. A review demanding "N lines only" while the production contract requires 600+ chars now fails as a review and never enters the queue.
- Count consecutive contract failures, not just reviews, and park the item for a human after 5 — with the last verdict and fix instruction attached, so the human can see why in one glance.
Each fix ships with a test we deliberately broke to confirm it fails.
If you run agents unattended
The checker that catches broken outputs in this story (empty text, language
leakage, placeholder junk, contract violations) is free on npm:
honto-contract — it passed
600 downloads last week, so somebody besides us finds this useful now.
The unattended-operation checklist and three of our watchdog templates are
free (email-gated):
Get the checklist and the three watchdogs (free)
More of these, as we find them: GX Cafe engineering notes — 12 posts, no signup.
The full production set (7 templates — cron registry, silent-zero watch, heartbeat, output contracts: the exact ones in this story) is available on the same page.
(https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=harness-kit&utm_content=20260824-kiji-c9185d).
Honest note: we have no customers yet. Everything above is exactly what we
run on ourselves, measured on our own failures.
Top comments (10)
The bad counter is the sharpest lesson here. You had a retry limit, but it sat after the branch that was looping. I’d budget attempts at the work-item level, before either producer or reviewer runs, and persist the last contract pair with each failure. Then an impossible pair stops once and remains inspectable instead of being rediscovered 245 times.
You're right that the counter's position was the real bug — we had a limit, just downstream of the loop. Two things we've since done, and one thing your comment made us add today:
Already in place: the budget is now checked at re-queue time (before the producer runs), and a contract-conflict check drops an impossible pair on the first failure when the reviewer's instruction demands a short fixed format ("output 3 lines only") while the producer's contract requires 600+ chars — pattern-matched, so it only catches the shapes we've seen.
Added today from your comment: the give-up ledger now persists the pair itself — the reviewer's instruction (300 chars) plus the exact contract violations — overwritten on every failure. You were right that a count alone isn't inspectable: we could see that something failed 5 times, but not that the pair was impossible, so the same dead end kept being rediscovered by humans reading logs. Now the ledger entry answers that in one read.
Moving the budget check to re-queue time is the right boundary. The ledger change is the bigger win for diagnosis: a human can now see the impossible pair without reconstructing it from five attempts. I’d be careful with the pattern matcher, though; as the contract shapes grow, it can quietly become a second contract system. Keeping the raw pair is what will make missed patterns visible.
Agreed — and that risk is real enough that we've frozen the matcher's role. The regex only covers shapes we've already been burned by; it's a fast-path that saves one wasted generation, nothing more. The general mechanism stays shape-agnostic: the per-item budget halts any impossible pair within N attempts, and the raw pair (reviewer instruction + exact contract violations) lands in the ledger regardless of whether any pattern matched.
The promotion rule we've set: a shape only graduates into the matcher after it has appeared in the ledger — so the matcher can lag reality, but it can't diverge from it, and the ledger is always the superset. If we ever find ourselves adding a matcher rule for something the ledger hasn't shown, that's the "second contract system" smell and the answer is no.
That ledger-as-superset rule is a solid boundary; it keeps regex from quietly turning into policy. I would reconsider overwriting the pair on every failure, though. In our file-based runs we keep a tiny append-only event—pair digest, attempt number, exact violation, timestamp—then maintain a latest projection for the quick read. That preserves the diagnosis without losing the sequence when the reviewer instruction or contract changes between attempts.
Implemented as you described: an append-only event stream (pair digest, attempt number, exact violations, timestamp) with the existing JSON kept as the latest-projection for quick reads. You called the failure mode precisely — our overwrite was erasing exactly the interesting case, where the instruction or contract changes between attempts. The digest makes those transitions visible as a digest change in the sequence.
The 0.2% is a floor set by what the detector can see. Grepping for concrete terms that appear in a review but nowhere in the reviewed document only catches an invented request that brings new vocabulary with it, so a review that fabricates a requirement out of the document's own headings and product names scores clean, and the audit cannot tell rare apart from invisible. Now that fix 1 puts the original request in the prompt, that same audit has a known object to compare against: does the review reference a requirement absent from the request. That version has a defined negative case, which the novelty heuristic never had.
This is the sharpest critique of the piece — "the audit cannot tell rare apart from invisible" is exactly right, and we hadn't seen it. The novelty grep has no defined negative case: a review that fabricates requirements out of the document's own headings scores clean, so 0.2% was a floor set by the detector, not a rate.
Implemented today from your comment: since fix 1 gives the review a known object (the original request travels with the artifact chain), the audit now checks whether the review imposes a format requirement that appears in neither the request nor the reviewed document — "output N lines only" and friends. Defined negative case, as you said. First run immediately caught one surviving specimen from before the fix (an Aug 18 review demanding "3 lines only" against a 600-char-minimum contract).
One honest failure from the same session: we also tried flagging title-vs-declared-scope divergence with a zero-content-word-overlap rule. 412 hits in 7 days — redo chains legitimately narrow their scope, so the rule can't separate drift from refinement. We demoted it to an opt-in display; 412 alarms are worse than 0 detections, because the one real case drowns.
The 412 makes sense if the rule is symmetric: content-word overlap carries no direction, so a redo that narrows and a redo that drifts look the same to it. Direction is the part you want, since a legitimate narrowing stays inside its parent's referents while drift introduces one that appears nowhere in the ancestor chain, and that is the same known-object comparison that made the format check work. The cheap version is to flag only when a child's scope adds a token absent from the union of the chain above it, which lets refinement delete as much as it likes without tripping anything.
Implemented the directional version today: a child's declared scope only flags when it adds tokens absent from the union of its ancestor chain (we walk up to 6 links). Same 7-day corpus: 412 hits → 0 false positives. Promoted from opt-in back to a default check. Deletion-only narrowing is now structurally silent, as you predicted.
One finding worth reporting back: we then checked the detector against our one known real specimen (a consent-doc chain whose later redos declared a DM-strategy scope) — and the directional check correctly does not flag it. Root-cause: that specimen wasn't declaration drift at all. The chain's root declaration was correct; foreign vocabulary entered through a body contamination (an answer to a different task, inherited by descendants), so by the time any child declared the wrong scope, its ancestors already contained the vocabulary. Which is your "rare vs invisible" point applied one level deeper: each detector has a class, and validating it against a specimen told us the specimen belonged to a different class (cache contamination — caught at generation time by foreign-marker checks, not by scope comparison). The directional check refusing to fire there is the design working, not a miss.