My coding agent ends almost every task the same way. "Done! All tests pass." For months I just believed it, and for months I kept finding the lie a day later. The test that never ran. The env var that never got set. The endpoint answering 500.
Turns out this is measured. A paper from June (arXiv 2606.09863) looked at agent runs that graded themselves and found that among the failing ones, 75.8% still claimed success. The same paper tried using LLM judges to catch it. Best AUROC across 5 judges and 5 prompt strategies was 0.65, worst was 0.54. A coin flip, because judges read the confident closing tone instead of the actual state of the machine.
A test runner detects a failing suite at 1.0. So I built a test runner wearing a Stop hook.
The receipt
nuhuh treats the agent's final message as a list of hypotheses. It extracts every claim ("all tests pass", "created src/x.ts", "endpoint works", "set DATABASE_URL") and re-runs reality, fresh. Whole suite in a clean process reading real exit codes, files checked on disk, localhost actually called. Then it prints a receipt it wrote, not one the agent dictated.
🧾 receipt
✅ src/login.ts exists (33 bytes)
❌ src/login.test.ts does not exist
❌ "All tests pass." ran npm test fresh, exit 1 ("Tests: 1 failed, 3 passed")
✅ "The build succeeds" exit 0
2 of 4 claims verified, 2 failed.
In gate mode a false "Done" gets rejected and the failing evidence goes straight back to the agent, which returns to work. After 3 bounces it hands you the receipt instead of arguing.
Two things that happened during testing
First, I rigged a project with a test that passes once and fails on every rerun, then asked a headless agent to report whether tests pass. It ran them once, truthfully said "tests pass, exit 0", and the gate bounced it with the fresh failure. What happened next was the best part. The agent investigated, found the trap, and refused to tamper with it, quoting the gate's own instruction that weakening checks is forbidden. It revised its claim instead. The whole loop worked on the first real try.
Second, an agent I had asked to lie refused, and quoted "All tests pass" inside its refusal. An early version of nuhuh extracted that quote as a claim and would have blocked the honest refusal. That became the first entry in a false accusation regression suite that now has six real cases in it. The tool is tuned to miss rather than accuse, because one wrongful block costs more trust than ten misses.
The numbers
The repo ships a False Done Rate benchmark. Deterministic ground truth scripts that know nothing about nuhuh, so it can expose nuhuh's own blind spots too. Three rounds of 18 tasks per harness, 54 runs each.
| model | declared done | false dones | FDR |
|---|---|---|---|
| frontier Claude | 54 | 0 | 0.0% |
| Codex | 49 | 2 | 4.1% |
| Haiku 4.5 | 49 | 3 | 6.1% |
One more lesson the data forced on me. Single runs lie. Codex measured 0% on round one and I nearly published that number. Over three rounds it settled at 4.1%. Haiku went the other way, 12.5% on round one down to 6.1% over three.
The frontier model was more honest than the discourse suggests, at least at this task size. The smaller models produced the interesting failures. One wrote a lint script whose failures BSD find silently swallows on macOS, then truthfully reported that its broken check passes. A true claim about a defective check. Another declared victory on a config chore with a hardcoded port still in place, and the Done message carried no checkable claim at all. Claim verification can't catch either class, only independent ground truth can, which is exactly why the benchmark exists.
Design rules that survived contact
- Zero LLM calls in the verification path. Deterministic, local only, no API key
- unverifiable is never failed, and timeouts are never failures
- Only commands from the project's own manifests ever run, never text from the agent
- Claim patterns are a data file (English and Korean today), so a new language is a PR, not a fork
npx nuhuh demo shows it catching a staged lie in 10 seconds without touching anything. GitHub at https://github.com/sjh9714/nuhuh
Top comments (1)
The receipt idea is the right shape. I would make the gate print the command, exit code, and touched files every time, even on success, because otherwise the agent learns that a green sentence is enough. The three-bounce handoff is a nice boundary too.