DEV Community

Cover image for Claude Code said "Done." My tests said otherwise. Here's the 20-line fix.
elijahmanlockedin112
elijahmanlockedin112

Posted on AI-assisted

Claude Code said "Done." My tests said otherwise. Here's the 20-line fix.

If you use Claude Code, you've seen this:

claude: Done! Partial refunds are implemented. ✅
Enter fullscreen mode Exit fullscreen mode

Then you run the tests yourself and four of them fail.

Claude isn't lying. It stops when the work looks done, because it has no other signal. If nothing tells it the tests are red, "looks done" is all it has. So you become the test runner: you check, paste the errors back, wait, and check again.

This post shows how to make Claude run that check itself, every time, before it's allowed to say it's finished.

Why this is the highest-leverage fix

Anthropic's best practices for Claude Code put this first: give Claude a way to verify its work, such as tests, a build, a linter or a script, anything that returns pass or fail. Boris Cherny, who created Claude Code, has said that this feedback loop improves the quality of the result 2–3×.

There are three places you can put that check:

Where How reliable
In your prompt ("run the tests after") Works when you remember to type it
In CLAUDE.md Advice. Claude usually follows it, sometimes not
In a hook Runs every time. Claude can't skip it

Hooks are scripts Claude Code runs at fixed points in its loop. The one we want is the Stop hook, which runs whenever Claude tries to end its turn. If a Stop hook exits with code 2, Claude Code doesn't let the turn end. It sends whatever the hook printed to stderr back to Claude, and Claude keeps working.

That's the whole trick.

The 20-line version

Save this as .claude/hooks/verify-gate.py in your project:

#!/usr/bin/env python3
import json, os, subprocess, sys

data = json.load(sys.stdin)
project = os.environ.get("CLAUDE_PROJECT_DIR", os.getcwd())
config = os.path.join(project, ".claude", "verify.txt")
if not os.path.exists(config):
    sys.exit(0)  # no check configured: let Claude stop

if data.get("consecutive_blocks", 0) >= 4:
    sys.exit(0)  # blocked 4 times in a row: let Claude stop instead of looping

command = open(config).read().strip()
result = subprocess.run(command, shell=True, cwd=project,
                        capture_output=True, text=True, timeout=300)

if result.returncode != 0:
    tail = (result.stdout + result.stderr).strip().splitlines()[-40:]
    print("Verification failed:\n" + "\n".join(tail) +
          "\nFix the failures above, then finish. Do not weaken or skip tests.",
          file=sys.stderr)
    sys.exit(2)  # exit 2 = block the stop; stderr is sent back to Claude
Enter fullscreen mode Exit fullscreen mode

Register it in .claude/settings.json:

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "python3 \"${CLAUDE_PROJECT_DIR}/.claude/hooks/verify-gate.py\"",
            "timeout": 330
          }
        ]
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

(On Windows, python instead of python3.)

Then put your check in .claude/verify.txt:

npm test && npx tsc --noEmit
Enter fullscreen mode Exit fullscreen mode

Restart Claude Code. The next time it says "Done" with red tests, this is what happens:

claude: Done. Partial refunds are implemented.
Stop hook: Verification failed:
  ✗ refund.test.ts › partial refund rounds to cents
  Expected 12.35, received 12.349999
  Fix the failures above, then finish. Do not weaken or skip tests.
claude: The rounding is off. Fixing src/orders/refund.ts:42…
claude: All 48 tests pass and types are clean. Done.
Enter fullscreen mode Exit fullscreen mode

You didn't paste anything. Claude found out it wasn't done, and fixed it.

Writing a good check

The hook is only as good as the command in verify.txt. A good one:

  • Exits non-zero on failure. Every normal test runner already does.
  • Runs without prompts. No watch mode, no "press q to quit". vitest needs vitest run, and jest needs --watchAll=false in some setups.
  • Is fast enough to run on every stop. If your full suite takes 10 minutes, point it at the part you're working on: pytest tests/billing -q.
  • Actually covers the feature. Typecheck plus the relevant tests beats either alone.

Examples:

npm test && npx tsc --noEmit
pytest -q && ruff check .
go test ./... && go vet ./...
cargo test && cargo clippy -- -D warnings
Enter fullscreen mode Exit fullscreen mode

Windows gotcha: the command runs through cmd.exe, which doesn't treat single quotes as quotes. python -c 'exit(1)' silently passes there, because Python evaluates a string literal and exits 0. Use double quotes inside verify.txt. I found this out when my own README example "passed" a test that should have failed.

Where the 20-line version falls short

It works, but a real session exposes some edge cases:

  1. It runs even when nothing changed. Ask Claude a question, and your whole suite runs before it can answer. Better: skip when git status is clean, or when the tree hasn't changed since the last passing run.
  2. A hanging command wedges the turn. timeout=300 raises an exception but can leave child processes running (a dev server, a test watcher). Better: kill the whole process tree, and report the timeout as a failure with advice.
  3. consecutive_blocks may not always be there. Keep your own per-session counter as a backup.
  4. Crashes should never break a session. Any unexpected error should exit 0 and let Claude stop.

I handled all four in a stdlib-only version that works on Windows, macOS and Linux. It's free and MIT licensed: verify-gate-hook on GitHub. Setup is the same three steps.

Two habits that make it work better

Write the check before the code. If you write verify.txt first and run it once, it should fail, because the feature doesn't exist yet. If it passes, it isn't testing the feature. Tighten it before you start.

Watch for weakened tests. The hook's message tells Claude not to weaken or skip tests, and it usually listens. Still skim the diff for edited assertions. A gate is only as honest as the test it runs.

Going further

The gate answers "is it done?" The harder part is deciding what "done" means before Claude starts. I ended up building a full Claude Code setup around this loop:

  • /lock-in has Claude ask you the hard questions, then write a spec with a single verification command. It writes that command to verify.txt, checks that it fails first, builds step by step against it, and finally has a fresh subagent review the diff against the spec.
  • Hooks block rm -rf ~, force-pushes to main, and reading .env before those commands ever run.
  • /handoff → /clear → /pickup saves your progress to a file and resumes from it in a fresh session, so your context stays small and your usage limit lasts longer.

That's the Locked In Kit: 18 skills, 6 subagents, 5 hooks, 8 CLAUDE.md templates and a playbook. The first 20 people get 30% off with code LAUNCH.

The hook above is free and works fine on its own, though. Add it to one project today and see how often Claude "finishes" with failing tests.


Independent project, not affiliated with Anthropic. Hook behavior is from the Claude Code hooks reference. If something behaves differently in your version, trust the docs.

Top comments (3)

Collapse
 
skillselion profile image
Skillselion •

Worth checking that loop guard against the hooks reference. The Stop input fields it documents are stop_hook_active, last_assistant_message, background_tasks and session_crons; I can't find consecutive_blocks there. If the key is absent, data.get("consecutive_blocks", 0) is always 0 and the guard never trips. The same section says Claude Code already caps stop hooks at eight consecutive continuations, adjustable with CLAUDE_CODE_STOP_HOOK_BLOCK_CAP. So an unfixable failure (a flaky network test, a missing env var) still ends, but after eight runs of your suite instead of four. That makes the counter from your point 3 the part that actually enforces your limit of four.

Collapse
 
hamid_ahmadian_3570449f72 profile image
Hamid Ahmadian •

This is a great pattern, and the exit-code-2 mechanic is the right primitive for it. One gap I'd flag though: the Stop hook and verify.txt both live inside the same working tree the agent is editing, so a sufficiently long/confused session can (and in my testing occasionally does) "fix" the failing test by editing verify.txt or loosening an assertion instead of the actual bug — especially once consecutive_blocks starts climbing and the model gets desperate to end the turn. The fix we landed on is putting the verification command and its expected-pass baseline in a path the agent's own permission tier can read but not write (a separate config layer gated by the harness, not just convention), so "editing the test to make it green" and "editing the code to make it green" aren't the same action anymore. Worth considering as a hardening step on top of your consecutive_blocks escape hatch, since that counter itself becomes a race the model can learn to hit.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.