Your agent said the tests passed. Check whether they ran.

There is a failure mode in agent-assisted coding that does not look like a failure. You ask for a fix, the agent works for a while, and the summary says the tests pass. They did not fail. They also did not run.

A failing test is information. A test that never ran, reported as a pass, is worse than having no tests at all, because now you believe something.

This is not about any one tool being bad. It is a reporting gap, and it shows up the same way in Claude Code, Cursor, Copilot Workspace or a shell script somebody wrote in 2014. Here is what causes it and the smallest thing you can do about it.

The three ways a test run turns into a lie

1. The command was never found.

The agent does not know how you run tests. It guesses npm test, your project uses pnpm vitest run, the guess errors out, and the error gets folded into the summary as "no test failures". Technically true. Completely useless.

2. A flag made nothing count as something.

npm test -- --passWithNoTests

That flag does exactly what it says. Zero tests collected, exit code 0, green checkmark. Jest and Vitest both have it, and it exists for good reasons in a monorepo, which is precisely why it turns up in commands where it does not belong.

3. Somebody appended a shrug.

pytest || true

Now the command cannot fail. Whatever pytest thinks, the shell reports success. You see this a lot in CI files that somebody was trying to unblock at 6pm on a Friday, and once it is there nobody removes it.

How to tell, in one question

When an agent tells you the tests pass, ask for three things. It takes one line and it is the difference between a verified change and a claimed one.

Ask for	Good answer	Bad answer
The command it ran	The exact string, matching your repo	A paraphrase, or nothing
The output	A count: "42 passed, 0 failed"	"Tests pass"
The exit code	0, and it says so	Not mentioned

If you get a count and an exit code, something ran. If you get the word "pass" and nothing else, treat it as unverified. That is not cynicism, it is the same standard you would apply to a pull request from a contractor you have not met.

Half the fix is writing the command down

If you use Claude Code, the file it reads at the start of every session is CLAUDE.md at the root of your repo. Put the real commands in it, verbatim, flags included. It is copied, not interpreted, so a paraphrase is worse than nothing.

## Commands
- Test: `pnpm vitest run --silent`
- Lint: `pnpm eslint .`

## Never
- Never add `--passWithNoTests` or `|| true` to a command that is supposed to prove a change works

Then let it actually run them without asking, in .claude/settings.json:

{
  "permissions": {
    "allow": ["Bash(pnpm vitest:*)", "Bash(pnpm eslint:*)"]
  }
}

Miss that second half and you have traded a "cannot find it" prompt for a "may I run it" prompt, every single time, forever. Most people quietly stop reading those prompts after a day, which is its own problem.

The other half is a hook, because writing it down is not enforcement

An instruction in a markdown file is a request. Some days it gets followed. If you want the masked test run to be impossible rather than discouraged, Claude Code will run a script before it executes a Bash command, and if that script exits with code 2, the command is blocked and the reason goes back to the agent so it can correct itself.

Here is the whole thing. Save it as .claude/hooks/no-skip-tests.sh:

#!/usr/bin/env bash
# Blocks the flags that make a test run report success whatever happens.
set -uo pipefail
payload="$(cat)"

if command -v jq >/dev/null 2>&1; then
  cmd="$(printf '%s' "$payload" | jq -r '.tool_input.command // empty')"
else
  cmd="$payload"   # no jq: match against the raw payload rather than failing open
fi
[ -z "$cmd" ] && exit 0

case "$cmd" in
  *--passWithNoTests*|*"|| true"*|*--exitcode-zero*)
    echo "BLOCKED: that flag makes the test run report success even when nothing ran." >&2
    echo "Run the suite for real, or say out loud that you are skipping it." >&2
    exit 2 ;;
esac
exit 0

Wire it up in .claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "bash \"$CLAUDE_PROJECT_DIR/.claude/hooks/no-skip-tests.sh\"" }
        ]
      }
    ]
  }
}

Two details that matter more than they look:

Exit 2, not exit 1. Exit 2 is the code that blocks the tool call and hands your message back to the agent. Other non-zero codes get treated as the hook itself being broken, which is not what you want.

Handle a missing jq. If your hook parses the payload with jq and jq is not installed, the naive version silently allows everything through. A security or safety hook that fails open is worse than no hook, because you think you are covered. The fallback above matches against the raw payload instead, which is cruder and still blocks the thing.

Test the hook, because an untested hook is a comfort blanket

You can check it without launching an agent at all. The hook reads JSON on stdin, so hand it some:

echo '{"tool_input":{"command":"npm test -- --passWithNoTests"}}' | bash .claude/hooks/no-skip-tests.sh; echo "exit $?"
# exit 2

echo '{"tool_input":{"command":"npm test"}}' | bash .claude/hooks/no-skip-tests.sh; echo "exit $?"
# exit 0

Run both. A hook that blocks everything passes the first check and fails you on a Tuesday, so the allow case matters as much as the block case. That is also the difference between a hook you keep and one you delete in a fortnight when it gets in the way.

What this actually buys you

Not much, individually. It stops one specific lie. The reason it is worth ten minutes is that it moves a rule out of your head and into the repo, where it applies to every session, every contributor and every agent that touches the project, without anybody remembering to be careful.

That is the whole pattern: write the commands down so they cannot be guessed, allowlist them so they actually run, and hook the two or three things you never want to happen so being tired is not enough to cause them.

---

If you want the starting point rather than building it from scratch, our setup is public and free: kit-claude-code-starter is a real CLAUDE.md, an allowlist and a doctor script, and kit-plugins is four small Claude Code plugins that each install on their own. Nothing is gated behind an email.

There is a longer write-up of this specific problem here: Claude Code cannot find my test command.

And if you would rather not do the reading, we set it up for one repo for $29 and hand it back as a pull request you review: kit.sdvsignal.com. Take the free starter first though. We would rather you had that than bought something you did not need.