Your AI agent says "done". Your tests say 3 failing. The Stop hook that settles it

You ask an agent to fix a bug. It edits four files, writes a cheerful summary, and says the task is complete. You run the tests and three of them fail. Sometimes it never ran them at all. Sometimes it ran them, saw red, and wrote "done" anyway.

If you're running several agents in parallel, it's worse: each one reports success on its own branch, and the feature still doesn't work when you put the branches together.

This isn't a model being lazy. An agent's "done" is a self-report, and the agent is grading its own homework. Adding "ALWAYS RUN THE TESTS" to your rules file in capital letters doesn't change that. A check that runs whether the agent wants it to or not does.

This post covers three things:

1. Where agent rules actually go in Cursor, Claude Code and Codex (and one gotcha that silently unloads them).
2. What to commit and what to keep local.
3. How to move the rules that must never break out of prose and into a Stop hook and CI.

1. Where the rules go

Each tool reads a different set of files. Here's the current picture, checked against each tool's docs this week:

Tool	Shared project rules	Scoped rules	Personal / local
Cursor	AGENTS.md (root and nested), .cursor/rules/*.mdc	.mdc with globs:	User Rules (Settings)
Claude Code	CLAUDE.md or .claude/CLAUDE.md (can also read AGENTS.md)	.claude/rules/*.md with paths:	CLAUDE.local.md, ~/.claude/CLAUDE.md
Codex	AGENTS.md at each level from the repo root down	Nested AGENTS.md / AGENTS.override.md	~/.codex/AGENTS.md

A few details that matter:

• Cursor only picks up .mdc files in .cursor/rules/. A plain .md file there is ignored. If you want plain markdown, use AGENTS.md.
• Codex stops reading at 32 KiB of combined instructions by default (project_doc_max_bytes). A sprawling rules file can quietly lose its tail.
• Claude Code's docs suggest keeping each CLAUDE.md under about 200 lines, and moving anything that only applies to part of the codebase into path-scoped rules.

The setup I use: one AGENTS.md as the source of truth, plus a tiny CLAUDE.md that imports it:

@AGENTS.md

## Claude Code only
- Use plan mode before touching migrations or billing code.

Claude Code expands @path imports at launch, so the AGENTS.md content ends up in context and you only maintain the rules in one place. Codex and Cursor read AGENTS.md directly.

For scoped rules, Cursor and Claude Code use different frontmatter keys for the same idea:

---
description: API route conventions
globs: src/app/api/**/*.ts
alwaysApply: false
---
- Validate every request body with zod before touching the database.

---
paths:
  - "src/app/api/**/*.ts"
---
- Validate every request body with zod before touching the database.

The first is .cursor/rules/api.mdc; the second is .claude/rules/api.md. Scoped rules only load when the agent works on matching files, so your backend rules stop eating context while you edit CSS.

2. Commit the team's rules, keep your habits local

This came up a lot this week: should AGENTS.md be committed? Yes. A simple test for each line in it:

Would a reviewer reject a PR that breaks this rule? If so, commit it. If it's just how you like to work, keep it local.

Commit: AGENTS.md, CLAUDE.md, .cursor/rules/, .claude/rules/, .claude/settings.json (where Claude Code hooks live), .cursor/hooks.json and the hook scripts themselves.

Keep local: "explain your plan before editing", "I prefer pnpm", your sandbox URLs, test accounts, tone preferences. These go in CLAUDE.local.md, .claude/settings.local.json, ~/.claude/CLAUDE.md, ~/.codex/AGENTS.md or Cursor's User Rules.

# .gitignore
CLAUDE.local.md
.claude/settings.local.json

The gotcha: CLAUDE.local.md can switch off your AGENTS.md

By default, Claude Code reads AGENTS.md only when there's no CLAUDE.md in the project. And a CLAUDE.local.md counts as one.

So if your team keeps everything in AGENTS.md with no CLAUDE.md, and you add a personal CLAUDE.local.md, Claude Code reads your local file instead of the team rules. Nobody else sees the problem, because their checkout doesn't have your local file.

Two fixes:

• Commit a CLAUDE.md that contains @AGENTS.md (the setup above). The import always loads.
• Or set Project instructions to claude-md-and-agents-md in /config.

Also note that Claude Code doesn't read AGENTS.override.md or AGENTS.local.md. Those are Codex conventions.

3. Prose is a request. Hooks and CI are a gate.

Here's the rule I'd tape to the monitor:

If breaking a rule causes a bug or an incident, it isn't a rule. It's a check.

"Prefer small functions" can stay as prose. "Tests must pass before you say done" can't, because the agent only follows it when it remembers to and agrees with itself that it applies.

You get two layers:

1. A Stop hook catches it in the session, while the agent can still fix things.
2. CI with a required status check catches it at merge, whatever agent or human wrote the code.

Claude Code: a Stop hook that won't accept "done" on red

Claude Code fires a Stop event when the agent finishes responding. If a Stop hook exits with code 2, Claude Code blocks the stop and sends the hook's stderr back to the agent as the reason. In other words, the agent gets your failing test output and has to keep working.

.claude/settings.json (committed):

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "node \"$CLAUDE_PROJECT_DIR\"/.claude/hooks/require-green.mjs",
            "timeout": 300
          }
        ]
      }
    ]
  }
}

Stop doesn't take a matcher, so there isn't one. The default command timeout is 600 seconds; set your own if your suite is slow.

.claude/hooks/require-green.mjs:

#!/usr/bin/env node
// Blocks Claude Code from finishing while the test suite is red.
import { execSync, spawnSync } from "node:child_process";
import { readFileSync } from "node:fs";

const input = JSON.parse(readFileSync(0, "utf8")); // hook JSON arrives on stdin

// Pure Q&A turn with no changes in the working tree? Let it stop.
const changed = execSync("git status --porcelain", { encoding: "utf8" }).trim();
if (!changed) process.exit(0);

const run = spawnSync("npm", ["test", "--silent"], {
  encoding: "utf8",
  shell: process.platform === "win32",
});
if (run.status === 0) process.exit(0);

const tail = `${run.stdout}\n${run.stderr}`.trim().split("\n").slice(-40).join("\n");
process.stderr.write(
  (input.stop_hook_active ? "Still failing after the last attempt.\n" : "") +
    "The test suite is failing, so this task isn't done. " +
    "Fix the code rather than editing or skipping the tests, then run npm test.\n\n" +
    tail + "\n"
);
process.exit(2);

Some notes from the docs:

• stop_hook_active is true when the agent is already continuing because a Stop hook blocked it. The docs tell you to check it so you don't block forever on something that can't be fixed. The script above uses it only to change the message.
• There's a built-in cap. After Stop hooks continue the turn 8 times in a row, Claude Code ends the turn anyway. The count resets every time the agent calls a tool. You can change it with CLAUDE_CODE_STOP_HOOK_BLOCK_CAP. If you'd rather give up after a single retry, exit 0 when stop_hook_active is true.
• Use exit code 2, not 1. Exit 1 is a non-blocking error: the agent stops anyway, and all you get is a notice in the transcript.
• The git status shortcut keeps the hook from running your whole suite after "what does this function do?". If you often start sessions with uncommitted changes, compare against a stash or a saved hash instead.

The line "fix the code rather than editing or skipping the tests" is there on purpose. When an agent is under pressure to go green, the shortest path is sometimes it.skip or a looser assertion. Saying so in the block reason puts it right next to the failure.

Cursor: the same idea with followup_message

Cursor has its own hooks system (.cursor/hooks.json). Its stop hook can't block, but it can return a followup_message that Cursor submits automatically as the next user message, which works out to the same thing:

{
  "version": 1,
  "hooks": {
    "stop": [{ "command": "node .cursor/hooks/require-green.mjs", "loop_limit": 3 }]
  }
}

// .cursor/hooks/require-green.mjs
import { spawnSync } from "node:child_process";
import { readFileSync } from "node:fs";

const input = JSON.parse(readFileSync(0, "utf8"));
if (input.status !== "completed") { console.log("{}"); process.exit(0); }

const run = spawnSync("npm", ["test", "--silent"], { encoding: "utf8" });
if (run.status === 0) { console.log("{}"); process.exit(0); }

const tail = `${run.stdout}\n${run.stderr}`.trim().split("\n").slice(-40).join("\n");
console.log(JSON.stringify({
  followup_message: `Tests are failing, so the task isn't done. Fix the code, not the tests:\n\n${tail}`,
}));

Project hooks run from the repo root, so use .cursor/hooks/... paths, not ./hooks/.... loop_limit (default 5) caps how many automatic follow-ups one script can trigger.

CI: the check that doesn't care which agent wrote the code

Hooks run on one machine for one tool. A teammate on Copilot, a Codex cloud task, or you at 1 AM with hooks turned off all skip them. CI doesn't.

.github/workflows/test.yml:

name: test
on:
  pull_request:
  push:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npm test

Then make the check required on main (Settings → Branches or Rulesets → require status checks to pass). A workflow nobody is required to pass is just a suggestion with a green badge.

If you run parallel agents, this is also where the integration problem gets caught. Each agent's branch can pass on its own. Merge them into one integration branch (or use a merge queue) and let CI run the full suite on the combined result before anything lands on main.

4. What stays in the rules file

Once tests are enforced, the rules file gets shorter and better. Use it to tell the agent what the checks are, so it runs them itself and the hook rarely has to fire:

## Definition of done
- `npm test` passes and `npm run typecheck` passes. A Stop hook and CI enforce this.
- Never skip, delete or weaken a test to make it pass. If a test is wrong, say so and stop.
- In your summary, list the commands you ran and their results. Don't claim checks you didn't run.

That last line is underrated. It turns "all done! ✅" into something like "ran npm test: 142 passed", which is a claim you can actually check.

A quick sort for your existing rules:

Rule	Where it goes
Naming, structure, preferred libraries	Rules file (prose)
"Run X before saying done"	Rules file and Stop hook
Tests/typecheck must pass to merge	CI, required check
Never commit secrets, never edit generated files	PreToolUse hook or scanner and CI
Your personal workflow habits	Local file, gitignored

Wrap-up

• Put rules where your tools actually read them, and keep one source of truth (AGENTS.md, imported from CLAUDE.md).
• Commit what a reviewer would enforce; keep the rest local. Watch out for the CLAUDE.local.md gotcha.
• Anything that must never break belongs in a Stop hook and a required CI check, not a paragraph.

If you want a starting point, I keep free, MIT-licensed rule sets for Next.js + Supabase + Stripe, Unity 6, Godot 4 and discord.js v14, in .mdc, AGENTS.md and CLAUDE.md formats:

👉 https://github.com/otctrading101/shiprules-free

If you'd rather not wire the hooks up yourself, the paid packs add the full rule sets plus ready-made Claude Code hooks, a zero-dependency scanner and a CI workflow at shiprules.gumroad.com.

What's the most creative way you've seen an agent get to "done"? Tell me in the comments.

---

Disclosure: I make ShipRules (the paid packs above); the free GitHub repo is MIT. This post was written with AI assistance, and the hook and config details were checked against the Claude Code, Cursor and Codex docs in October 2026.

Comments

[makayis]

Moving "ALWAYS RUN THE TESTS" out of prose and into a Stop hook is the right diagnosis — a self-report of "done" is the agent grading its own homework. The AGENTS.md-as-source-of-truth plus a tiny CLAUDE.md that imports it is also a nice way to stop maintaining three rule files. One question: when the Stop hook fails, do you block the session from ending, or let it end and surface a red card the human has to clear?

[ShipRules]

Thanks! It blocks. When the test command fails, the hook exits 2 with the failing output on stderr, so Claude Code doesn't end the turn and the agent gets sent back to fix the tests. The one escape hatch is stop_hook_active: if the hook already fired this turn and tests are still red, it lets the turn end so a stubborn failure can't loop forever, and the red output is what you see. Required CI is the backstop for anything that slips past that.

[makayis]

That stop_hook_active escape hatch is the detail I'd have missed. Without it, a test that can't pass turns into an agent burning tokens forever. Blocking by default and only letting the turn end on a second red run feels like the right order. Do you cap how many fix attempts it gets before you let it end, or is one retry enough in practice?

[ShipRules]

One retry is what the hook gives you out of the box, since stop_hook_active is only true on the second stop. In practice that covers most cases: the first red run is usually a missed import or a forgotten fixture, and if it's still red after a real fix attempt, it's usually a bad test or a design question that needs a human anyway.

If you want more room, keep a small counter file keyed by session_id, bump it each time the hook blocks, let the turn end once it hits 3, and reset it when the tests go green. I'd keep the cap low, though. Past two or three attempts, agents start editing the test to make it pass, which is exactly what required CI is there to catch.

[makayis]

The counter keyed by session_id is a neat way to do it without touching the hook logic. And yes, that last point matches what I've seen: around attempt three the "fix" quietly becomes loosening the assertion. At that point I'd rather the turn end red and a human look at it than get a green run that means nothing. Keeping the cap low it is.

[ShipRules]

Exactly. A red turn is information; a green run from a loosened assertion is worse than no run at all. One cheap guard to pair with the cap: have the Stop hook also fail if the diff touches test files the task didn't ask to change (a quick git diff --name-only check against your test paths). That turns "quietly loosen the assertion" into a visible block instead of a silent pass.

[makayis]

That's a good one, and cheaper than anything I was planning. The only tweak I'd make is letting the task declare which test files it's allowed to touch, so "add a test for X" doesn't trip it. Otherwise git diff --name-only against the test paths and a non-zero exit is all it takes. Stealing it.

[ShipRules]

Agreed, that's the right refinement. Easy way to wire it: have the task write the test paths it's allowed to touch into a small file (say .agent/allowed-tests), and have the hook drop those from the git diff --name-only list before checking. Anything left still fails the hook, so "add a test for X" goes through but quietly editing an existing assertion doesn't.

[makayis]

That's the clean version. I like that the allowlist is written before the work starts, so the agent can't widen it halfway through without that showing up in the diff too. Going to try it on the next long run.

[ShipRules]

Right, and you can lean on that: have the hook also fail if .agent/allowed-tests itself shows up in the diff, so the list can only be set before the run, not during it. Curious how it holds up on the long run.

[asagiman]

Your Stop-hook/required-CI split is exactly the control layer I want to compare with role/task preflight. A green hook can prove that the command exited 0, but it may stay green when the wrong role performed the task, the approved task scope changed, or an in-scope edit weakened an acceptance check.

Would you be open to one small combined fixture using shiprules-free? Baseline green; then (1) the task requests an out-of-scope file or role and (2) tests stay green after a weakened check. We would publish which layer catches each case, setup time, overlap, false positives, and whether the extra preflight is worth keeping. ARC checks supplied declarations offline; it does not replace hooks or CI. Template: github.com/suirindo/agent-role-con...

[ShipRules]

Fair split. A Stop hook only proves the test command exited 0, so it won't notice a wrong role or a task scope that drifted, and it can't tell that an acceptance check got weaker. My guess for your two cases: the out-of-scope file or role is a preflight or review catch, and the hook stays green. The weakened check is where required CI plus protected test files (CODEOWNERS on the test directory, say) earns its keep, since the hook just runs whatever tests the agent left behind. Happy to look at the fixture, and shiprules-free is public if you want it as the baseline repo.

[asagiman]

Thanks—that’s exactly the split. I’ve pinned shiprules-free main at 8b8302e9551fb8784d7cbe7fa61eeecf127bf247 and opened the reproducible comparison here: github.com/suirindo/agent-role-con...

The fixture has four cases:

1. Clean Godot 4 input inside the declared fixture path → ARC PASS, scanner PASS.
2. Scanner-green edit outside the declared task/role scope → ARC expected FAIL, scanner expected PASS.
3. In-scope Godot 3 syntax regression → ARC scope expected PASS, scanner expected FAIL.
4. Repair → both PASS.

I’ll publish the pinned source, exact commands/results, overlap, setup cost, and false refusals. I’m keeping weakened-test semantics as a separate follow-up because the free repo does not contain the paid hook/CI files, so I won’t infer those implementations. Until you review or run it, I’ll label this maintainer-executed—not external hands-on evidence.

[ShipRules]

Nice, that four-case layout makes the split easy to read. Case 2 (scanner green, out of scope) and case 3 (in scope, Godot 3 syntax) are the two I'd expect each layer to own, so seeing them fail on opposite sides is the useful result. 8b8302e is current main, so the pin is right. Agreed on keeping weakened-test semantics separate since the hook and CI files aren't in the free repo. I'll read through the issue and run the cases when I get a chance, and I'll say so there if I do.

[s t]

Good breakdown, especially the CLAUDE.local.md gotcha. One thing worth flagging for anyone layering a "done" gate with other Stop hooks: Claude Code's Stop config is an array, so if you already have a notification hook writing to settings.json's hooks.Stop list (I build CatCode, a desktop pet that reacts to the Stop event), adding require-green.mjs means appending a second entry under that same Stop key, not replacing the file — overwrite it and you silently lose whichever hook was there first. Exit code 2 from your test gate also means any later hook in that array won't see a "clean" turn until the agent actually fixes things, which is correct, just worth knowing if you're chaining a few of these.

Disclosure: I'm the author of CatCode.

[ShipRules]

Good catch, thanks. Yes, hooks.Stop is an array, so add the gate as another entry and keep what's already there. Pasting over settings.json is the easiest way to lose a hook without noticing.

One thing on chaining: Claude Code runs all the matching hooks for an event in parallel, not one after another. So a notifier like yours still fires when the test gate exits 2, which means it can react to a turn that's about to get sent back to fix tests. Worth knowing if the pet celebrates a little early.

[pang]

Treating the agent's "done" as a self-report and letting a Stop hook settle it is a clean split.

[ShipRules]

Thanks! That split is the whole idea: the agent can say whatever it likes, but the turn only ends when the test command actually exits 0. One tip if you try it: check stop_hook_active in the hook input so a stubborn failure can't loop forever.