Over on r/ClaudeCode, in a thread titled “Claude runs 1800 useless tests; creates more against my guidance”, an engineer described the purest agent problem of the year: they told the agent to stop writing so many tests, and the agent — politely, helpfully, immediately — wrote more tests.
Picture the scene — invented, but you have lived a version of it. You ask for tests for one small fix. You go and make chai. You come back to a small army of new test files, all passing. The coverage number (we will get to it) has gone up. CI — continuous integration, the automated checks that run on every change before it can merge — takes noticeably longer. And with 1,800 tests standing guard, you trust the suite less than you did this morning.
That feeling is the correct response to a suite that grew in count and shrank in meaning. Here is why it happens, and the guardrails that stop it.
The agent is not disobedient. It is obedient to the wrong score
Start with sympathy, because it makes the fix obvious. An agent — an AI tool that writes and runs code for you, like Claude Code — can see some things clearly and other things not at all.
A test count is visible. It goes up. It photographs well in a summary: “I added 240 tests, all passing.” Put that next to its honest rival: “I added 6 tests, and decided the other 234 were not worth writing.” The second sentence is better engineering — and much harder for an agent to say, because nothing rewards it.
Value is invisible at writing time. Whether a test catches a real bug next March is a question about the future; whether it exists is a question about right now. Agents, like all of us on a deadline, optimise for what can be shown. A test is cheap to produce, looks like diligence, and almost never gets argued with in review — who deletes a passing test and feels safe?
The fix: if count is the score the agent can see, make value visible too — and put a ceiling on count. That is what the rest of this post does, one guardrail at a time.
Coverage went up. Confidence did not.
Now the number. Line coverage means exactly this: of all the lines in your project, what percentage were executed — actually run — while the tests ran. That is the whole definition. A coverage tool ticks off every line it sees touched and reports the fraction.
Notice what is missing. Nothing in that definition asks whether the test checked anything.
A test can call a function, assert nothing about the result — an assertion is the line that says “this result must equal that, or fail” — and every line it touched still counts as covered. The Stryker documentation — Stryker is a mutation testing tool, hold that thought — puts it with a sandwich: coverage tells you the bread is 80% covered with paste. It does not tell you whether the paste is chocolate or something you would not serve anyone.
This is why 1,800 tests can leave you colder than 200 did. The old 200 were written by people with a specific fear in mind: this calculation once shipped wrong, this edge case once bit us. Each was a small promise about one behaviour. The new 1,600 were written to exist. They visit the code. They do not interrogate it.
And they are not free. Every one runs in CI on every change, forever. Every one can break when you rename something harmless, teaching the team the most dangerous lesson a suite can teach: red does not necessarily mean broken. A team that learns that starts merging on red — and then you do not have a test suite. You have a very slow random number generator with opinions.
CLAUDE.md is advice, not a lock
The thread poster’s real complaint was not the first 1,800 tests. It was the “against my guidance” part. They had guidance. It was written down. It was ignored anyway.
Here the official Claude Code documentation is blunt: CLAUDE.md files — the instruction files you put in a project to tell the agent how to work — are loaded at the start of every session as context, not enforced configuration. The agent reads it, tries to follow it, and can still drift — especially as the file gets longer (the docs warn that longer files reduce adherence) or when two instructions pull against each other (the docs say the agent may then pick one arbitrarily).
This is not a flaw to be angry about. It is a category to be clear about. CLAUDE.md is a note on the fridge. Good notes genuinely help. But “please do not eat the cake” is not a lock on the fridge, and the day the cake matters, you will wish you had not confused the two.
So yes, write the test budget into CLAUDE.md. Write it short, write it specific, write it like this:
## Testing rules
- Budget: at most 10 new tests per change. Need more? Ask first, with a reason.
- One behaviour per test. Name the test after the behaviour, not the function.
- Every new test must fail without the fix and pass with it. No exceptions.
- No test that only checks a mock was called. Assert the result a user would see.
- If a test duplicates an existing one, extend the old test instead.
A mock, in that line, is a fake stand-in for a real part of the system — a pretend database, a pretend payment service. A test that only checks “the mock was called” proves your code phoned a friend. It says nothing about whether the friend gave the right answer.
Keep the budget in CLAUDE.md. Just know what it is: a strong request to a reader who usually honours it. For rules that must hold every time, you need the machinery below.
Make every new test fail first
Here is the cheapest value-detector in all of testing, and it costs one extra run.
Before you trust a new test, run it against the code without the fix. If it passes there, it has told you something devastating: it cannot tell broken from fixed. It would have passed on the day the bug shipped, and it will pass on the day the bug comes back. It is not a guard. It is a spectator with a ticket stub that says “guard”.
Delete it, or fix it until it fails for the right reason — until the failure message is the bug, wearing a name tag.
Fail-first also gives the agent a target it cannot fake. “Make it pass” is gameable. “Here is a test that currently fails — make it pass by fixing the code, not by touching the test” is a contract.
One honest warning: it only works if somebody actually runs the test against the old code. Left as a CLAUDE.md line, it is advice again. Left as a habit in review — “show me this test failing before the fix” — it becomes culture. Culture, unlike context, compounds.
Break your own code on purpose
Now the heavy weapon, and the one that answers the coverage problem directly: mutation testing.
The idea is beautifully rude. A mutation testing tool — Stryker is the well-known one — takes your production code and deliberately breaks it in small ways, one break at a time. It changes a >= to a >. It flips a true to a false. Each broken version is called a mutant. Then it runs your tests against each mutant.
If at least one test fails, the mutant is killed. Good — your tests noticed. If every test still passes, the mutant survived, and a survivor is an accusation with an address: here is a specific change to your real code that your entire suite would wave through.
Against a mutant, there is no rhetoric. Either some test noticed that the age check now lets a younger user through, or none did. As Stryker’s docs put it: the higher the percentage of mutants your tests kill, the more effective your tests are — a claim coverage cannot make about itself.
Use it with proportion. It re-runs your tests for every mutant, so it is slow — run it on the modules where a silent bug would hurt (money, permissions, the core calculation), not as a toll every small pull request must pay. A toll everyone resents gets removed by Friday. A check aimed at dangerous code survives, because it keeps catching things.
Put the guardrails where they cannot be talked out of
Claude Code gives you two harder layers, both configured in a settings file (.claude/settings.json) rather than in prose. The first is permissions: allow, ask, and deny rules for what the agent’s tools may do. A deny rule prevents the use it names — rules are evaluated deny first, then ask, then allow, and an allow cannot carve an exception out of a deny. The second is hooks: your own commands that the tool runs automatically at fixed points in its work. A PreToolUse hook runs before a tool call executes and can block it — exit code 2 from the hook script means “no, this does not run”. A PostToolUse hook runs after a tool succeeds; it cannot un-run anything, but it can react — log it, check it, complain loudly.
Advice in CLAUDE.md. Enforcement in settings. That sentence is the whole architecture.
Here is the shape of it, as an illustrative pattern — your script names and thresholds are yours to choose:
{
"permissions": {
"deny": [
"Read(./.env)",
"Read(./.env*)"
]
},
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{ "type": "command", "command": ".claude/hooks/check-test-budget.sh" }
]
}
],
"PostToolUse": [
{
"matcher": "Write|Edit",
"hooks": [
{ "type": "command", "command": ".claude/hooks/note-new-test.sh" }
]
}
]
}
}
Read it top to bottom. The deny rules say the agent never reads the secrets file, in any mood, in any session. The PreToolUse hook inspects a shell command before it runs and can veto it — that is where a script counting new test files against the budget belongs, because a veto cannot be talked out of anything. The PostToolUse hook cannot veto a file just written, but it can count it and put the running total in front of everyone while there is still time to act.
Then the last gate, the one nobody can charm: CI itself. A CI gate (a check the build must pass before merging) is team policy in pipeline code. Two gates fit this problem: fail the build when a change adds an absurd number of tests with no explanation, and run the mutation check on critical modules so a suite that only looks strong cannot merge quietly. The agent does not experience these as suggestions. That is rather the point.
The test-bloat checklist
Pin this one. Work it the next time an agent offers you a stadium of tests.
Set a test budget: A number, per change — at most 10 new tests is a sane start. Over the budget needs a reason, stated first.
One behaviour per test: If the name needs the word “and”, it is two tests wearing one coat.
Fail first: Run every new test against the code without the fix. A test that passes on broken code proves nothing — delete or repair it.
Assert the outcome, not the mock: Check the result a user would see, not that a fake dependency received a phone call.
Delete the duplicate: If a new test guards a behaviour an old test already guards, extend the old test instead.
Keep CLAUDE.md short and specific: It is context, not law — longer files get followed less reliably. Every line must earn its place.
Move must-never rules to hooks and deny: Anything that must hold regardless of what the agent decides belongs in settings — a PreToolUse hook that can block, a deny rule that beats every allow.
Gate it in CI: Make unexplained suite growth and a falling mutation result on critical code fail the build. A gate does not negotiate.
Mutate the critical code: Run mutation testing where a silent bug hurts most. Every surviving mutant is a named weakness — fix those before adding any new test.
Review the count, not just the diff: Ask out loud, every time: “How many tests did this add, and what does each one promise?”
Coverage counts the lines your tests touched. It never asks whether they were holding anything.
Fewer tests. Stronger promises.
The goal was never 1,800 tests, and it was never 18 either. The goal is a suite you believe on the worst morning of the year — when it goes red at the moment you least want it to, and you stop and look, because its red has always meant something.
That suite comes from budgeting tests, failing them first, breaking the code underneath them on purpose, and putting the rules that matter where a polite paragraph cannot be talked out of them. Do that, and the next gift the agent brings you will be small, sharp, and worth keeping.
Staging warning: try new hooks, deny rules and CI gates on a staging or throwaway branch first — a PreToolUse hook that blocks too much will block your real work just as faithfully, and you want to learn its temper where it costs nothing.
Disclaimer
- This post describes my personal experience with my own systems and accounts, in my own words. It is a first-person account of what I observed and what I do in my own work — not a statement of any company's intentions, policies, or internal decisions. Where I do not know something, the post says so.
- I am not affiliated with, endorsed by, or speaking for any company, product, or vendor named in this post. All product and company names are trademarks of their respective owners and are mentioned only to describe the experience accurately.
- Any error messages, documentation excerpts, or figures quoted here are reproduced from my own records or from the public sources linked in the post, believed accurate at the time of writing. Product behavior changes over time; verify against current official documentation before relying on any detail.
- Nothing in this post is professional, financial, legal, or security advice for your specific situation. Examples are illustrative — validate in your own environment before running anything in production.
- Corrections are welcome. If any company named here publishes an official statement, or if any detail is wrong, I will update this post and note the correction. The goal is an accurate public record, not a verdict.
© 2026 Honey Kiran. All rights reserved.
Honey Kiran · DevOps & Platform Engineering
Claude Code, AI agents, CI/CD, testing, and DevOps — explained the practical way.
How many tests did your agent write this week — and if you deleted half of them tonight, which half would you actually miss? Tell me in the comments.
Examples are illustrative — validate in your own environment before running in production.
Top comments (0)