Rules files don't work unless they're testable

Every team I know that adopted AI coding agents did the same thing first: they wrote a rules file. CLAUDE.md, .cursorrules, AGENTS.md. Pages of conventions, style guides, and stern warnings.

Then, three weeks later, the agent force-pushed to main anyway.

Here's the uncomfortable truth: a rule that isn't testable is a suggestion. And agents treat suggestions the way a junior dev treats a comment that says "don't change this" — with curiosity.

Why rules files fail

Rules files fail the same way documentation fails: nothing checks them. When a rule says "never commit directly to main," there is no mechanism enforcing it. There's just a paragraph of English hoping a probabilistic model feels cooperative today.

The failure modes are predictable:

Context window eviction. Long sessions push your rules file out of the effective context. The agent isn't disobeying. It literally cannot see the rule anymore.
Instruction dilution. A 400-line rules file is 400 lines of equal-weight text. The model can't tell "we prefer semicolons" from "you will break production if you do X."
No feedback loop. When the agent violates a rule, nothing happens. The violation ships. You find out in review, or in an incident.

The fix: turn rules into checks

The teams getting real value from agents figured out one thing: every rule worth writing down is worth expressing as a test, a hook, or a CI gate.

The pattern looks like this:

Rule (English): "Never commit secrets."
Check (enforced): a pre-commit hook that scans staged diffs for high-entropy strings and known key patterns, and refuses the commit.

Rule: "All new endpoints need tests."
Check: a CI job that fails when git diff --name-only shows a new route file without a matching spec file.

Rule: "Don't force-push shared branches."
Check: a server-side pre-receive hook or branch protection rule. Full stop, no exceptions, no agent creativity.

The English version still belongs in your rules file — it explains why. But the check is what makes it true.

The testable-rule audit

Go through your rules file right now and mark every rule with one of three labels:

Enforced — a hook, test, linter, or CI gate fails when it's violated. Keep.
Testable but not enforced — you could write a check. Write it this week. These are your highest-risk gaps.
Judgment call — genuinely not testable ("prefer composition over inheritance"). Keep these, but move them to a short "taste" section and accept the agent will get them wrong sometimes.

When I run this audit on real setups, the split is usually 15% enforced, 55% testable-but-not, 30% judgment. That middle bucket is where agents hurt you.

A worked example

One of the highest-leverage conversions I've done:

# .git/hooks/pre-push
branch=$(git rev-parse --abbrev-ref HEAD)
if [ "$branch" = "main" ] || [ "$branch" = "master" ]; then
  echo "Direct pushes to $branch are disabled. Open a PR."
  exit 1
fi

Four lines. It replaced a paragraph of rules-file prose that three different agents had independently ignored. The paragraph explained policy. The hook is policy.

The deeper win

Something unexpected happens once your rules are checks: your rules file gets shorter and better. You delete the rules that are now enforced (the hook is the documentation) and what remains is the actual hard stuff — architecture intent, naming philosophy, when to break the rules.

Agents do noticeably better with a 60-line file of real judgment calls than a 400-line file of unenforced commands.

I packaged my full setup — the testable CLAUDE.md, the git hook suite, the subagent patterns, and the audit checklist — into The Agentic Coding Kit. It's $19, one-time, with free v1.x updates. If it saves you one afternoon of wrangling a runaway agent, it paid for itself.

DEV Community