Line 3 of my rules file said "never use any in TypeScript." Line 3. Not buried in an appendix. Third line of the file.
Then I watched an agent write catch (e: any) for the fifth time in one week, in a repo where the linter would have caught it if I hadn't been lazy about the lint config.
So I stopped arguing with it and ran an experiment instead. I hid canary instructions all through my rules file, at different depths, in different phrasings, and logged which ones came out the other side. Your AI coding agent ignores your rules file, but not randomly. It ignores it in a shape, and once you see the shape you can work with it.
TL;DR
- I planted harmless "canary" instructions at different positions in a
CLAUDE.md-style rules file and tracked compliance across roughly 60 real working sessions in two repos. Scrappy personal log, not a benchmark. - Top-of-file rules held up well. Middle-of-file rules were the worst by a wide margin. Bottom-of-file rules recovered partially.
- Rules phrased as prohibitions ("never do X") lost to rules phrased as actions ("when you touch a route file, do Y").
- Compliance decayed as the session got longer, even for rules that were obeyed perfectly in the first ten minutes.
- The fix isn't better prose. Anything you actually care about belongs in a hook, a linter, or CI. The rules file is taste; the check is law.
What is a canary test for an AI agent's rules file?
A canary is an instruction that is harmless to follow, impossible to follow by accident, and trivially greppable afterward. That last part is what makes it a measurement instead of a vibe.
Mine looked like this, scattered through a rules file of about 120 lines:
- "When you create a new helper file, prefix the filename with
zz_." - "When you edit a test file, add the comment
// canary-7above the first test." - "Start your first reply in every session with the word
MOCKINGBIRDon its own line." - "When you write a SQL query, put the keyword
SELECTon its own line."
None of them change behavior I care about. All of them show up in a git diff or a transcript grep. I put duplicates of the same canary at different depths in different repos so position was the variable, not wording.
Then I just... worked. Normal tasks, normal length, normal mess. No synthetic prompts, because synthetic prompts are exactly where rules files look great.
Why your AI coding agent ignores your rules file
Because your rules file is not an instruction. It is context, and it competes with everything else in the window.
Here's what actually happens. Your rules file gets injected once, near the front. Then the agent reads six files, runs the test suite twice, gets a 400-line stack trace, greps the repo, and reads three more files. By the time it writes the code where your rule applies, the rule is thirty thousand tokens back, sitting behind a wall of your own tool output that is far more recent, far more specific, and far more relevant-looking.
The model isn't defying you. It's doing what the local evidence suggests. And the local evidence is a stack trace, not your style guide.
Rough shape of what I logged:
| Canary position in file | Held up |
|---|---|
| First ~15 lines | Consistently |
| Middle third | Rarely, the clear worst bucket |
| Last ~15 lines | Often, but flaky |
| Repeated in two places | Nearly always |
The middle is a dead zone. Everyone who has read about long-context retrieval has seen a version of this curve, but it lands differently when the thing being forgotten is the rule you wrote specifically because the agent kept breaking it.
The second effect was time. A canary obeyed cleanly in the first few edits of a session would quietly stop appearing forty tool calls later. Same file, same rule, same session. It just aged out of relevance.
Which rules survive, and which ones evaporate?
Four patterns showed up strongly enough that I now write rules files differently.
1. Rules with a trigger beat rules with a principle. "Prefer small functions" evaporated. "When a function passes 40 lines, split it and say why" survived. A trigger gives the model a matchable condition at the moment of writing. A principle gives it a vibe to weigh against a stack trace, and the stack trace wins.
2. Positive rules beat prohibitions. "Never use any" underperformed "when a type is unknown, use unknown and narrow it." Prohibitions require the model to notice it's about to do something. Positive rules give it a thing to do instead. Same content, different survival rate.
3. Short rules beat paragraphs. My most-ignored rules were the ones I had written most carefully, with rationale, examples, and exceptions. The rationale is for humans. Every extra clause is another sentence for the important part to hide behind.
4. Rules the agent can verify beat rules only I can verify. Anything with a command attached ("run npm run lint:types before you claim it works") held up dramatically better than a rule about the shape of the output. A checkable rule closes its own loop.
The uncomfortable summary: my rules file was mostly a document about my preferences, written to convince a reader who does not exist.
How do you make an AI agent actually follow your rules?
Stop writing rules and start writing gates. I now sort every rule I have into three tiers, and the tier decides where it lives.
Taste goes in the rules file. Naming style, comment density, how much I like early returns. If the agent misses it, I shrug and fix it in review. This is the only tier where prose is the right tool, and it should be short.
Habit goes in the prompt, at the moment it matters. If I'm about to have an agent touch the auth layer, I say the auth rule out loud in that message. Recency is free leverage and I stopped being too proud to use it. This is also where re-stating a rule mid-session pays for itself, because relevance decays with distance.
Law goes in a hook, a lint rule, or CI. no-explicit-any as an error, not a warning. A pre-commit hook that rejects the thing. A test that fails. This tier is where every rule I have ever complained about the agent breaking actually belonged, and me writing it in Markdown was avoidance dressed up as configuration.
The tell is simple: if you'd be genuinely annoyed to find the rule broken in a PR, it isn't a rule. It's a check you haven't written yet.
Two practical things I changed the same afternoon. First, I cut my rules file roughly in half, because a shorter file has less middle. Second, I moved my three most-violated rules into the lint config, and the violations went to zero, which is the least surprising and most embarrassing result of the whole experiment.
The canaries are still in there, by the way. They're a smoke test now. When MOCKINGBIRD stops showing up on long sessions, I know my context is bloated before the code quality tells me.
So does your AI coding agent read your rules file?
It reads it, and then it gets outvoted. Your rules file is loaded once at the front of the context and then buried under file reads, test output, and stack traces that are more recent and look more relevant at the moment the code gets written. In my canary test the top of the file survived, the middle third mostly didn't, and everything decayed as the session got longer. So put taste in the rules file, keep it short, phrase rules as triggers and actions rather than prohibitions, repeat the ones that matter in the message where they matter, and move anything you'd actually block a PR over into a linter, a hook, or CI, where compliance doesn't depend on attention at all.
Top comments (0)