You wrote the AGENTS.md. Or the CLAUDE.md, or the team best-practices doc. A month later, how many of those practices does any agent actually follow? If you can't answer with a number, you don't have practices — you have wishes.
I learned that twice. The second time was worse.
Documented ≠ adopted
I run three production projects on Claude Code, and I'd written a model-routing rule into my shared config: mechanical work — inventory, renames, boilerplate — goes to the cheap model; hard reasoning — architecture, security, migrations — goes to the expensive one. I wrote it, promoted it to an auto-adopted practice, felt good about it.
Then I built a small usage analyzer to see where the tokens actually went. Zero dependencies — it just reads the session transcripts the tool already writes to disk. The cheap model came back at 0.03% of output: 18k tokens against tens of millions. Not "underused." A rounding error.
Documentation is not a control surface. The doc had not routed a single task.
The fix that doesn’t work
The reflex is to write it again, bigger. Add a reminder. Write a third markdown file about the two markdown files nobody is applying.
This is the same mistake as putting a budget in a prompt. "Be mindful of the model tier" is a wish, exactly like "max 8 searches" is a wish — a model sails past both the moment the task still feels unfinished. If your practice lives only in a doc the session may never open, you do not have a practice. You have a hope with good formatting.
Measure, then enforce below the decision
A practice that lives only in a doc fails for the same structural reason a budget in a prompt fails: nothing forces it into the path where the decision is made. So move it below that path.
Measure adoption from data you already generate. You cannot enforce what you cannot see. The transcripts already record which model ran, which tools fired, which skills were invoked. A zero-dependency reader turns that into an adoption number per practice. That is how I learned the routing rule was near zero — by counting, not guessing.
Enforce at the boundary the model cannot skip. Every session hits session-start. That is the injection surface. Instead of a doc the session may never load, the practice gets printed into the session-start context deterministically: static strings, no extra generation cost, fails open. Same principle as a budget belonging in the harness rather than the prompt — put the practice in the layer the model passes through by construction, not the one it has to choose to read.
Then I audited the counter
A month later I went to re-measure, and found the reader had been lying to me the entire time.
Subagent work is written to a separate sidechain file under the session directory. My reader was not recursive. It never opened those files. Every token a subagent spent was invisible to it.
That matters because of where the cheap model actually runs. Measured properly: 100% of the cheap model's output was subagent work. Zero on the main thread. The one surface the rule governs was the exact surface the counter could not see.
So 0.03% was never an adoption number. It was an artifact of a blind spot. The commit that fixed the reader is titled, accurately, read subagent sidechains — "Haiku=0" was a measurement artifact.
Here is the part that rescues the story: the corrected figure is 0.75% — 370k tokens out of 49.4M. Still terrible. The conclusion survived. The evidence for it did not.
That is the real lesson, and it is less comfortable than the first one: a measurement you never audit is also a wish. It just wears a number, which makes it harder to doubt. I spent a month more confident than my instrument had earned.
The honest caveat
Injection is not adoption either. Firing the reminder every session is necessary, not sufficient. The arbiter is the next measurement cycle — if the cheap-model share climbs, the injection worked; if it does not, the problem is deeper than reminding, and I will know, because I am measuring. A practice you inject but never re-measure is just a wish that loads faster.
The moves, in order
- Measure adoption from data you already generate. The transcript is a telemetry stream you are not reading.
- Audit the instrument. Do not ask whether it looks right — feed it a case whose answer you already know and check that the number comes back. A counter that has never been made to fail has never really passed. Mine had no such test, and when the missing surface finally showed up it moved the figure by more than twenty times.
- Inject the practice where the model passes through by construction, not into a doc it may never open.
- Re-measure. The doc was never the mechanism. The number is — once you have earned the right to trust it.
Step 1 usually tells you your existing docs are decorative. That is not a reason to write better docs; it is the reason to stop treating docs as controls. Step 2 occasionally tells you step 1 was decorative too.
Top comments (0)