Your AGENTS.md is lying: how configs drift and how to catch it

#ai #devtools #programming #git

A config does not have to be wrong to be harmful. It has to be stale.
Last month I audited a repo where AGENTS.md said tests run with
npm test, but the repo had moved to a Makefile two refactors ago.
The instruction was harmless-looking, confidently wrong, and every
agent that read it trusted it. This post is about drift: how it
happens, a ten-minute audit, and the habit that slows it down.

Why configs drift faster than code

Three reasons, all boring:

Config has no test. A stale import fails CI. A stale instruction fails silently, in a session you are not watching.
Config is everyone's to edit and no one's to own. People add rules when something annoys them. Nobody deletes rules when the annoyance is fixed.
Config describes moving targets. Commands, paths, scripts, and conventions all move. Prose about them freezes the last known state.

The result: config rots at the speed of refactor, and the rot is
invisible until an agent follows a dead instruction into a wall.

The ten-minute drift audit

Run this per repo, per month:

# 1. Extract every command your config claims
grep -nE '\b(npm|yarn|pnpm|make|cargo|go|pytest|python)' AGENTS.md

# 2. For each, check it exists where the config says
grep -n '"test"' package.json          # does the script exist?
ls Makefile 2>/dev/null                 # did the entry point move?

# 3. Extract every path reference
grep -nE '(/[a-z-]+/)+[a-z-]+' AGENTS.md | grep -v http
# then: test -e each path

# 4. Extract tool names, check they're still in the lockfile
grep -iE 'eslint|prettier|jest|vitest' AGENTS.md
grep -iE 'eslint|prettier|jest|vitest' package-lock.json

Anything that fails a check is drift. In my audit above, three of
eleven command references were dead. The config looked well-maintained
and was a third fiction.

Rules that age worst

From auditing my own configs repeatedly:

Exact commands age worst and matter most. Drift here is guaranteed; budget for it.
File paths rot on every refactor. Reference directories, not files, when possible.
Tool names rot on migrations. "The formatter" survives; a pinned tool name needs a maintenance trigger.
Style prose barely ages because it barely says anything. This is another argument for pushing style to linters: the linter never drifts from itself.

The decisions ledger

The habit that slows drift: a dated decision log, one file, append
only.

## 2026-08-14: moved from npm scripts to Makefile
- AGENTS.md commands section updated (PR #212)
- old npm aliases kept for 2 weeks, then removed

The ledger does two things. First, it forces the question "what does
this change make stale in the config?" at the moment of the change,
when the answer is cheap. Second, it gives future auditors (including
agents) a timeline: a rule with no recent entry decays in trust.

Verification you can automate

Two checks that catch most drift mechanically:

CI job that runs the config's own commands in dry or help mode: make -n test, npm run test -- --help. If the config says it, CI executes it. Dead commands fail the build.
A config test that greps paths: extract path-like tokens from AGENTS.md and test -e them. Ten lines of script, catches moved directories instantly.

We run both in the validator for our own kits. The first drift check
is also the cheapest quality signal a config can have: configs whose
commands run are configs whose other claims deserve trust.

DEV Community