My linter reported 700 problems. Almost all of them were wrong.

Aashish — Wed, 04 Nov 2026 09:18:09 +0000

AI coding agents read your AGENTS.md or CLAUDE.md on every request. Those files rot like any other docs: a folder gets renamed, a script disappears, and the agent keeps following instructions that stopped being true months ago.

I built rulecheck to catch that. It reads your rules file and checks it against your repo: do the paths exist, do the npm run and make commands point at real scripts, are rules repeated, do two rules recommend competing tools. It runs offline, needs no API key, and never executes your commands.

Then I ran it on real projects, and it embarrassed me.

The first run

I pointed the first version at 10 popular open-source repos (Next.js, n8n, Grafana, Airflow, Deno, Codex, LangChain, Langflow, llama.cpp and Excalidraw), 92 rules files in total. It reported more than 700 problems. Airflow scored 4 out of 100 and n8n scored 0.

Almost every finding was wrong.

What the false positives looked like

Code, not paths. console.log, it.skip and nodes.value.push were read as files.
Version numbers. 3.11 and 0.2.1 were read as folders.
Placeholders. path/to/file.ts, my_script.py and vYYYY_MM_DD.py are examples, not files.
Build output. ./target/debug/deno does not exist until you build it.
Prose. dark/light and scroll/hover are words separated by a slash.
Claude Code imports. @AGENTS.md is an include directive, not a file named @AGENTS.md.
Repo names. vercel/next.js is a GitHub slug, not a folder.
Built-in commands. pnpm add is not a script missing from package.json.
The same file twice. Deno keeps shared rules in more than one file, and the copy was flagged as a duplicate.

The rule that fixed most of it

A linter that cries wolf gets uninstalled, so I adopted one rule: precision beats recall.

rulecheck now reports a missing path only when it can be sure. src/utlis/ is reported because src/ exists and the last piece does not. Anything that looks like an example, a placeholder, build output, a gitignored file, or something the rule itself calls optional is skipped. It will miss some real problems, and I am fine with that.

The results after tuning

Repo	Score
Codex	100
llama.cpp	100
Deno	100
Excalidraw	100
Next.js	96
LangChain	96
n8n	92
Langflow	90
Grafana	84
Airflow	76

Most of the lost points are not mistakes. Several rules files run past 5,000 tokens (Grafana's router file is about 11,000), and Airflow repeats a block of bullet points in one file.

The one real catch

Langflow's AGENTS.md tells agents that components live in src/backend/base/langflow/components/. That folder no longer exists. The components live in src/lfx/src/lfx/components/, so an agent following the file would look in the wrong place.

I opened a pull request with the fix: langflow-ai/langflow#16140. At the time of writing it is still open.

What I took away

Ten well-maintained projects is a small sample, but their rules files were in better shape than I expected. Rot is real, just smaller than I feared.
The hard part of a linter is not finding problems. It is refusing to invent them.
Test on real repos early. The fixtures I wrote myself never contained anything as messy as @AGENTS.md.

Try it

npx rulecheck

It also has a GitHub Action that annotates pull requests, and it can generate a README badge. The code is at github.com/aashish1876/rulecheck.

Limits: contradiction detection is heuristic, token counts are estimates (characters divided by four), and it checks references, not whether your rules are good advice. If it flags something wrong in your repo, please open an issue. Those are bugs, and they are the ones I most want to hear about.

DEV Community: Aashish