AGENTS.md generators vs validated kits: what a 1,280-rule generator can't check

If you search for AGENTS.md templates or generators right now, one tool owns the results. agents-md.app is free, MIT licensed, runs entirely in your browser, and ships 1,280 short, specific rules in 161 packs covering everything from Next.js and FastAPI to Kubernetes. It scans your repository, preselects matching rules, fills in your commands, exports the same selection as AGENTS.md, CLAUDE.md, Cursor rules, or Copilot instructions, and a GitHub Action re-scans your manifests on a schedule and opens a pull request when the generated block needs a refresh.

Use it. It is a genuinely good tool and I am not going to pretend otherwise. The interesting question is not which generator has more rules. It is what a generator can verify about the file it hands you, because two studies published this year measured exactly where generated config fails, and both failure modes sit on the generation side of that line.

What 532,000 config files say about how config goes wrong

In June, researchers at the Federal University of Minas Gerais published the first formal catalog of configuration smells in agent instruction files (arXiv:2606.15828, accepted at SCAM 2026). They analyzed 532,000 files across the 100 most-starred open-source repositories that ship an AGENTS.md or CLAUDE.md, and 91 of those 100 repositories carried at least one of six named smells:

Smell	Share of the 100 studied files	What it is
Lint Leakage	62%	Rules a linter or formatter already enforces
Context Bloat	42%	Files past 200 lines, where adherence decays
Skill Leakage	35%	Rare-task workflows loaded every session
Conflicting Instructions	28%	Rules that contradict each other
Init Fossilization	24%	Generated once, never revisited
Blind References	16%	Doc paths with no note on when to read them

Read the top row again, because it is the punchline of this piece. The most common defect in the wild, by a wide margin, is files full of generic style rules that duplicate deterministic tooling. One studied repository shipped a full Python style guide, indentation, line length, naming, docstrings, inside its AGENTS.md, and the largest bloated file in the sample ran 1,477 lines across 27 sections.

Now look at what a large rule pack is made of. Formatting rules. Naming rules. Logging rules. A generator offering 1,280 toggleable rules optimizes for exactly the category the study flags as waste, because it cannot know which of those rules your linter, formatter, and pre-commit hooks already enforce with perfect reliability at zero token cost.

To its credit, agents-md.app says this out loud on its own template pages: "Delete rules that do not match your project. Shorter files are followed more reliably." Their open-source-project template combines 99 rules from 14 packs into 142 lines. That is careful work, and it concedes the real point. The value was never the 1,280 rules. It is the culling, and the culling is left to you.

Generated context measured worse than no context

The second study is the harder one. An ETH Zurich team (arXiv:2602.11988) ran coding agents on real tasks with and without context files. LLM-generated context files came out slightly below running with no file at all, 2% lower task success on CTXbench and 0.5% lower on SWE-bench Lite, neither difference statistically significant, while inference cost rose 20 to 23% for both generated and human-written files. Human-written files added 2.4% success, also within noise.

Generated is not a discount version of curated. On this benchmark it measured as slightly negative value: a file the agent pays tokens to load and then follows into marginally worse outcomes. The mechanism is not mysterious. Generation optimizes for plausible coverage, and plausible coverage is the raw material of Context Bloat and Skill Leakage.

One smell generators genuinely fixed is fossilization. A quarter of the studied files were generated once and never touched again, and agents-md.app's scheduled rescan with a review pull request is the right answer to that. Credit where due. But fossilization was the smaller half of the problem, and the two biggest smells, leakage and bloat, are the ones no generator checks. The UFMG detection heuristics caught Conflicting Instructions at only 57% precision, the weakest of the six. Contradiction is a semantic question. Nothing that assembles rules from a catalog is positioned to notice that rule 12 and rule 61 fight each other in your repository.

What validation checks instead

This is the half of the workflow generators skip, so here is what it looks like when you take it seriously, from the kit work I do. Every kit we ship passes a nine-gate validator before release, and the checks map one to one onto ways these files actually fail:

• Required files exist in all four harness formats, no missing-file escapes.
• CLAUDE.md stays under a line budget, enforced in the build, not suggested in a style guide.
• Every Cursor rule parses, with a declared activation mode, because malformed frontmatter fails silently and a rule you believe protects your test files simply never loads.
• The baseline carries its required sections, safety rules included.
• No placeholder text anywhere, TODO through lorem ipsum, case-insensitive.
• No unrendered template variables, a check that exists because it caught a real bug in our own build: a double-formatting trap left the literal text lint_cmd_placeholder inside every kit's definition of done, and the validator flagged it on its first run.
• Skills exist and are non-trivial, no stub files.
• A failing run is a failing release. The exit code is the gate.

Measured this morning, all 12 kits sit at 72 or 73 lines of AGENTS.md, a 40-line CLAUDE.md, and four path-scoped rule files each, comfortably under the 200-line threshold the UFMG study used to define bloat. That is not an accident of writing style. It is a budget with a gate behind it.

The honest limit: validation is structural. It proves the file is well-formed, under budget, internally consistent, and free of rot. It cannot prove the model obeys. Instruction-following accuracy falls as instruction count rises, IFScale measured about 68% at 500 instructions (arXiv:2507.11538). Rules are a strong nudge, not a guarantee. Where something must never happen, the deterministic tool is a hook or a permission, not a sentence, and that is a separate layer from anything a generator or a validator emits. I wrote a fuller breakdown of which config rules actually hold earlier this year.

The five-minute audit, whichever route you take

Generate or curate, run this against your file today:

1. Delete every rule your linter, formatter, or pre-commit hook already enforces. This alone removes the smell carried by 62% of the studied files.
2. Count lines. Past 200 you are in the band where the study put 42% of files. One pull request in their review cut a config file from 598 lines to 149 on the argument that models do better with roughly 150 to 200.
3. Move rare-task workflows into skill files that load on demand instead of every session.
4. Read the file looking for one thing only: two rules that disagree. Resolve them.
5. If a rule cites a doc path, add one line saying when to read it.

If you generated your AGENTS.md from a large pack, the highest-value twenty minutes you can spend on it is deleting a third of it.

That is also why we build the way we build at Piekwerk. AgentConfig Studio ships 12 stack kits, version-pinned, every file under budget, through nine validation gates before release, and the free Next.js sample is on Gumroad if you want to inspect the full format first. Generated gets you a first draft in seconds. Validated is what makes it safe to leave in the repository.