AGENTS.md Mistakes That Cost Teams Time and Money
Over 60,000 open-source repositories now ship an AGENTS.md file, and the open standard—stewarded by the Agentic AI Foundation under the Linux Foundation—has become the de facto way to give AI coding agents project context. Here's the uncomfortable part: most of those files are making agents slower, more expensive, and less accurate. Research from ETH Zurich and arXiv studies in early 2026 found that poorly written AGENTS.md files reduce task success rates and raise inference costs by over 20%. The pattern I've observed across teams is what I call the Instruction Optimization trap—where the pursuit of comprehensive coverage directly undermines the very performance gains the standard promises.
The Auto-Generation Trap
LLM-generated AGENTS.md files reduce agent performance while quietly inflating your API bill. The data is unambiguous: auto-generated context files reduce task success rates by 0.5-2% and increase inference costs by 20-23% compared to human-written files, per AI Coding Patterns. These files also add 2.45-3.92 extra steps per task without improving outcomes.
Why does generation backfire? The model producing your AGENTS.md doesn't know which conventions are non-inferable versus which ones the agent would deduce from the codebase anyway. It duplicates information, adds verbose explanations, and buries critical rules under layers of prose. Every unnecessary line gets injected into every API call. At 15,000 tokens of base system prompt overhead in some tools, per OpenClaw's token analysis, that's real money spent on noise.
The fix isn't to abandon AGENTS.md. Human-written files improve performance by 2-6% on average. The catch: they must be written as operational policy, not documentation. Command-first, task-organized, with explicit "done" criteria. Not a style guide essay. Not an architecture overview. The agent needs to know pnpm test not your philosophy on testing.
For a deeper dive on writing minimal, high-impact files, see How to Write an AGENTS.md File.
The Length Ceiling Nobody Talks About
Frontier models show instruction-following degradation beyond 100-250 instructions, with even the best model scoring only 68% at 500 instructions, per Paddo's analysis of the IFScale benchmark. This isn't gradual decline—it's threshold decay. Performance holds steady, then falls off a cliff.
Here's what makes this particularly insidious for AGENTS.md: primacy bias means earlier instructions get more attention, and middle-context dead zones cause agents to skip rules entirely rather than follow them poorly. It results in zero compliance on rules the model never processes.
The research on AGENTS.md optimization confirms files exceeding 100 lines cause measurable instruction decay. Yet some developers report reactive rule addition—adding a new line every time an agent makes a mistake—can lead to files growing to 300, 500, even 1000 lines, per Paddo's analysis. Each addition feels like improvement. The aggregate effect is a file that costs more and helps less.
The 32 KiB default cap in the AGENTS.md spec exists for a reason. Treat it as a hard constraint, not a generous limit.
The Three Configuration Smells Wrecking Most Files
A systematic study of 100 popular open-source repositories found configuration smells are widespread. The arXiv research identified six distinct smells with alarming prevalence:
-
Lint Leakage: 62% of files include linter or formatter rules that duplicate existing config (
.eslintrc,biome.json, etc.), wasting tokens on information the agent could reference directly - Context Bloat: 42% contain verbose explanations, redundant instructions, or unnecessary examples that increase token costs without improving results
- Skill Leakage: 35% mix in capability descriptions or workflow explanations that belong in separate skill definitions, not the core instruction file
These smells frequently co-occur. A file with Context Bloat often has Skill Leakage too—the same verbose tendency pollutes multiple categories.
The underlying issue is categorical confusion. AGENTS.md should focus on behavioral expectations, rules, and constraints—what the agent must do and must not do. Goals, motivation, and architectural philosophy belong in README.md or other documentation. When you write AGENTS.md as human documentation—prose paragraphs, vague directives like "be careful," ambiguous priorities—the agent ignores most instructions, per pattern research on what actually changes agent behavior.
What Effective AGENTS.md Files Actually Look Like
The best files share a structure that respects the agent's processing constraints. Here's how the successful patterns break down:
| Element | What Works | What Fails |
|---|---|---|
| Length | Under 100 lines, under 32 KiB | 300+ line files with reactive rule addition |
| Style | Command-first: exact invocations, explicit "done" criteria | Prose paragraphs, vague directives, philosophy |
| Content focus | Behavioral rules, constraints, build/test commands | Goals, motivation, architecture overviews |
| Generation method | Human-written, surgically curated | LLM-generated from existing docs |
| Maintenance | Reviewed when conventions change; stale rules removed | Accumulated reactively without pruning |
The cross-tool portability of AGENTS.md—working across 21+ tools from a single file—is its genuine superpower. But that portability only helps if the file's contents are correct. A broken standard file propagates broken behavior everywhere.
For teams looking to optimize further, AGENTS.md Best Practices covers how curated minimal files with only non-inferable rules cut task time and reduce agent-generated bugs.
The Maintenance Tax Nobody Budgets For
AGENTS.md is not a set-and-forget convenience. It's a runtime operational contract that requires active maintenance to avoid performance decay. For large engineering teams, the open standard is not a set-and-forget convenience but a runtime operational contract that requires active maintenance to avoid performance decay, as explored in AGENTS.md for Large Engineering Teams.
The decay mechanism is straightforward: conventions evolve, dependencies change, build commands get updated. An AGENTS.md that still references npm test after your team migrated to pnpm test doesn't just fail to help—it actively misleads. The agent follows the wrong command, gets an error, and either burns tokens on recovery or produces broken output.
Large engineering teams face a compounding problem. With multiple contributors, AGENTS.md becomes a shared editing surface prone to edit wars: one engineer adds a strict rule after a bad experience, another softens it after false positives. Without ownership and review discipline, the file bloats and contradicts itself.
The sustainable model treats AGENTS.md like a Makefile or CI config—versioned, reviewed, tested. Changes go through the same PR process as code. And just as you wouldn't let your CI config reference deprecated commands, your AGENTS.md needs the same freshness guarantee.
Making the Tradeoff Explicit
So should your team use AGENTS.md at all? The answer depends on your constraints, not the hype.
Use it if: You run multiple AI coding tools (Cursor, Copilot, Codex, etc.) and need consistent behavior; your conventions are non-obvious from code alone; you can commit to active maintenance.
Skip it if: You're single-tool and that tool's native format (CLAUDE.md, .cursorrules) suffices; your codebase is small and conventions are obvious; you can't maintain freshness.
Be careful if: You're tempted to auto-generate from existing docs, or you think more context is better. The research is clear: LLM-generated files reduce success rates, and files over 100 lines cause instruction decay.
The teams getting value from AGENTS.md treat it as surgical instrumentation—precise, minimal, human-curated. The ones burning money treat it as documentation dumping ground. The standard itself is sound. The mistake is in how we use it.
What's your AGENTS.md line count right now—and when was the last time you removed a rule that had become obsolete?
Top comments (0)