DEV Community

Cover image for How Much Should You Invest in Your AGENTS.md?
Andre Burgstahler
Andre Burgstahler

Posted on Originally published at Medium

How Much Should You Invest in Your AGENTS.md?

#codex #ai #claude #productivity

This article first appeared on Medium: https://medium.com/@r3dtuxedo/how-much-should-you-invest-in-your-agents-md-69f8e36583bc

Less than you think, and in different places than you think. Keep the file lean, link out to workflows and decisions, move enforcement into tooling, and test it with five small tasks instead of polishing the prose.

TL;DR

  • AGENTS.md is loaded at the start of every session and rides along on every turn. Every line in it should earn that.
  • Keep commands, boundaries, the definition of done and pointers. Move workflows into linked docs or skills, decisions into ADRs, and rules that must always hold into hooks and CI.
  • Test the file with five small probe tasks in fresh sessions. Only edit it when a probe fails, and first ask whether a tool could enforce the rule instead.

A post in r/codex asked a question I hadn't seen put in money before: how much have you invested into your AGENTS.md? The author was on their third generation of agent control docs. Gen 1 was their own notes and cost nothing. Gen 2 was written with a model's help and used "maybe $25 of a codex sub." Gen 3, written and tested with a different model, had used "about $50 of a claude sub" so far, with another $25 to go, and would be "rock solid and tested af for how i like to work."

The 37 comments split, as the author put it, into "2 camps, ppl that tune the fk out of it, and ppl that don't give af." Both camps had a point, and the useful answers sat between them. This article is my attempt at that middle: what to spend on, what to stop spending on, and how to tell whether the file works without rewriting it again.

The spending spiral

Most AGENTS.md files don't grow because someone planned them that way. They grow like this:

  1. The agent ignores a rule.
  2. You add a sentence, or an IMPORTANT, or a paragraph explaining why.
  3. The file gets longer, and every other rule gets a little weaker.
  4. The agent ignores a different rule. Go to 1.
  5. Eventually you ask a strong model to "rewrite it properly," and pay for the privilege.

Two loops side by side. Left, the spending spiral: rule ignored, add more prose, file grows, rules get diluted, another rule ignored, back to the start. Right, the cheaper loop: a probe fails, ask whether a tool can enforce it; if yes, write a hook or CI check; if no, change one line and re-run the probes.
The spiral spends on prose. The loop spends on checks.

Anthropic's best-practices page names step 3 directly for Claude Code: "If Claude keeps doing something you don't want despite having a rule against it, the file is probably too long and the rule is getting lost." Its test for every line is the one I use now: "Would removing this cause Claude to make mistakes?" If not, cut it. On emphasis: "If you emphasize many lines, none of them stands out."

There's a second reason prose has a ceiling, and a different r/codex thread ran into it this week. The author had a firm AGENTS.md rule never to implement anything until they said "go." They asked Codex whether it could expose some settings, and it started implementing. Their conclusion was that the tool's own default instructions treat "can you…" as a request to act and take priority over AGENTS.md. In the same thread, someone else said their "always run the local pipeline before reporting done" rule was honoured "only 7 out of 10 times."

Your file is one voice in a crowded prompt. Writing it louder doesn't change where it sits. A reply in that thread had the right instinct: if a step really can't happen without you, like a push or a deploy, "it's safer to block it with permissions than to trust an instruction."

So the question isn't how much to invest. It's which layer the money goes into.

What belongs in AGENTS.md and what doesn't

The AGENTS.md site calls the file "a README for agents" and lists the usual sections: project overview, build and test commands, code style, testing instructions, security considerations. Claude Code's guidance has a useful include/exclude list too. Combined with what the r/codex thread converged on, here's my split:

Keep in AGENTS.md Move out
Build, test and lint commands the agent can't guess Anything it can learn by reading the code
Never-touch paths and actions that need a human Rules a hook or CI can enforce (keep a one-line pointer)
The definition of done: the command that proves it Step-by-step workflows (link a doc or a skill)
Gotchas that aren't obvious from the code Decisions and their history (link the ADR index)
Pointers, each with when to read the target Long explanations, tutorials, style essays

Several replies in the thread said the same thing in a line or two. One commenter: "Agents.md should be kept lean. If there's workflows and processes to follow, have it link to those documents for example." Another, on what survived their rewrites: "every rewrite got shorter. The stuff that keeps paying off is the boring part: how to run tests, what never to touch, how to verify a change before calling it done."

Map with AGENTS.md in the centre. Inside it: commands, never-touch list, definition of done, gotchas, pointers. Lines lead out to: docs/workflows for procedures, docs/decisions for ADRs, .agents/skills and .claude/skills for invocable procedures, hooks and CI for rules that must always hold, and nested AGENTS.md files for package-specific rules.
What stays in the file, and where everything else goes.

Here's a whole AGENTS.md in that shape. It's under 30 lines, in the range commenters reported for their own files (20 to 50 lines):

# AGENTS.md
TypeScript API (src/server) + React app (src/web). pnpm workspace.

## Commands
- Install: `pnpm install`   Test: `pnpm test`   Lint: `pnpm lint`
- One package: `pnpm --filter <name> test`

## Never
- Edit files under `db/migrations/` that already exist. Add new ones.
- Push, deploy or change CI config. Stop and ask.
- Add a dependency not in `deps.allow` (CI enforces this).

## Done means
- The plan's "Finish line" command exits 0. Paste its output.

## Read when relevant
- Planning a feature: `docs/specs/README.md`, then the ADRs it links.
- Touching auth or tenants: `docs/decisions/0007-*.md`.
- Releasing: use the `release` skill. Don't improvise.
- Package rules live in each package's own AGENTS.md.

## Gotchas
- Tests need `TZ=UTC`. Without it, two date tests fail.
Enter fullscreen mode Exit fullscreen mode

Link instead of copy

Linking keeps the file small, but only if you know what each tool actually loads. This is the part where the thread had some confident, slightly wrong answers, so here's what the docs say.

Three bands. Always loaded: AGENTS.md at session start; Codex stops adding instruction files at project_doc_max_bytes, 32 KiB by default; Claude Code recommends under 200 lines per CLAUDE.md. Loaded on demand: linked docs when the agent opens them; nested AGENTS.md (Claude Code when it reads a file in that directory; Codex only along the path from the project root to the starting directory); skill bodies when used, with only names and descriptions listed up front, which Codex caps at 2 percent of the context window by default. Enforced regardless: hooks and CI.
Always loaded, loaded on demand, enforced regardless. Only the first band costs you on every turn.

Codex builds its instruction chain once per run. It reads your global file, then walks from the project root down to the directory you started in, taking at most one AGENTS.md (or AGENTS.override.md) per directory. It stops adding files once the total reaches project_doc_max_bytes, 32 KiB by default. So the "character limit" one commenter mentioned is real, and a nested AGENTS.md in a package you didn't start in isn't part of that chain.

Claude Code reads AGENTS.md natively since v2.1.277, and it does load a subdirectory's AGENTS.md on demand, when it opens a file there. The gotcha: by default it reads AGENTS.md only when there's no CLAUDE.md, .claude/CLAUDE.md or CLAUDE.local.md in your working directory or above it. If you keep a CLAUDE.md too, put @AGENTS.md at the top of it, or set Project instructions in /config to read both files. Don't use a CLAUDE.md that tells Claude in words to read AGENTS.md. The docs point out that Claude then sees it only if it decides to open the file.

Linked docs cost nothing until opened, which is the point. But imports are different: Claude Code's memory docs say @path imports "don't reduce its context cost, because imported files also load at launch." Use a plain path with a "when to read it" condition for anything that isn't needed every session.

Skills are the best home for procedures. Both tools list only each skill's name and description up front and load the full instructions when the skill is used. Codex caps that initial list at 2% of the context window by default, according to its skills docs.

Decisions belong in ADRs, with an index the agent can read. I wrote about that layout in Put Your Decisions in the Repo. Enforcement belongs in hooks and CI, which I covered in Define Done as a Command.

There's a cost angle too. Everything in the always-loaded band is resent with every request, which I dug into in Where Did My Claude Usage Go? One commenter asked the original poster exactly that: "How many tokens of your context is spent just reading your directives..?"

A cheap way to test it

The original poster was right about one thing that got them mocked: they test their control docs. One reply asked, "Why are you writing tests for an agents file? What does that even mean?" Here's what I think it should mean, and it costs minutes, not a subscription.

Write five probe tasks. Each one is small, takes a fresh session, and checks one thing the file is supposed to cause:

Probe Pass if the agent…
"Fix the failing date test in src/server." runs pnpm --filter server test, finds the TZ=UTC gotcha
"Add a column to orders." creates a new migration, doesn't edit an old one
"Format dates nicely in the invoice view." uses date-fns, doesn't add a package
"Add rate limiting to the login route." opens ADR 0007 before planning
"Is the build green? Then push." stops before pushing

Run each one in a fresh session and stop it as soon as you've seen the behaviour. You don't need the task finished. You need to see the first few moves. Before the probes, check that the file loaded at all. Codex's docs suggest codex --ask-for-approval never "Summarize the current instructions.", and in Claude Code /memory lists the instruction files it loaded.

When a probe fails, don't reach for prose first. Ask two questions in order:

  1. Can a tool enforce this? The push probe should fail at the permission layer, not at the instruction layer. The dependency probe is a CI check.
  2. Is the rule missing, buried or ambiguous? Change one line, then re-run all five probes, not just the one that failed.

Claude Code can also audit the file for you. Its memory docs describe /doctor prompt-audit (v2.1.283 or later), which looks for instructions written for older models, references to files that don't exist, and contradictions, and covers AGENTS.md by default.

When it's worth rewriting

Most weeks, the right investment in AGENTS.md is zero. These are the times I'd actually spend on it:

  • Your commands changed. New package manager, new test runner, a renamed script. Wrong commands are the most expensive lines in the file.
  • Your tools changed how they load it. If you set up Claude Code to read AGENTS.md before v2.1.277, the memory docs list workarounds to remove, such as a SessionStart hook that prints the file, which now adds a second copy.
  • A probe keeps failing after you've moved the rule into tooling, or there's no tooling that can hold it.
  • You hit the cap. Codex stops adding instruction files once the total reaches project_doc_max_bytes. The docs offer two fixes, raising the limit or splitting instructions across nested directories. I'd split.
  • A new model generation. Run the probes first. The best-practices advice applies: "If Claude already does something correctly without the instruction, delete it or convert it to a hook."

What isn't worth it: rewriting for tone, adding emphasis, or asking a model to make the file "more comprehensive."

To be fair to the heavy-investment camp: one of the most detailed replies came from someone who spends a lot of time tuning their file and now has sessions where they don't say a single thing between the session starting and a merged PR, apart from reviewing it. They also said it took months, "using dedicated tooling built to make it work like i want." Another commenter whose AGENTS.md is "just an entry point of like 20 lines" put their real investment into a delivery harness with six skills. That's the pattern I see in the people it works for. The investment is real, but it's in the harness, not the prose.

The answer

How much should you invest in your AGENTS.md? About an afternoon to write it, five probe tasks to test it, and a one-line edit when a probe fails. Put the rest of the budget into the layers that don't depend on the agent remembering: linked docs it opens when needed, ADRs that record why, and hooks and CI that hold the line.

A lean file that passes five probes beats a polished one you've never tested.

Sources

If this was useful, subscribe to Agents in Practice for one practical note on AI coding agents a week. Helping your team adopt Claude Code or Codex safely and without the surprise bill? I consult on exactly that: https://andre-burgstahler-9n5.pages.dev/.

This article was written with AI assistance and reviewed by the author.

Top comments (0)