DEV Community

Cover image for How to structure CLAUDE.md, Skills and Agents
Hamid Shoja
Hamid Shoja

Posted on • Edited on

How to structure CLAUDE.md, Skills and Agents

Outdated agent docs silently break code

Hi friends

Here's a tip for when you're setting up Claude Code (or any coding agent) in a real codebase, this came in clutch for me when I discovered our agent docs were actively generating broken code.

The problem

Most projects end up with four kinds of instruction files:

  • CLAUDE.md / AGENTS.md files
  • skills (.claude/skills/...)
  • agent definitions (.claude/agents/...)
  • big reference docs (UI library API, internal tooling, etc.)

And the same knowledge gets copy-pasted into all of them. Each copy drifts separately. When I finally audited every claim in ours against the actual code, it was worse than I thought:

  • The styling docs showed a CSS-modules pattern that produces class names that never match the build config. Any agent following it ships broken styles.
  • The canonical test template set up mocks in beforeAll, but the test setup file restores all mocks after every test. So the mock silently dies after test one.
  • The same template rendered a data-fetching component without its required provider. Instant crash on render.
  • The "recommended" test helper was used by exactly 1 of 50 real test files.

The docs looked complete. They were confidently wrong. And every wrong doc costs you a multi-thousand-token debug loop when the agent hits it.

The rule of thumb

It's all about when it loads and who needs it:

Surface Loads Should own
CLAUDE.md / AGENTS.md always, every session rules true for every edit
Skill on demand, per task type deep how-to knowledge
Agent when delegated workflow and gates, not knowledge
Hook enforced by code rules that must never be skipped

Four questions to route any piece of content:

  1. Applies to every change in the folder? Then CLAUDE.md: import style, naming, commands, project structure. Keep it small, it pays token rent every single session.
  2. Only needed for one kind of task, but deep? Then a skill: styling mechanics, data-fetching patterns, testing recipes. Costs nothing while idle, only the description loads until it triggers.
  3. Is it a role or a process rather than knowledge? Then an agent: the workflow order, the definition of done, which skill to load. Keep agents thin, knowledge buried in an agent file is invisible to everyone else.
  4. Not acceptable to ever skip? Then a hook. Instructions can be rationalized away, code can't. Run your formatter from a hook, not from a sentence asking nicely.

The one rule that stops the rot

Every fact lives in exactly one file. Everything else links to it.

Our drift existed precisely because one test template was pasted into three places and evolved independently. After the cleanup:

CLAUDE.md / AGENTS.md  -> rules in 3 lines max + link to the skill
skill                  -> the deep patterns, citing real source files
agent                  -> workflow + "run the check command and observe it pass"
reference doc          -> single home for the heavy API reference
Enter fullscreen mode Exit fullscreen mode

So we can define the boundary in one sentence: if a pattern needs more than a few lines, it goes in the skill and CLAUDE.md gets a link.

Verify your docs like you verify code

This was the fun part. Docs claims are testable, so test them:

First, grep before you trust. Every example in your docs should exist in the codebase. Ours didn't:

# "best practice" from our docs vs reality
grep -rl "prefetchQuery" src/ | wc -l      # 0 uses
grep -rl "recommendedHelper" src/ | wc -l  # 1 of 50 test files
Enter fullscreen mode Exit fullscreen mode

Then run a retrieval test with a subagent. Give it ONLY the doc files, no repo access, and make it answer real implementation questions:

"Write a test for a component that fetches data."
"Which package does this component come from?"
"Where do global stores live?"
Enter fullscreen mode Exit fullscreen mode

Grade the answers against the codebase. If the agent following your docs writes code that would fail, your docs failed the test, fix them before merging. We caught every regression this way, before it cost anyone a debug session.

What it achieved

  • ~11% smaller context per typical feature task, leaner always-on footprint
  • a convention change now touches 1 file instead of 3
  • and the big one: the docs no longer generate broken styles and failing tests, each of those was a few thousand tokens of debugging every time an agent stepped on it

this example demonstrates that agent docs are a system with loading semantics, not a wiki: put each fact where its loading model matches, keep one home per fact, and test docs like code.

Hope that helped!
Hash

Top comments (8)

Collapse
 
ahmetozel profile image
Ahmet Özel

The rule that has held up for me: put things in the config file only if they change the agent's behaviour, and put everything else in the repo where it belongs. Instructions that restate what the code already shows just eat context and go stale. The failure mode is subtle too - once one line in that file is wrong, you cannot trust any of it, and the agent confidently follows the wrong instruction instead of reading the code. Short and verified beats thorough and rotting.

Collapse
 
hash01 profile image
Hamid Shoja

That trust point matches exactly what I saw. Once the styling doc was wrong, the agent didn't fall back to reading the code, it confidently shipped broken CSS from the doc. Wrong instructions are worse than no instructions.

Collapse
 
ahmetozel profile image
Ahmet Özel

That's the part I've stopped trying to fix with better wording. If a doc states a fact the code owns, it will eventually be wrong, and a confidently wrong instruction outranks the source for the agent every time. What helped me was writing config entries so they point instead of restate: "styling lives in tokens.css" rather than the actual values. Then the doc can only ever be stale about where something lives, and the agent still has to read the real thing. The other half is making staleness loud - a CI check that fails when a doc references a path or symbol that no longer exists. It misses semantic drift, but it catches most cases where the doc outlived the code it was describing.

Thread Thread
 
hash01 profile image
Hamid Shoja

Pointers over restatements is a clean version of this rule, but I think it trades one problem for two smaller ones.

One: your CI check catches dead pointers, not live ones whose meaning drifted. tokens.css still exists, half its values moved to a theme provider. Passing-but-wrong seems more dangerous than failing loudly!

Two: pointing gets expensive when the pointed-at thing is big. Component library: the agent needs ten lines of props per component, the pointer hands it the full implementation - hundreds of lines of hooks and styles each. A cheat-sheet restating just the public interfaces would be a fraction of the context. Does your rule allow that cheat-sheet, or does "never restate what the code owns" ban the one restatement that actually pays for itself?

Thread Thread
 
ahmetozel profile image
Ahmet Özel

I would allow that cheat-sheet, but only if it is generated from the source that owns the interface rather than maintained as a second manual truth. For a component library, the compact contract could be derived from types or schema, stored beside the API with a source hash, and regenerated or diffed in CI. The agent reads that contract first and opens implementation only when behavior is ambiguous. That keeps the context advantage while turning semantic drift into a detectable build artifact.

Collapse
 
skillselion profile image
Skillselion

The retrieval test is the part I'd steal. One extension worth adding: run a second version of it against the skill descriptions alone, not the bodies. Before a skill triggers, the description is the only thing the model can see, so routing accuracy is bounded by how well a sentence of frontmatter discriminates between neighboring skills. I hit a case where two skills both described themselves as testing best practices and the deeper one never loaded, because every task matched the shallower description first. Rewriting descriptions as trigger conditions (use when, covers, NOT for) fixed routing where rewriting the bodies had done nothing. Also agree on hooks over sentences, with one caveat: a hook that fails loudly teaches the agent the rule after one violation, while a hook that silently fixes output leaves the agent generating the wrong pattern forever.

Collapse
 
hash01 profile image
Hamid Shoja

Great points, I'll write some posts about them.

Collapse
 
allenrichard12 profile image
Allen Richard

I learned this the hard way too. Clean documentation isn't enough if it's outdated. I'd rather have one accurate source than multiple docs saying slightly different things.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.