I pulled 480 real AGENTS.md/CLAUDE.md files from 272 repos — and a regex bug taught me not to trust my own heuristics

Ruslan Griban — Wed, 12 Aug 2026 19:14:23 +0000

What it is

A corpus of real AGENTS.md / CLAUDE.md files, the instruction files coding agents like Claude Code, Copilot, and friends read before touching a repo, pulled verbatim from 272 production repos (React, Grafana, LangChain, Home Assistant, and 268 others). Not a curated list of links: the actual files, stored in full wherever the license permits it, greppable in one tree.

Repo: https://github.com/sattva2020/agents-md-in-the-wild

A few things that surprised me in the data

Explicit prohibitions ("never", "do not") show up in 60% of files, more common than build instructions (38%)
Only 23% mention anything about secrets, in files that hand an AI agent the run of the repo
Median file is 60 lines, much shorter than most advice threads suggest

The bug that taught me the most

One analysis pass tries to detect literal directory-tree diagrams, the kind with box-drawing characters or ASCII branch markers. First version was naive: look for tree-shaped characters on a line, count matches. Ran it across the corpus and got 69% of files "containing a directory map." That felt way too high, a directory tree in an instructions file is a specific, deliberate thing, not something 7 in 10 projects bother writing.

Turned out the regex was matching markdown tables. A row like | src/ | entry point | has a pipe and something path-shaped on the left, and my "line looks like a tree branch" check didn't care what came after the first slash. Fix: require a run of 3+ consecutive tree-shaped lines, and explicitly reject lines that parse as table rows before counting them. Dropped the number from 69% to 17%, which matches manual spot-checks.

Same story with the "mentions secrets" heuristic. First cut matched bare "token," which turned out to catch files talking about LLM context-window tokens, not API tokens. Had to tighten the pattern to require "api," "access," or similar nearby.

Lesson, twice over: a keyword or shape that looks specific to you is rarely as specific as it looks once you run it against real-world text at scale. 480 files was enough to expose both bugs, I doubt either would've shown up against the 5-10 examples I originally tested with.

License handling

Storage is tiered by SPDX license: files under a curated allowlist of about 26 redistributable licenses (MIT, Apache-2.0, BSD variants, GPL/LGPL/AGPL, CC0, etc.) are stored verbatim with provenance. Everything else, including repos with no declared license, gets metadata-only storage: headings and structure, no body text. Default is "all rights reserved" unless the license says otherwise; unlicensed files still get counted in the structural analysis, they just don't get their text redistributed.

Honesty about the state of it

It's meant to refresh weekly via a GitHub Action, full disclosure, that's currently stuck behind a billing issue on my GitHub account, so treat "weekly" as aspirational until I sort that out.