DEV Community

Cover image for State of AI Instruction Quality: A Third of Repos Shout
Gábor Mészáros Subscriber for Reporails

Posted on Originally published at reporails.com

State of AI Instruction Quality: A Third of Repos Shout

IMPORTANT. CRITICAL. You MUST. NEVER.

A third of the repos in the Q3 report, 3,319 of the 9,784 with an agent setup, write at least one word like these in capitals in their instruction files, 36,482 times between them. It's the most common habit we counted, and the rest of the list reads like a prompt-writing history: emoji, "please", "think step by step", a warning about hallucinating.

Each habit travels with a particular kind of setup, and some with particular findings.

Share of 9,784 repos with each habit in their instruction files: ALL-CAPS emphasis 33.9%, emoji 15.8%,

Who has them

The habits live in the big setups. The median repo in the report has 2 instruction files. For repos with one of these habits, the median runs from 5 files to 14.

Repos with… Median instruction files With an agent hook
All repos 2 7.6%
ALL-CAPS 5 13.3%
"Please" 7 13.1%
Emoji 8 15.6%
A destructive-command rule 10 20.0%
"Hallucinate" 11 21.1%
TODO, TBD, FIXME 12 15.6%
"Step by step" 14 17.9%

Part of that is arithmetic. More files mean more text, and more chances to contain any given word. They have more of everything else too. Repos with these habits are two to four times as likely to have sub-agents, and more of them write for more than one agent.

ALL-CAPS: a third of repos

MUST is the most common capitalized word, then CRITICAL, NEVER and ALWAYS. It's least common in CLAUDE.md and AGENTS.md main files, where about 22% of setups have it, and most common in Gemini, Cursor and Copilot ones, at 31% to 35%.

Some of it is RFC-style MUST in a spec, a convention with a defined meaning. Much of it reads like this line from a popular project's Copilot review instructions:

If test coverage is sufficient, DO NOT COMMENT.

Repos that write in capitals are firmer in general. They carry fewer hedged instructions ("should", "try to"), 4.7 per 100 against 5.8, and fewer or the same at every repo size. More of their headings double as instructions, like ## Always Run Tests Before Pushing, at 5.4 per 100 against 3.4. And despite NEVER and DO NOT, they write slightly fewer prohibitions, 16.4% of their instructions against 18.4%. The most common of those capitals, MUST, asks for something.

Anthropic's and OpenAI's prompting guides both say newer models don't need the capitals. Anthropic's says its newer models are "more responsive to the system prompt" and may overtrigger on prompts written to push older ones: "The fix is to dial back any aggressive language. Where you might have said 'CRITICAL: You MUST use this tool when...', you can use more normal prompting like 'Use this tool when...'." OpenAI's GPT-5 guide has the same example from the other side. A line from Cursor's prompt, "Be THOROUGH when gathering information. Make sure you have the FULL picture before replying.", worked with older models and turned out "counterproductive with GPT-5, which is already naturally introspective and proactive at gathering context."

Emoji, "please", and thinking out loud

1,541 repos (15.8%) use emoji in their instruction files, 29,311 times in total. Like the ALL-CAPS repos, more of their headings double as instructions.

839 repos (8.6%) say "please" to their agent, 2,051 times. In small setups, politeness travels with softer instructions. In the smallest quarter of repos, "please" repos carry twice as many hedged instructions as the others, 8.7 per 100 against 4.3. In the largest setups the difference mostly disappears.

798 repos (8.2%) write "step by step", "think hard" or ultrathink. Almost all of it is "step by step", which covers both asking the agent to reason and describing a procedure in steps. Where it asks the agent to reason, reasoning models no longer need it. OpenAI's guide for them: "Since these models perform reasoning internally, prompting them to 'think step by step' or 'explain your reasoning' is unnecessary." ultrathink and "think hard" turn up in a few dozen repos. In Claude Code, ultrathink in a prompt requests deeper reasoning for that turn, while "think" and "think hard" pass "through as ordinary prompt text".

TODOs and hallucinations

787 repos (8.0%) have TODO, TBD or FIXME in their instruction files. Some of those are rules about TODO comments in code. Others are the file's own unfinished business, shipped to the agent as-is.

246 repos (2.5%) mention hallucinating in their instruction files, some as a rule ("don't hallucinate") and some because the project itself works with model output. A bare "don't hallucinate" names nothing to check an answer against. Naming the source does: the schema file, the API docs page, the command that lists what exists.

Never force-push

360 repos (3.7%) write a rule against a destructive command, such as force-pushing, rm -rf, git reset --hard or dropping a table. The line from one machine-learning project is typical:

NEVER push to main. NEVER force push.

That's a rule the agent can follow or miss, and a force-push is the kind of miss you'd rather not find out about after the fact. A hook, a script the agent's harness runs at a set moment such as before a tool call, can stop the command before it runs. Claude Code can also refuse it with a deny rule in the project's .claude/settings.json. 72 of the 360 repos (20%) ship an agent hook of some kind, against 7.6% of all repos, and 6 deny a destructive command in a committed settings file. (Prompts steer, hooks enforce covers the difference.)

Of the 360 repos with a rule against a destructive command in their instruction files, 72 (20%) ship an agent hook of any kind, and 6 (1.7%) deny the command in a committed .claude/settings.json. For comparison, 7.6% of all 9,784 repos ship an agent hook.

The repos that write these rules are firmer elsewhere too, with 3.9 hedged instructions per 100 against 5.2, fewer at every repo size. They also carry more of one finding at every size: a "never" written more strongly than the permission that goes with it, most clearly in small and mid-size setups. The agent may then apply the prohibition more widely than intended, unless the case where the thing is allowed is written as clearly.

The settings files have habits of their own. Of 1,466 committed Claude Code settings files, 19 allow any shell command, and 3 turn permission prompts off entirely.

The 129,121-word CLAUDE.md

The median repo's largest main instruction file is 697 words. 57 repos have one over 10,000.

The largest runs to 129,121 words. It opens with the /init line, and its Architecture section alone is 107,019 words across 374 lines, close to 300 words a line. The repo is an autonomous coding-agent project, and its CLAUDE.md has grown with it, one architecture note at a time.

Largest main instruction file per repo, in words: 697 for the median repo, 129,121 for the largest, a CLAUDE.md whose Architecture section alone is 107,019 words across 374 lines. 57 repos have a main instruction file over 10,000 words.

Claude Code reads CLAUDE.md "at the start of every session", and its docs set a target of "under 200 lines per CLAUDE.md file. Longer files consume more context and reduce adherence." At 129,121 words, the file is well over 100,000 tokens before the agent has read a line of code.

Check yours

When a rule isn't being followed, look first at the places this post found: a capitalized line that fires everywhere, a heading doing an instruction's job, a "please" in front of a "maybe", a TODO that never got finished, a file long enough that the rule you care about is a few thousand words in.

The checker flags a TODO left in the file, a "never" with no matching permission and a hedged "try to" on the line where they sit, and it runs the checks behind this post's finding rates. Each run checks one agent's files; the second command switches to Copilot, and cursor, codex and antigravity (for GEMINI.md) work the same way.

npx @reporails/cli check -v
npx @reporails/cli check -v --agent copilot
Enter fullscreen mode Exit fullscreen mode

The capitals, the "please" and the file length are quick to count yourself. Swap in the files your agents read, such as GEMINI.md, .github/copilot-instructions.md or the files under .cursor/rules/:

grep -cwE "IMPORTANT|CRITICAL|MUST|NEVER|ALWAYS|DO NOT" CLAUDE.md AGENTS.md
grep -niE "please|step[- ]by[- ]step|ultrathink|TODO|FIXME|hallucinat" CLAUDE.md AGENTS.md
wc -w CLAUDE.md AGENTS.md
Enter fullscreen mode Exit fullscreen mode

From The State of AI Instruction Quality: Q3 2026: 10,000 of the most-starred GitHub repos, 15,237 agent setups, September 2026 snapshot.

Habits are counted in each repo's instruction files with code blocks removed; shares are of the 9,784 repos with a setup. ALL-CAPS counts the words IMPORTANT, CRITICAL, MUST, NEVER, ALWAYS, DO NOT, DON'T, YOU MUST, MANDATORY and REQUIRED in capitals. Finding rates count findings per 100 directives and constraints, in repos with at least 5; "every repo size" means each of four equal groups of repos by setup size.

None of the habits moves the overall score in a consistent direction: compared at similar repo size, the differences shift from one size group to the next.


I work on Reporails, deterministic diagnostics and governance for the instruction files, rules, and prompts that steer coding agents. It reads the instruction files your coding agent loads, from CLAUDE.md to Copilot and Cursor rules, measures which instructions couple to behavior, and points to the lines worth tightening.

Top comments (0)