DEV Community: William Zhu

I measured what my AI coding assistant loads before I type

William Zhu — Wed, 15 Jul 2026 05:00:00 +0000

Open a project in Claude Code, Cursor, or Copilot and type your first prompt.

Before your request even reaches the model, the tool may already have loaded CLAUDE.md, AGENTS.md, .cursorrules, Copilot instructions, imported rule files, and scattered project notes.

So before you tell the agent what to do, the repository has already told it how to behave. That's usually fine. It's how these tools are meant to work.

...but it also makes the instruction layer easy to set and forget. That invisible context can run to thousands of tokens, and none of it shows up in the message you just typed.

So when an agent ignores a rule, the obvious conclusion is that the model failed. But what if the rule is buried under a few thousand tokens of instructions? Or the same policy lives in three files for three different tools, and the copies no longer agree?

I wanted to know how large that invisible layer had become in real projects. So I scanned 100 popular open-source repositories.

72 had AI instruction files. The median adopter carried just under 2,200 tokens of always-loaded instructions across the AI tools configured in the repository. One carried nearly 24,000.

But the most interesting number wasn't the largest file. Of the 55 repos with multiple instruction files, 15 had rules duplicated across files.

The invisible prompt wasn't just growing. It was forking.

The layer we rarely diff

AI instruction files are behavioral configuration. They change how an agent writes code, runs tests, structures commits, and names files. They live in Git, but we rarely give them the same discipline as the code around them. No size budget, no review that treats a rule change as a behavior change, no check that the copies agree.

They grow a section at a time. Rules get copied between tools. Old guidance outlives the project decision that motivated it. And that costs you twice:

Context. Always-loaded instructions occupy part of the effective context before your task, your code, or your error output even arrives.
Attention. The one rule that matters has to compete with a few thousand tokens of other instructions and boilerplate.

And the whole thing is invisible. Most people have never seen their config layer as a single number.

What 100 real repos look like

I ran a linter over the AI configuration of 100 well-known open-source projects. It's a hand-picked list, deliberately skewed toward AI-native tooling (SDKs, agents, frameworks). Read this as "what popular, AI-forward projects look like," not a random sample of GitHub. The list is public if you want to reproduce it.

A few things stood out:

Nearly three-quarters ship AI instruction files. 72% had at least one. In this AI-forward sample, that makes instruction files common rather than exceptional, and over half (55%) carried more than one.
The median adopter carried 2,198.5 tokens of always-loaded instructions across the AI tools configured in the repository.
The heavy adopters carried a lot. Counting only always-loaded content (the rules set to load automatically, with path-scoped and on-demand files excluded), coder tops the sample at 23,870 tokens across 13 always-on files, 10 of them pulled in by @import. cline carries 14,673 across 9 always-loaded rule files, and bun 9,084, effectively all of it in a single CLAUDE.md. That's an order of magnitude past the median.
Half carried config for two or more tools side by side: CLAUDE.md, AGENTS.md, and Cursor rules in the same tree. Among the 55 repos with multiple instruction files, 15 had rules duplicated across files.

What the numbers do (and don't) mean

The footprint counts only always-loaded rules. I excluded files a tool attaches only for certain paths or pulls in on demand. That's why microsoft/vscode, with 34 instruction files, contributes only ~2,600 always-on tokens rather than the 30,000+ you'd get by adding every file it ships.

The combined footprint also sums the configurations for the different AI tools detected in the repository. Claude Code loads Claude's applicable instructions; Cursor loads Cursor's. The total shows how much always-on AI policy the repository is maintaining across its tools. The obvious objection: no single assistant reads all of it.

Well, more than half the time it does. In 58% of the repos with instruction files, one tool loads every single token of that combined number. Across 8 in 10, one assistant reads at least 70% of it. For about one in six, a single assistant reads only half to two-thirds.

Which puts that median back where it started. Roughly 2,200 tokens isn't an accounting total spread across a toolchain. More often than not, it's one assistant, reading all of it, before your first word.

Multiple instruction files are not automatically a problem, but they create opportunities for the same guidance to be copied between tools. A real example: documenso keeps a block of coding rules in both .cursorrules and AGENTS.md: four exact duplicates and one near-match. One identical rule in both files says: "never use 1 line if statements." Today that's harmless. But the policy now has two sources of truth. The day someone refines the rule in one file and forgets the other, the copies can diverge without anyone noticing. Multiply that across a 250-line file maintained in parallel for three tools, and drift stops being hypothetical.

About that duplication count: ContextGuard originally flagged 18 repos. The three beyond the manually verified 15 are borderline by design: a vendored skill copied at two paths, a line shared by two project templates, a deliberate boilerplate header.

I also excluded MCP overhead because the meaningful context cost comes from runtime tool schemas, not the static config file itself.

What I built

I turned the linter into a small VS Code extension, ContextGuard, that gives this layer the visibility we already expect from code:

A status-bar counter for your combined instruction footprint (the always-loaded tokens summed across your tools), with an estimated monthly cost on hover.
A preview of the detected instruction context, in load order, with imports expanded, always-loaded rules separated from scoped/on-demand ones, and secrets redacted.
Warnings for oversized files, repeated rules within a file, broken @import references, and secret-like values that need review.

A per-assistant view, showing what each tool actually loads in a single request, is on the roadmap.

It's local-first and deterministic. Every finding is tied to a file and line, there are no AI calls, and no telemetry. On the free tier it makes no network calls at all.

Sometimes the model really does ignore a perfectly clear instruction. But sometimes we've handed it a few thousand tokens of accumulated policy, copied the important rule into three places, and then expected it to know which sentence mattered most.

If instruction files can change an agent's behavior, they are part of the software. I think we should start treating them that way: review them, keep an eye on their size, and avoid maintaining three slightly different copies of the same rule.

Before blaming the agent, inspect the prompt you didn't type.

→ ContextGuard on the VS Code Marketplace · ContextGuard on the Open VSX Registry · the repo list I scanned

How many AI instruction files are sitting in your main repo, and when did anyone last review them together?

GitHub Copilot went usage-based, so I built a linter for AI context files

William Zhu — Tue, 30 Jun 2026 15:00:00 +0000

As I'm sure a lot of you saw, GitHub Copilot recently moved from request-based billing to usage-based billing :(

Copilot is the main AI coding tool I use day-to-day, so the change made me look a lot harder at how much context I was actually feeding these tools. No more casually attaching half the repo and saying "loop until done" without wondering what that costs.

...not that I ever did that...

But the more I looked into it, the more I realized the token count was only part of the problem.

The part I didn't expect

The weirder issue was that I didn't really have visibility into what gets loaded into the AI assistant's context.

For me, that meant Copilot instruction files. But depending on your stack, it might be CLAUDE.md, AGENTS.md, MCP configs, old AI notes nobody cleaned up, or other agent-related config sitting around the repo.

When I pointed my extension at a real public project, the open-source Cline repo, it found around 29k tokens spread across its AI instruction and configuration files.

That doesn't mean Cline loads all 29,000 tokens into every request. It's a combined footprint across different tools, each of which may use a different subset of those files.

And honestly, I couldn't have told you what half of it was.

That layer can quietly become messy: stale notes, broken references, files you forgot existed, or even secrets that really should not be sitting in AI-visible config.

So I built ContextGuard

ContextGuard is a VS Code extension for inspecting and linting the AI config/instruction layer in your workspace.

It's more than a token counter. It's built to answer:

What is my assistant already seeing, and is any of it weird?

What it does:

Shows your auto-loaded token baseline in the status bar
Previews what config/instruction files are being loaded, in order
Flags broken import references and stale instruction files
Flags likely secrets in AI-visible files, with exact line numbers
Lets you bundle selected files/context for web agents
Optionally estimates monthly cost from that baseline, with configurable usage assumptions

It's free, runs fully local, and makes no network calls.

The honest caveats

Big disclaimer: the secret detection is heuristic, so it's a safety net, not a guarantee. It'll catch some obvious "oh no, that's a key in my CLAUDE.md" cases, but please do not treat it as airtight secret scanning.

It's also brand new and my first real VS Code extension, so I'd genuinely love feedback. If the UX is confusing, if it misflags something, or if it doesn't match how your AI setup works, please tell me and I'll patch what I can.

I'm sharing it here partly because I'm curious whether other people's AI context baselines look as weird as mine did.

Try it

If you do try it, I'm honestly curious what your baseline comes out to. I have a feeling some of these are bigger than people expect.

Marketplace: https://marketplace.visualstudio.com/items?itemName=bearinblue.contextguard

Website: https://bearinblue.github.io/contextguard/

The GIF is from the open-source Cline repo, which I used because it's a public project people can inspect. The secret example is from a test repo I made with fake keys. Not a real leak, thankfully.

Thanks for reading, and happy Fourth to everyone celebrating!