DEV Community

Prince Jain
Prince Jain

Posted on

I audited my AI agent stack: 92,413 tokens burned before the first message

A while back there was a thread on r/mcp: "Running 15+ MCP servers can burn 60,000 tokens before your agent reads a message." Everyone in the comments was fixing it piecemeal: lazy-loading hacks, a homemade get-tool-help tool, grepping /context output by hand.

Nobody had built the diagnostic. So I did.

What it measures

ctxvitals launches each stdio MCP server in your config, sends a real initialize + tools/list, and counts the actual schema tokens each one injects into context. Then it does the same for skills, rules files, commands, and hooks across Claude Code, Cursor, Windsurf, Copilot, Codex, and Devin configs.

My results were bad

metric value
skills installed 901
skill description tokens (always loaded) 39,402
MCP servers 15
tokens burned per turn 92,413
grade F

The embarrassing part wasn't the number. It was why:

  • the same MCP server was configured under 3 different names, paying schema cost 3 times
  • a plaintext API key sitting in an env block
  • one server pointing at a binary that doesn't exist anymore

And after shipping per-tool rankings today (a Reddit suggestion), I found out claude-code's Read tool alone costs 7,487 tokens of schema. One tool.

It also checks security

  • hardcoded secrets in configs and skill files
  • exfil patterns (credential stores + network calls in the same file)
  • curl | sh installers
  • prompt-injection phrases
  • unpinned npx/uvx packages
  • duplicate registrations resolving to different binaries, which means which one answers your tool calls then depends on load order. supply-chain roulette.

Run it

npx ctxvitals
Enter fullscreen mode Exit fullscreen mode

Zero dependencies, Node 18+. --html report.html for a shareable report, --json for piping, --no-probe if you don't want it launching servers. There's also a Claude skill so your agent can audit itself.

GitHub: https://github.com/princejain756/ctxvitals

Honest question: what does your stack burn? I've seen 92k (mine, embarrassing). Someone tell me the ceiling.

Top comments (2)

Collapse
 
max_quimby profile image
Max Quimby •

The duplicate-registration finding is the one I'd flag hardest, because it's not just wasted tokens — it's a correctness bug. When the same server answers under three names resolving to different binaries, which one services a tool call depends on load order, and that's silent until it isn't. "Supply-chain roulette" is the right phrase.

On your honest question about the ceiling: the number that actually hurt me wasn't the always-loaded schema, it was the per-turn re-read. A fixed preamble that gets re-sent every single turn compounds with conversation length in a way a one-time skill description never does — I've watched ~70k of fixed context get paid on every turn before any real work happened, and a no-op early-exit still burned the full preamble.

So the axis I'd add to ctxvitals if it isn't there already: always-loaded vs lazy-loadable vs re-paid-per-turn. The last bucket is the expensive one and the easiest to miss, because /context shows you a snapshot, not the integral over a session. Does it currently distinguish those three?

Collapse
 
arhancanli profile image
Arhan Canli •

Subtracting your skills line (39,402) from 92,413 leaves at most 53,011 tokens for the 15 MCP servers plus rules, commands and hooks, so the servers are roughly 3.5k each on average, and a single Read at 7,487 is more than two of those. That makes me think per-tool ranking matters more than per-server totals: a handful of tools probably account for most of it.

One thing a probe like this can't see is servers that keep the full list out of tools/list. I measured our own server, canli-mcp, at 1,109 tokens for 306 tools by exposing three front tools (search, describe, call) instead of 306 schemas, versus 34,508 tokens when all are listed. Does ctxvitals count tools/list only, or would it also flag a server whose cost moves into later find/describe calls? Worth a note in the grade either way, since the per-turn number and the on-demand number differ.

Disclosure: I built canli-mcp (free, MIT, no key): npx -y canli-mcp, github.com/arhancanli/canlicapital.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.