Stop Paying for Tools Your AI Isn't Using
Three recent results point at the same costly habit:
Anthropic reports an 85% token reduction from its Tool Search Tool, which discovers tools on demand while keeping the full library reachable.
An arXiv paper shows 8.9x faster tool selection and 23.8x lower token usage by loading tools on demand instead of dumping every schema into context, at a scale past 3,000 tools.
A second paper reports 31 to 42% fewer GPU-hours simply by routing on estimated total token budget.
Different layers. Same diagnosis: you are still paying a flat rate for capacity the current turn does not need.
That diagnosis is now consensus. What is still rare, and far more useful to a manager, is a number you can generate yourself, on your own install, with one command.
The real cost is not "AI." It is idle schema.
An MCP host that serializes every tool's schema into context on every turn creates two separate bills:
- A fixed per-turn token tax for tools you did not call.
- Dispatch dilution: the model must choose correctly from a larger and larger menu.
Retrieval-based loading attacks both. Keep a tiny resident surface. Describe the catalog only when asked. Dispatch by name.
What we actually shipped
21 June 2026 (v1.108.66)
A three-tool front door in front of the entire catalog:
-
order(action, args): dispatch any catalog action by name -
menu(query?): search and browse the catalog as compact rows -
route(task, ...): map natural language to the best action (and optionally execute it)
No capability removed. Every action remains one call away.
Safety boundary: order is read-only at the edge. State-changing or file-write actions require an explicit flag or are refused outright.
9 July 2026 (v1.108.115)
The collapsed surface became the default for brand-new installs. Existing installs keep their previous surface, enforced by construction rather than by hope.
21 July 2026 (v1.108.154)
The part that actually matters for cost conversations:
jcodemunch-mcp surface
Read-only. Instant. No session required. It reports:
- visible tools vs. full catalog
- estimated schema tokens for the active surface
- tokens avoided
- the heaviest schemas
One real install (v1.108.199, measured today):
6 of 94 tools resident, 1,038 of 25,801 estimated schema tokens, 24,763 avoided.
That measures resident schema cost only, so it is not comparable to the 85% or the 23.8x above, which measure total token usage. Compare it to your own full run instead.
That is not a paper number. It is the number that appears on your machine when you run the command.
Two different meters, deliberately
surface is an advisory receipt. It uses a bytes/4 estimate so it stays instant and dependency-free.
The CI schema-budget gate is a different instrument: tiktoken on a frozen baseline with a 5% drift tolerance. Its job is to stop schema bloat from shipping, not to inform a human.
They will never match digit-for-digit. That is intentional. A receipt that pretends to be a tokenizer count is worse than one that labels its own estimator and stays cheap.
Why this is the interesting number
An 85% figure in a blog post is something you accept.
A figure generated on your own repo, your own config, and your own tool surface is something you can take into a budget meeting.
The general discipline is simple: if you measure one side of a comparison live and leave the other side as a constant in documentation, the comparison rots, usually in your favor. Nobody audits a flattering constant.
We fixed our own README the same way. Absolute numbers went stale as the catalog grew; the multiple happened to survive. The line now names the version, states that bytes/4 is an estimate, and points at the command instead of a bare digit.
A constant in prose has no way to notice it is wrong.
A command re-derives itself every time someone runs it.
The broader pattern
Retrieval shrinks what you load to answer a question.
The front door shrinks what you pay before you have asked one.
The field is converging on the second lever. That is progress.
What should converge next is shipping the meter alongside the optimization.
If your tool-loading change saves 85%, put a command in the package that tells the next person what it actually saved on their install.
They will believe that number, because they generated it.
Run it:
jcodemunch-mcp surface
The interesting number is no longer ours...
-jjg
Top comments (0)