Your MCP agent re-sends 7,000 tokens of tool schemas on every turn

I wired three MCP servers into my agent, then did something I should have done first: I counted what they cost.

One filesystem server, 28 tools: about 7,000 tokens. Just the tool definitions. Before a system prompt, before any conversation, before the user has typed anything.

Then I noticed the part that actually matters.

It is not a setup cost

An LLM API call is stateless. There is no server-side memory of a conversation — you re-send everything, every time. That includes the tool definitions.

So the 7,000 tokens are not paid once when the agent connects. They are paid on turn one, and turn two, and turn twenty. A 40-turn session with three MCP servers connected pays for tool schemas forty times.

Meanwhile, on any given turn, the agent calls one tool. Maybe two.

You are paying full price for 26 tools that were never in play.

Count your own

Do not take my number. Capture your own tools/list response and measure it:

import json, tiktoken

enc = tiktoken.get_encoding("cl100k_base")
tools = json.load(open("tools.json"))["result"]["tools"]

payload = json.dumps(tools, separators=(",", ":"))
print(f"{len(tools)} tools")
print(f"{len(payload):,} bytes")
print(f"{len(enc.encode(payload)):,} tokens per turn")

Most people I have shown this to guess low by a factor of three. The schemas are bigger than they feel, because nobody reads them — the client fetches them and hands them straight to the model.

What is actually in there

Here is one tool from my fixture, verbatim, as a real MCP server would send it:

{
  "name": "fs_write_file",
  "description": "Create a new file or completely overwrite an existing file with new content. Use with caution as it will overwrite existing files without warning. Handles text content with proper encoding.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "path":    { "type": "string",  "description": "Destination path of the file to write." },
      "content": { "type": "string",  "description": "Full textual content to write into the file." },
      "encoding":{ "type": "string",  "enum": ["utf-8","utf-16","latin-1"], "default": "utf-8" },
      "createDirectories": { "type": "boolean", "default": false },
      "mode":    { "type": "string",  "description": "POSIX file mode in octal." }
    },
    "required": ["path","content"],
    "additionalProperties": false
  }
}

605 bytes. And that is a small one — the GitHub and Slack tools in my set run past 900.

Now split that into two piles.

The model needs the name and a rough sense of what the tool does in order to decide to call it. It needs the full parameter schema only once it has already decided — to actually format the call.

Almost every tool in your list is in the first situation on any given turn. You are shipping pile two for all of them anyway.

The idea: only pay full price for what is about to be used

That is the whole thing. Keep the tools the agent is plausibly about to call at full fidelity. Compress the rest down to what is needed to name them.

The hard part is the word "plausibly".

Predicting the next tool without a model

The obvious reach is embeddings: index the tool descriptions, embed the conversation, retrieve top-K. I think that is the wrong instrument here, for two reasons.

First, tool names are not prose. They are fs_read_file, git_commit, github_create_pull_request — dense identifiers with maybe a sentence of docs. Embeddings smooth exactly the signal you want sharp.

Second, you are on the hot path of tools/list. Every millisecond you add is a millisecond in front of the user, on every turn.

There is a much cheaper signal sitting in plain sight: agent workflows are extremely repetitive.

git_status → git_diff → git_commit. fs_read_file → fs_edit_file → fs_read_file. Over and over.

So build a graph. Every time a tool executes, draw an edge from the previously executed tool to this one and increment its weight. After a handful of turns you can read off a probability directly:

P(next = B | last = A) = weight(A → B) / total_out_weight(A)

That is a first-order Markov chain over tool calls, and it is embarrassingly cheap — a map lookup and a division. In my implementation it accounts for the largest single weight in the ranking.

Notice the graph must not be acyclic. Real workflows loop; read → edit → read is signal, not noise. Every "DAG" framing of this problem I have seen throws away information.

Four more signals fill the gaps, each normalised to [0,1] and summed with a weight:

• Second-order transition — same probability conditioned on the tool before last, which catches A → B → A alternation.
• Recency — 2^(-steps_since_last_use / halflife).
• Frequency — the tool's share of all executions this session.
• Lexical overlap — a decaying bag of terms harvested from live traffic (tool arguments, tool results, prompt requests) matched against each tool's name and description.

That last one needs one detail to work at all. Tool names are snake_case, so you have to split on _ and on camelCase boundaries before matching. fs_read_file becomes read + file, which then matches an agent that has been talking about reading a file. Skip that split and the lexical signal is dead on arrival — I know because mine was, for an afternoon.

Total cost of ranking 28 tools: 76 microseconds. For 500 tools, 1.3 ms.

Compression that stays valid

For the tools that lose, the compressed form is:

{
  "description": "Create a new file or completely overwrite an existing file with new content. Use with…",
  "inputSchema": { "type": "object" },
  "name": "fs_write_file"
}

605 bytes → 152.

The one decision worth explaining is inputSchema. The tempting move is to delete it. Don't — an absent inputSchema is invalid per the MCP schema, and strict clients will reject the tool outright. {"type":"object"} is a legal, maximally permissive object schema that satisfies every validator for 19 bytes.

What happens when the prediction is wrong

This is the part that decides whether the whole idea is usable or a footgun.

A compressed tool keeps its name. The model can still see that it exists and can still decide to call it — it just does not have the parameter schema in front of it. So when the prediction misses, the model reaches for the tool anyway.

At that moment the proxy:

1. marks the tool as revealed and forwards the call untouched;
2. emits notifications/tools/list_changed to the client;
3. the client re-requests tools/list, and now that tool comes back at full fidelity.

Worst case is one extra round-trip. Nothing is ever unrecoverable.

One protocol detail: that notification is only sent if the upstream server declared capabilities.tools.listChanged during initialize. Inventing a capability the server never advertised is how you get a client that ignores you, or worse, errors. If the capability is missing, compression still applies and the reveal lands on the client's next natural tools/list.

Actually removing tools from the list is possible in my implementation, but it is off by default. A compressed tool is recoverable; a deleted one is not. That asymmetry should decide the default, and in most tools of this kind it doesn't.

The trap that eats your savings

Here is the part I did not anticipate, and the reason I think most naive implementations of this idea lose money.

Both Anthropic and OpenAI cache prompt prefixes. The cache key is an exact byte prefix. Tool definitions sit near the front of the prompt, which makes them prime cached content.

Now imagine a pruner that re-ranks and re-orders the tool list on every turn, putting the most relevant tool first. It feels right. It is a disaster: every turn produces a different prefix, so every turn is a cache miss. You save 50% of the tool tokens and lose the discount on everything that follows them.

So the output has to be deterministic, and it has to be positionally stable:

• Preserve upstream ordering. Only tool contents change, never their positions. The prefix stays identical up to the first tool whose tier actually changed.
• Forward kept tools as the exact upstream bytes. Never decode and re-encode a tool you are not modifying — you will normalise something and not notice.
• Build compressed objects with fixed key order. Hash-map iteration order is a determinism bug waiting to happen.
• Break ranking ties on original position, not on whatever the sort happened to produce.

I have a test that replays the same event sequence 50 times from a cold state and asserts byte equality of the output. It is the single most valuable test in the repo.

A bug I did not see coming

While verifying that pass-through was truly byte-identical, a test failed on a diff I could not explain at first:

 got <email>."}},"required":["repo","message"]
want <email>."}},"required":["repo","message"]

Go's encoding/json HTML-escapes by default. <, > and & become six-byte \uXXXX sequences. Every schema containing something like Name <email> was being silently inflated — by the tool whose entire job is to shrink schemas.

The fix is one line:

enc := json.NewEncoder(&buf)
enc.SetEscapeHTML(false)

Worth checking in any Go proxy that re-encodes JSON it did not author. You are not just wasting bytes, you are modifying a payload you promised to forward transparently.

Numbers

Apple M5, Go 1.26, 28-tool fixture of realistic MCP schemas:

Metric	Value
Tokens per tools/list	7,079 → 3,074 (-56.6%)
Bytes per tools/list	26,283 → 10,943 (-58.4%)
500-tool list	-78%
Ranking latency, 28 tools	76 µs mean, 0.50 ms worst
Full frame rewrite (decode + prune + re-encode)	0.44 ms
Retained state after 20k tool calls	2.1 MiB

Two honesty notes, because both invite a fair challenge:

Token counts are estimates. The token figures come from a built-in heuristic — word runs at roughly 4 chars per token, punctuation runs at roughly 2 — not from a real BPE tokenizer, because I did not want to ship a vocabulary file. It is checked against cl100k_base reference counts and stays within a 0.6×–1.8× band. Byte counts are exact. Both sides of the before/after ratio carry the same bias, so the reduction holds even where the absolute number drifts.

Cold start is weaker. The first tools/list of a session has no execution history, so it ranks on lexical signal alone and falls back to upstream order. It still roughly halves the payload — the compression does that on its own — but the selection gets meaningfully better after a few tool calls.

Try it on your own tool list

I packaged this as mcp-diet — a transparent stdio proxy, Go, no dependencies outside the standard library, MIT.

You can measure your own setup without installing anything into your agent:

go install github.com/albererinofigo-droid/mcp-diet/cmd/mcp-diet@latest
mcp-diet analyze your-tools-list.json

tools       28 (full 8, compressed 20, dropped 0)
bytes       26283 -> 10943  (-58.4%)
est tokens  7079 -> 3074  (-56.6%)
prune time  0.455 ms

To actually use it, wrap the server command you already run:

{
  "mcpServers": {
    "filesystem": {
      "command": "mcp-diet",
      "args": ["--server", "npx -y @modelcontextprotocol/server-filesystem /srv"]
    }
  }
}

Nothing else changes. Same protocol on both sides, and everything that is not a tools/list response is forwarded byte-for-byte.

Current limitation worth stating plainly: stdio only. SSE and streamable-HTTP transports are not implemented yet, though the pruning core is transport-agnostic and usable as a library.

Repo: https://github.com/albererinofigo-droid/mcp-diet

If you run the analyze command on your own setup, I would genuinely like to see the number. My guess is that most people are paying more than they think, on every single turn.