Last month I ran 72 trials to settle what an MCP tool should return, because a maintainer would not take opinion for an answer. That left the other half open: before an agent does any work, how much of its context window have the attached servers eaten?
Vendors published numbers for this in 2026. I checked the six that get cited: exactly one is a real measurement study, StackOne's from 2026-03-31, which measured GitHub and Atlassian and worked through Cloudflare's code-mode case. Of the other five, one uses hypothetical unnamed servers, three contain no server-specific token study at all, and one I could not locate at any domain. So the published state of the art is a single snapshot, and it still shows March's numbers. A number that is not re-measured is a screenshot, and MCP servers change every few weeks. So I built a standing measurement, called loadline: 14 servers, methodology 0.2.0, one run dated 2026-08-18, monthly from here, every row reproducible from the repo with your own credentials. The front door is a stack calculator, not a table: the useful question is what my stack costs in the client I use.
Claude counts the same schema bytes around 64 percent higher than tiktoken
All three count one fixed string, the canonical serialization from methodology section 1.5. Same bytes, three counters.
| server | o200k_base | claude-opus-5 | premium |
|---|---|---|---|
| filesystem (14 tools) | 2,697 | 4,432 | +64.3% |
| linear (53 tools) | 17,780 | 29,026 | +63.3% |
| kubernetes (20 tools) | 4,430 | 7,552 | +70.5% |
| github (47 tools) | 59,084 | 86,843 | +47.0% |
Across the 12 rows that produced counts the premium runs from 47.0 percent (github) to 70.5 percent (kubernetes), median 64.1, with 8 of the 12 between 60 and 66. Gemini stays within 11 percent of o200k on every row, so this is not a bigger-models-count-more effect. It is specific to Claude's tokenizer on schema text.
Every MCP cost study I have seen counts with tiktoken, the only one of the three that runs offline and free. If your agent runs on Claude, those studies describe a context load roughly 60 percent smaller than the one you pay for on the same schemas. They are not wrong about the bytes. They count them for a different model than the one in your loop.
GitHub MCP, loaded naively, is 43 percent of a 200k window
The github row measures 47 tools at 59,084 tokens under o200k_base and 86,843 under claude-opus-5. That second figure is 43 percent of a 200k window, spent before the first user message.
Two qualifiers matter more than the number. First, 47 tools is what the server exposed to the classic personal access token this run used, and the spec lets a surface vary with the authorization presented, so the real GitHub surface is larger. Auth scope publishes per row, since that is the largest source of legitimate disagreement here.
Second, nobody on a tool-search client pays it. Claude Code has defaulted to progressive disclosure since January 2026, and on this surface that models out to roughly 4,300 to 6,800 tokens: a 500-token stub plus 3 to 5 tools at the measured 1,257-token average. A swing that size, driven entirely by the client, is why no row publishes one cost number. Three modes, always. And "modeled" is load-bearing there, which I get to below.
Three tools can cover a product line
Cloudflare's aggregate endpoint answers tools/list with three tools: docs (362 tokens), search (572), execute (660). Total 1,596 under o200k, 37 times smaller than github's naive load, and hygiene holds at B (85.19), so the compression did not come from deleting descriptions.
The qualifier: that is the aggregate endpoint measured as one server. Cloudflare also ships 16 product-scoped endpoints this run did not enumerate or sum.
Chrome DevTools ships 52 tools without the token bill you would expect
Chrome DevTools MCP, official from Google and newly measured this run, exposes 52 tools, second only to linear's 53, at 7,984 tokens under o200k: 153 tokens per tool, the leanest average in the corpus. Playwright's 24 tools cost 167 each, filesystem's 14 cost 192, linear's 53 cost 335. Hygiene holds at B (80.28), so, as with Cloudflare, the leanness is not from deleting descriptions. A server can carry more surface than all but one other in the corpus and still land near the cheap end of it.
Two of the 14 servers could not start, and one of them healed an hour later
fetch, the reference implementation, and postgres both failed to launch from a clean install when the run executed, 2026-08-18 at 15:38 UTC. Each declares an unbounded dependency on the MCP Python SDK, mcp>=1.1.3 for fetch and mcp[cli]>=1.5.0 for postgres, so both take whatever the resolver hands them. A resolve that lands on mcp 2.0.0 breaks both, in two different places.
fetch imports a symbol that 2.0.0 renamed:
ImportError: cannot import name 'McpError' from 'mcp.shared.exceptions'.
Did you mean: 'MCPError'?
postgres dies one import earlier, on a module 2.0.0 no longer ships at that path:
ModuleNotFoundError: No module named 'mcp.server.fastmcp'
Then, re-checking both rows about an hour later on the same machine with the resolver cache forced fresh, fetch came back. A clean resolve handed it mcp 1.29.0 instead of 2.0.0, and it started. Nothing upstream was withdrawn: 2.0.0 is still the latest release and is not yanked. The same command, on the same machine, on the same afternoon, resolved differently. postgres still fails.
So the fetch: unreachable row this post is built on went stale about an hour after it was written, and I am publishing it that way, labelled, with an entry in the corrections log, rather than quietly regenerating the dataset until it agrees with the prose. Next month's run will say whatever is true next month.
The more useful outcome is that it found a hole in my own instrument. The harness records whether the server package was pinned, but it never records which SDK version the resolve actually produced, so the artifact cannot explain why 15:38 and 16:45 disagree. That gets fixed before the next run.
I could have pinned around the breakage in a minute and decided not to. Pinning changes what "the fetch server" means here, publishing a constrained old version while implying it describes what you would install today. Server rot is part of the subject matter, so it is part of the dataset.
figma does not publish as a row at all this run: it failed gate 2 of the published selection rule, full surface enumerable under a free-tier credential, because its auth is OAuth-only and a reproducible harness has no credential to exercise that flow. It was excluded from the corpus rather than published as an auth row, and chrome-devtools was promoted to the freed slot.
Cheap and findable are different axes
Notion's redesigned server is cheap: 24 tools, 5,180 tokens under o200k. It also scores hygiene D (54.79) and a top-3 retrievability fraction of 0.5417, the lowest measured here, with an MRR of 0.4527. What sinks it: when_to_use_signal 0, disambiguation 0, parameter_descriptions 44.03. Context7, at 2 tools and 1,052 tokens, scores hygiene A (97.22), top-3 1.0, MRR 1.0.
Under a naive client that difference is invisible: both surfaces are just text in the prompt. Under a tool-search client it is the whole game, because a tool the search does not surface is a tool the agent does not have, however cheap it is. Rank on schema size alone and the winning move is gutting descriptions.
What is measured and what is not
Every cell carries a MEASURED or MODELED label. Naive full-load is measured: the token count of the canonical serialization. The per-tool costs in the tool-search column are measured, but its total is modeled, because k, the number of tools a session pulls, is an assumption, which is why it publishes as a 3-to-5 range and not a point. Code mode is modeled end to end and keeps that label until a Tier 2 run validates it.
Retrievability queries are derived from each tool's own description, so the metric measures within-server disambiguation, whether sibling tools shadow each other, not whether real phrasing finds a tool. It shares input text with the hygiene grade, so the two are not independent evidence.
Dollar figures are estimates, computed from a price table pending verification and shown only as a cold-write and cache-read pair. Tokens stay the primary unit: a single dollar figure is the cold-start number presented as though it recurs, overstating steady state by roughly 10x, since tool definitions are a near-perfect cache prefix. The rest is in docs/methodology-v0.md 0.2.0, with the rejected alternative for each judgment call.
The governance, as mechanism
Neutrality is not something to take on my word, so here are the checkable parts.
Selection runs on a published rule: five binary gates (protocol compliance, full surface enumerable under a free-tier credential, verifiable adoption evidence, a release in the trailing six months, no official successor), plus a category spread and a 15-slot ceiling. Coverage is not a claim this project makes. Anyone can submit a server through a public issue and gets the gate result back, pass or fail, with the failing gate named.
The corrections log is public and append-only, and it has two entries before the first ranking: a mismatch between what the methodology said the harness did and what it did, caught in review before any release, and the fetch row above. Before a server's numbers first publish, its maintainer gets 14 days with the rows, the artifacts, and the methodology version; a reply publishes verbatim beside the row, and non-response is recorded as "no reply received", not as agreement.
Every row ships three SHA-256 hashes of the tool surface plus the raw wire artifacts, so a disputed count is checked against the bytes it came from, not against my word. Recusal is published before the first ranking, and my own project is excluded from rankings by rule.
What lands next
Repo: github.com/lopster568/loadline. Calculator: loadline-dev.netlify.app.
The next run adds slack, once its operator credential lands, and starts Tier 2: scripted task runs through a call-logging proxy, measuring real call and response flows. Tier 2 is the capability axis, and if it stops running the leaderboard pauses, because a cost-only ranking is worse than none.
If a number here is wrong, tell me and bring the row. The harness, corpus file, artifacts, and derived query sets are in the repo, so disagreement can be specific, and corrections go in the log with your name on them.
loadline
STATUS: pre-release draft. Not published. All text pending owner review.
loadline is a standing, versioned measurement of what MCP servers cost an
agent's context window. It measures on a monthly cadence, run manually today
per docs/pc-sweep-runbook.md, across a curated set of servers, counts
tool-schema tokens with three tokenizer adapters, and reports the result
through a stack calculator: pick servers, a client mode, and a model, and
get total context footprint, window share, per-server attribution, and a
cold-write/cache-read dollar pair. Published rows carry counts from all three
tokenizers (OpenAI o200k locally, Claude and Gemini via their token-counting
APIs); a cell whose count could not be obtained publishes as
available: false, never as an estimate. Costs
are always reported per client mode (naive full-load, tool search /
progressive disclosure, code mode), never collapsed into one number
because the same stack costs a different amount depending on how the
client…
I do this work for hire: auditing what your agent's tool surface costs before it does any work, and cutting it down without gutting what the search can still find. Scope and pricing at roshansingh.systems/#hire, or write to inbox@roshansingh.systems and tell me what your agents are loading.
Top comments (1)
Context cost is a real architecture cost. MCP servers are powerful, but every tool description competes with the actual task for attention.