A version number is a promise: nothing a client depends on changed without me saying so. For MCP servers, almost nothing checks that promise.
The copy of a server in the MCP Registry carries a name, a version, the packages and the remotes. It carries no surface: no tools, no instructions, nothing about what answers without a token. So anyone comparing the registry with the live server can compare exactly one thing, the version. That comparison is only worth something if every surface change bumps the version.
This post is about turning that "if" into a failing test, and then about what happened when I ran the test backwards over the whole history of my seven servers: 142 published versions.
Where this came from
It started in the comments of my previous post, in a thread with @yahhi. The registry listing and the live worklore server both said 0.1.0, so a version diff passed cleanly. But, as @yahhi found, the server had moved on: tools/list now answered without a token, and the auth rules had moved into the instructions. The drift was one level deeper than the version.
Next came a working check, and a replay of the surface from the day of the listing against the current code. The registry said 0.1.0 for a server with four read-only tools. The server behind that number had six, two of which write on the user's behalf.
I said I'd build the same thing for my servers. It's now live on all seven, and packaged as @sbissoli/mcp-surface.
1. Lock what the client sees, not what the code says
The lock file, surface.lock.json, sits at the root of each server, next to package.json. It has two sections. Each stores the version it was locked under and a sha256 of its content.
-
Declared surface: the
initializeresult (instructions and capabilities included,serverInfowithout the version), plustools/list,resources/list,resources/templates/listandprompts/list. Keys are sorted and lists are ordered by name or URI, so the hash only moves when the surface does. A method the server doesn't serve is recorded asnull, not[]: "no prompts" and "prompts/list isn't served" are different surfaces. - No-token behaviour: which methods answer without a credential, on every MCP route, with the API key unset and set. No listing shows this, and it's exactly what had drifted in that server.
The declared surface is captured from the server factory in-process. The no-token section is measured at the HTTP edge, by calling the Worker's fetch with and without the key.
2. Make the version bump the only way out
The rule is short:
- measured ≠ locked and the version is the same → the test fails;
- the version changed → the test fails and asks you to re-lock.
The script that rewrites the lock follows the same rule: it refuses to lock a new surface under the old version. You can't "fix" a red test by regenerating the lock. You bump the version, re-lock, and commit the lock with the bump:
npm version minor --no-git-tag-version
npm run surface:lock # build + run the surface tests in write mode
The deploy runs npm test before wrangler, and the npm publish runs it before npm publish. So a surface change without a version change can't be deployed or published.
One side effect, and it's on purpose: an SDK upgrade that changes capabilities also turns the lock red. The client sees a different surface, so the version should say so.
3. Green tests prove the code; a post-deploy check proves what's served
Tests run against the code in the repository. The client talks to whatever is actually deployed. Usually those are the same thing. Wrangler configs, build steps, environment variables and caches are where they stop being the same.
So every deploy now ends with one more step that queries the live endpoint and compares it with the lock:
npx mcp-surface verificar https://<host>/mcp --tool <a tool that needs no network>
If what's being served isn't what was locked, the deploy job goes red after the fact, which is still much better than never.
4. The lock only protects you from now on, so replay the history
A lock written today says nothing about the versions that came before it. For those there's a replay:
npx mcp-surface replay --url https://<host>/mcp
For every version published on npm, it installs the package in a temporary directory (--ignore-scripts), starts it over stdio and captures the surface with the same normalisation the lock uses. Then it compares each version with the previous one, lists every removal that didn't come with a major bump, and checks the live endpoint against the surface of the version its /status reports. The output is a Markdown table and a JSON file in baselines/, committed to the repo.
Across the seven servers that came to 142 versions. In all seven, the live server matches the version it reports. It also found three things I didn't know.
5. Two releases that don't even start
Two old releases of my Senate server, 1.1.0 and 1.1.2, crash on startup:
Error: Dynamic require of "events" is not supported
The esbuild bundle emitted ESM that still contained a CommonJS require for a Node built-in. Nobody had noticed, me included, because nobody installs a two-version-old release on purpose. But npm would have offered both as valid versions indefinitely. Any version diff would have called them fine forever, because their version numbers are perfectly well-formed. They're deprecated now.
The lesson: "the version string is right" and "the version runs" are different claims, and only one of them shows up in a registry.
6. A minor release that removed six tools
My medical terminology server went from 1.0.2 to 1.1.x and six SNOMED CT tools disappeared from the default surface. They moved behind a licensing flag, off by default.
The CHANGELOG for that release even called it "technically breaking". It shipped as a minor anyway. There was a reason, since SNOMED access depends on a licence the user has to hold. But the client that was calling snomed_search yesterday doesn't read my reasoning. It gets "unknown tool".
The replay lists cases like this without judging them. Some removals have a good reason, and the CHANGELOG is where that reason lives. What matters is that the list exists, so the decision is made on purpose rather than discovered later by a user.
7. The replay was wrong once, too
The first version of the replay flagged ilo-mcp-server 0.5.0 → 0.6.0, where a parameter became required, as a breaking change outside a major release. But under semver, 0.x is allowed to break in a minor. The line of compatibility in 0.x is the minor, not the major.
The rule is fixed now: in 0.x, breaks are compared within the same minor. I mention it because a tool that reports drift has to earn its alarms. One false alarm on day one teaches people to ignore the next real one.
8. Check the probe before you write the lock
This is the one I'd most like other people to avoid.
In one server (sih-br-mcp) the MCP backend sits behind an edge proxy, so the no-token test can't call the real handler in-process. My first test double for that backend answered result to every method. The lock was written from it, and it faithfully recorded "prompts/list answers" for a server that has no prompts.
All the tests passed, because the tests were checking the lock against the same lying double. The post-deploy check against the live endpoint is what caught it.
The fix has two parts. A double that answers everything is not a double of your server, so where the real handler can run in-process, run it. And the probe's sanity checks (a method that must answer, a method that must not) now run before anything is written to the lock, not after. A lock written from a broken probe is worse than no lock, because it makes a wrong surface look verified.
What I would do first, next time
-
Lock the surface next to the version from the first release:
initialize, the four lists, and which methods answer without a token. - Make the lock script refuse to rewrite under the old version. Otherwise the red test just becomes a ritual.
- Check the live endpoint after every deploy. Green tests prove the code; only this proves what's served.
- Run the replay once over your whole history. It's cheap, it's one command, and the first run is the one that finds things.
- Prove the probe can say "no" before you trust anything it says "yes" to.
The package is MIT: @sbissoli/mcp-surface, source in mcp-br-commons. Fair warning: the API and CLI verbs are in Portuguese (travar = lock, verificar = verify), and the roadmap follows my own servers' needs.
Thanks to @yahhi for the thread that started all this. There is also a write-up of the other side of it, in Russian, on Habr. One comment thread changed how eight servers ship.
Top comments (20)
The lock makes the publisher keep the promise, but a client still can't check it, because the hash lives in your repo and the registry entry carries only the version. I'd publish the section hashes with each release. server.json already has room for it: the registry accepts custom metadata under _meta, in the io.modelcontextprotocol.registry/publisher-provided key. An agent host could then compute the same normalised hash from initialize and tools/list on first connect, and refuse or ask for re-approval when it differs from what the registry lists for that version.
That would have caught the instructions move in the worklore case from the client side, without anyone running your replay. It does depend on the normalisation being written down somewhere other than your code, so a host built by someone else hashes the same bytes. A short spec of the canonical form in the mcp-surface README would cover it.
You were right, and it's done. Thank you — this was the half I was missing.
From today, each release of my seven servers publishes the fingerprint in its MCP Registry entry, under
_meta→io.modelcontextprotocol.registry/publisher-provided: the sha256 of the declared surface (initialize, tools, resources, prompts) and which methods answer without a credential on the published endpoint. A host can recompute it on first connect and refuse, or ask again, when it differs from what the registry lists for that version.On the second point, the canonical form now lives in a SPEC.md rather than only in my code, with a test vector. Writing it down is what exposed two things that would have made a host built by someone else hash different bytes: lists were sorted with
localeCompare(locale and ICU dependent, now UTF-16 code-unit order), and the capture didn't follownextCursor. Neither changed any existing hash. There's also a second implementation, written from the spec with no dependencies (verify.mjs), and the tests require both to agree:The release pipeline now ends with the same check a client would make: registry entry against the live endpoint, without reading the lock.
What it proves: what you run is what the registry published for that version, and the registry won't let that entry change; with the source, anyone can also check that the hash is the public code's (SPEC §6.1). What it can't prove is that a declared tool is benign — that stays a human decision on re-approval.
One gap left on the host side: the check runs only on first connect. A server can answer tools/list one way at the start of a session and another way twenty minutes in, and the spec only says a server SHOULD send notifications/tools/list_changed when its list changes. A server that wants to switch quietly just won't send it. I'd have the host hash every tools/list response it receives, not only the first, and treat a mid-session mismatch with the registry entry the same as one on connect. Your capture already follows nextCursor, so it's the same function called more often.
SPEC.md seems like the right place to say it: the promise holds per response, not per connection.
Agreed, and it's in the spec now: SPEC §6.2.
One wrinkle shaped the wording. The registry hash covers the whole surface, never tools/list alone, so a later response can't be hashed by itself. The host swaps the new list into the surface it captured on connect and rehashes all of it. That keeps every hash already published valid, with no new field. The same goes for the other three lists.
Tried it today. worklore 0.5.1 now publishes the sha256 of every part of the surface a stranger can reproduce without a token (initialize without serverInfo.version, tools/list, and which methods and tools answer anonymously) in server.json under _meta → io.modelcontextprotocol.registry/publisher-provided. The deploy check recomputes them from the live server and compares them with the registry entry, the way a client would, and the README has a 20-line script so anyone can do the same: github.com/worklore/worklore-mcp#v...
One thing I left out on purpose: resources/list. Mine lists stories, so it changes every day, and its hash would break the promise daily without anything actually breaking. Thanks for the idea; it was the missing half.
Thank you — your cut is the one I used. What goes to the registry is only what a stranger can reproduce without a token: the declared surface, and which methods answer anonymously in the production configuration. The keyed variants stay in the lock, where only I can measure them.
One difference from yours: I kept
resources/listin. I measured the seven servers and none of them lists anything that changes, so here it holds still. Your reason for leaving it out is now written into the spec as the rule: a list that changes on its own does not belong in a promise. Your 20-line script becameverify.mjs, and it is the second implementation the tests compare against.That's the best possible outcome for a 20-line script: becoming the second implementation your tests check against. And your reason for keeping resources/list is the right one: the rule should be "only what holds still", not "never resources".
One thought now that there are two of us: is your spec public? If our _meta blocks used the same key and section names, a client could verify any server that publishes them, not just ours. I'm happy to change worklore's format to match yours, since yours has the tests.
Yes, it's public: SPEC.md, and every registry entry links to it.
I'd love worklore to use the same format. The key stays io.github.sidneybissoli/mcp-surface, with form: "mcp-surface/1" and the declared / anonymous sections. Renaming it would make it a new form and invalidate the hashes already published, so it's the one thing I'd rather not move. §7 has a test vector to check an implementation against, and verify.mjs should then accept worklore's entry as-is.
Two things landed in the spec today that you'll recognise: your "can't say no" probe is step 2 of §6, and §6.2 now says the check holds per response, not per connection.
Done: worklore 0.6.0 publishes mcp-surface/1 under your key, and your verify.mjs accepts it against production:
io.github.worklore/worklore@0.6.0 at worklore.dev/mcp
declared registry d5e4e0a80b32 live d5e4e0a80b32 ok
anonymous 7 methods ok
To follow §2, the stories moved behind a resource template (worklore://story/{slug}), and resources/list now holds three fixed documents: a guide, the story format, and the access rules. I reproduced the §7 vector byte for byte before publishing.
One thing I found along the way: verify.mjs builds the sorted object with Object.fromEntries, and JavaScript then puts integer-like keys first, in numeric order ("9" before "10"), while §3 and RFC 8785 order them as strings ("10" before "9"). Nothing in either of our surfaces has such keys, but a third server's could make the two implementations disagree.
Confirmed from here: verify.mjs accepts io.github.worklore/worklore@0.6.0 against production, d5e4e0a80b32 on both sides and the 7 anonymous methods as declared. Thank you for publishing it, and for reproducing the §7 vector before you did.
You were right about Object.fromEntries, and the bug was not only in verify.mjs: the package built its hash the same way, so the two agreed with each other and the tests had nothing to catch. Both now write the JSON key by key, so "10" comes before "9" as §3 says. Before changing anything I checked the seven servers: none has such a key in its locked surface, and all seven live endpoints still give their published hashes, so no fingerprint moves. The fix is @sbissoli/mcp-surface 0.5.3.
The SPEC now has a second vector, §7.2, built around exactly this case: a tool whose inputSchema has the properties "10", "9" and "a". Its sha256 is c7255a44a9d70ea6417ae2757c0530895b628551c3ae9fbfc440878a375539b1, and a third implementation (Python, json.dumps with sort_keys) agrees. Running worklore's implementation against it would close the loop on your side too. The note in §3 credits you.
Section 6 made me think the lock has a hidden input: the configuration it was measured under. With SNOMED behind a licensing flag,
1.1.xreally has two surfaces, and the lock only describes the default one. A user who turns the flag on gets six tools the lock never saw, and if one of those tools changes later, nothing goes red.Two cheap options I can think of. Record the env and flags used for each capture inside the lock, so the hash explicitly means "surface under config X". Or lock each documented flag combination as its own section. The second also helps the replay, since a minor that moves tools behind a flag would show up as moved rather than removed.
Per-tool hashes inside the
tools/listsection would help in a similar way, so a red test names the tool that drifted.Are flag-gated tools common across your seven servers, or is SNOMED the only one?
Good catch on the hidden input. In 1.1.x that was exactly the case: the six SNOMED tools only registered with the licensing flag on, so the lock described the default surface and nothing else.
That surface no longer exists: SNOMED was retired in 2.0.0, and the server has one surface again. Across the seven servers, no tool is gated by an env flag today. The one server with two surfaces (Senate) switches by route: the full set versus a smaller ChatGPT-app profile. Its lock stores each profile as its own section of the content, which is your second option. Auth configuration was already explicit: the no-token section is measured with the key absent and present. What the declared section still doesn't record is the configuration it was captured under. It's implicit in the test that captures it, and if a flag ever comes back, your first option is the fix.
Your third point I did today, slightly differently. Per-tool hashes would change the canonical form, and with it every hash already published in the registry. But the lock already stores the full content, so as of @sbissoli/mcp-surface 0.5.2 a red lock diffs it and names the drift: full.tools[senado_agenda_plenario].description: changed; full.tools[senado_ceaps]: removed.
Thanks — that one will save me a JSON diff by hand next time.
Thank you, Sidney, and what a follow-up: 142 versions replayed is the part I'd never have done. Two things grew on my side from the same thread. Since 0.5.0 my lock also records which tools answer without a token, after I made search and reading public and nearly shipped it without noticing that it changed the surface. And your section 8 hit home: my live check now asks a made-up method first, and if that gets a result, it refuses to compare anything, because a probe that can't say "no" proves nothing.
Thanks, Valentina.
Before today mine only half-did it. The no-token probe checked one method that must answer and one that must not before writing the lock. But the post-deploy check against the live endpoint never asked for a method that doesn't exist. A yes-to-everything endpoint would only have shown up as a diff someone had to read.
As of @sbissoli/mcp-surface 0.5.1, verificar and the registry check ask for a made-up method first, and refuse to compare if it gets a result. The standalone verify.mjs does the same, and the spec gained that as step 2 of "checking it". Before publishing I ran it against all seven servers in production; they all say no, so it would have been the canary rather than the culprit.
Glad the anonymous-surface piece is in your lock too.
The null vs [] distinction is the detail I keep coming back to. "No prompts" and "prompts/list isn't served" really are different surfaces, and most diffing tools I've used collapse them into the same empty result.
The crashed 1.1.0 and 1.1.2 releases hit close to home. I've been bitten by esbuild emitting ESM that still keeps a CommonJS require for a Node builtin, and it only shows up when somebody actually installs that old version. Registry metadata never catches that.
One question: does the lock hash each tool's inputSchema as part of tools/list, or just names? A changed argument shape on an existing tool has been the most common silent break in my experience, and I might be missing where that's covered.
Thanks — and yes, the esbuild case is exactly the kind of break that only shows up at install time. That's why the replay installs each old version instead of trusting the registry.
To your question: the whole tool, not just the name. Each item from tools/list goes into the lock as the server sends it — name, title, description, inputSchema, outputSchema, annotations — so a changed argument shape on an existing tool changes the sha256, and the failing test names the tool and the field that moved.
But your question exposed a real gap one step further: the lock saw the change, it didn't judge it. Any version bump was enough to re-lock, so a type change or a narrowed enum could ship as a minor. As of @sbissoli/mcp-surface 0.6.0 (out today), the lock rule classifies breaks — on the input, a parameter type or enum value no longer accepted, a tightened bound, a newly required parameter (nested properties included); on the output, a guaranteed field removed or made optional, or a value the client didn't expect — and refuses to re-lock them unless the major goes up.
Before shipping it I ran the rule over the git history of all seven servers' locks: 35 surface changes, one break found — the release that retired SNOMED, which had gone out as a major — and zero releases it would have wrongly blocked. All seven servers now run it.
The rule is described in the package README: github.com/SidneyBissoli/mcp-br-co... — thanks for the question.
The test double returning valid results for every method in section 8 is the exact failure mode that bites snapshot-based contract tests. If the harness cannot prove an invalid method or missing capability produces an explicit protocol error, the snapshot only records whatever the mock felt like emitting.
On the agent client side, the nastiest drift I hit with MCP servers is silent edits to tool descriptions. Under conventional semver, rewording a description looks like a no-op documentation tweak that never warrants a minor bump. Because agent runners inject tool definitions directly into the system prompt, a one-word change in a description busts prompt cache prefixes across active sessions and can shift tool selection behavior without a single parameter changing. Locking description hashes alongside schema definitions saves hours of debugging mysteriously evaporated cache hit rates.
Thanks, Reid — and yes, the double that answers everything is the trap I'd most like people to avoid.
On descriptions: the lock already covers them, though the article didn't make that explicit. It hashes each tool's full object — description, input and output schemas, annotations — plus the server's instructions, so a one-word rewording turns the build red until the version moves. It got exercised today: I reworded tool descriptions on six servers to say what each number is and which version of it, and the lock refused every one until it got a minor bump.
Your prompt-cache angle is a better argument for that rule than the one I gave. I framed it as "the client sees a different surface"; you're pointing at a cost the client pays even when nothing about the parameters changed. Thanks for that.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.