I run ProtocolProbe, a directory of facts about every server in the official MCP Registry: each server gets a page, read from public records (the registry, npm, PyPI, GitHub), with a date on every fact.
A publisher emailed us. They had renamed their server, marked the old registry entry as deprecated, and added a message pointing to the new name. Our page for the old entry still said active.
I told them it would update on our next daily run. That was wrong. This post is about why, and what we changed.
The bug
Our daily job pulls the full registry list, compares it with yesterday's, and loads what changed: new servers, new versions, removed servers. For those, it re-reads the package and repository facts the same day.
The publisher's change was none of those. The registry lets a publisher change an entry's status (active → deprecated) and its status message without publishing a new version. Same version, different status.
Our job noticed it: it counted it as a "metadata-only" change. It just never loaded it. The page kept the old status, and the run still finished with ok.
How big was it?
One report can be a coincidence, so we compared every stored record with the registry list of that day, field by field (read-only, 51,834 servers):
| mismatches | |
|---|---|
| servers missing / extra | 0 / 0 |
| version | 0 |
status (stored active, registry deprecated) |
53 |
| status message | 60 |
So the parts the job was built for were exact, and the gap was precisely the case it skipped: entries deprecated by their publishers that we still showed as active. Our own record check "deprecated in the registry" gave the wrong answer for all of them.
Fix 1: reconcile what the registry owns, every day
The registry is the source of truth for a few fields: status, status message, published and updated dates. Now, after every daily load, the job compares those fields for all servers with the registry list and patches the ones that differ at the same version:
- a backup of every value before the first write, and a journal, so a crash mid-way resumes instead of half-patching;
- every patched document read back; a mismatch rolls the whole step back;
- a cap of 2,000 patches a day (a registry-wide change needs a person to look);
- each change recorded publicly with its date, so anyone can see what changed and when.
First run: 59 servers patched (some had changed since the count above). A second comparison right after: 0 mismatches across 52,138 servers.
Fix 2: a job that says "ok" must check the page
The deeper problem was not the missing field. It was that the job reported success without looking at what visitors see.
So the job now ends with a live check: after the pages are refreshed, it takes a small fixed sample of the day's changed servers (new, new version, status patched) and opens their public pages.
We first tried reading our own public API instead. It was the wrong thing to check, for two reasons we found by reading our code:
- The API is not what was stale. The page was.
- The API answer is cached at the CDN and inside each function instance, and the instance copy comes back without a cache header. A check right after a load can read an old copy and raise a false alarm.
Each server page now carries a small machine-readable marker in its head: version, status, a short hash of the status message (never the message itself), and the snapshot date. The job compares that with what it loaded.
- A mismatch is a warning, never a rollback.
- Our own failures (rate limits, timeouts) are "not checked", never a mismatch.
- If half the sample can't be checked, that is a warning too.
Fix 3: a date range, not a date
While checking this, we found a quieter version of the same problem on our leaderboards. They said "facts checked 2026-10-29". That was the newest date among the records, which meant the servers that changed that day.
Most stars and download counts behind the ranking had been read days earlier. The page now shows the range: "facts checked 2026-10-24 to 2026-10-29".
How stale does a directory get?
We measured it. We re-read the free facts (npm/PyPI, OSV, GitHub) of 1,729 servers that had not changed in the registry, a 1/30 slice of the catalogue. It took about 14 minutes and about 1,200 GitHub API requests.
614 of them (35%) had at least one fact that had changed since our last read:
- 396 download counts
- 301 last-push dates
- 99 star counts
- 23 version-pin states
- 7 repository states (not found, found again, archived)
- 1 newly withdrawn package
None of these show up as a registry change. We built a daily rotation that re-reads such a slice every day, so every server is re-read at least monthly. It is switched off until we have watched a few normal runs of the new checks.
What I'd tell anyone building on a registry
- A listing is a snapshot. The official MCP Registry checks an entry when it is published (each new version), and nothing re-checks it on a schedule. We asked for that upstream: registry issue #1715. If you mirror it, you inherit the same gap.
- Same version doesn't mean same record. Compare the fields the source owns, not just the version.
-
Check the output, not the exit code. Our job said
okfor days while pages were wrong. - Show the age of your data honestly. A single date can quietly overstate how fresh it is.
- Say when you were wrong. I told the publisher the wrong thing first. Admitting it took one email. Our corrections are logged at protocolprobe.com/corrections.
Every change the job records is public, by day, at protocolprobe.com/mcp/changes, and in a free API with no key (/docs/api). If you run an MCP server and something on its page looks wrong, /operators says how to tell us.
Top comments (2)
The 53 deprecated entries still showing as active is a useful find, and showing a date range on the leaderboard is a nice honest touch. The same version, different record problem goes one level deeper for remote servers: the endpoint can change what tools/list returns with no new version and no registry change at all. Do you record anything about the tool list itself, like a hash per server, or only the registry and package facts?
Yes, the tool list itself. Every read-only listing we capture is stored as hashes: one for the whole list, and one per tool for its title, description, input schema and annotations. Each server's reads are compared in order, and the hosted endpoint and the package are tracked separately so they're never mixed up. The server page then shows tools added, removed or changed between two reads, and flags "changed without a new version". When we first measured it on 2026-10-25, 0 of the changed pairs had the same version, so we haven't caught that case yet, but it's flagged when it happens. We never call the tools, and a read we don't have counts as unknown, never as "unchanged".