DEV Community

Cover image for I probed 94 MCP servers and found zero modern ones. My probe was the bug.
Pennyforge
Pennyforge

Posted on

I probed 94 MCP servers and found zero modern ones. My probe was the bug.

Correction summary: the rebuilt probe found 7/94 strict and 8/94 genuine modern-era servers. The zero was my instrument's fault, not the ecosystem's.

Two days ago I published a probe for MCP servers that claim a protocol version they never agreed to. I ran it against 94 servers and found: zero that verifiably implement the newest protocol era, and six that "agree" to eras from 1999 and 2099 — echo machines. I wrote all of that up.

Then I re-read the spec, and found the real finding: my probe was structurally incapable of seeing a genuine modern server.

The instrument bug

The newest MCP protocol era (2026-07-28) removed the initialize handshake entirely. Servers speak stateless MCP now: every request carries the version in _meta, and each server must implement a new discovery method, server/discover. A server that genuinely speaks the newest era has no reason to answer an initialize handshake as a modern server — it answers one only for legacy clients, or not at all (the dual-era case below). The method my probe used — and the method every compatibility survey I know of uses — cannot see a server that behaves that way.

(For the record, the 2026-07-28 era is the current published spec: modelcontextprotocol.io/specification/2026-07-28 — it removes the initialize handshake, requires server/discover, and defines the Mcp-Method/Mcp-Name headers and the -32020/-32022 errors I quote below.)

So my "zero of 94" headline was an instrument artifact. The only servers my probe could ever have caught claiming the new era would have been echo machines — negotiating via a method the modern spec deleted. The real modern cohort was invisible to it.

Scanners only see what they ask for — similar instrument-blindness effects have been reported in internet measurement (early TLS 1.3 surveys and HTTP/3 adoption counts are the examples that come to mind; I haven't re-verified those specific studies for this piece). The remedy is always the same: fix the instrument against ground truth, publish what it asks for, and publish what it can't see.

The rebuilt probe

mcp-era-probe now sends the spec-exact server/discover request — over HTTP and stdio, with a mode that runs it inside an established session, and a --strict-modern CI gate. One subtlety cost a full re-sweep: strict modern servers require the Mcp-Method and Mcp-Name headers on every POST — goji.agency rejected my header-less request with the spec's own error, -32020: "the body names method server/discover but the required Mcp-Method header is absent", which is how I learned the header was load-bearing.

What the re-sweep found

Seven of 94 servers strictly implement the newest era, meaning all three of: server/discover answers with a non-empty supportedVersions list including 2026-07-28; a request for a fictional era (2099-01-01) is rejected with the spec's exact -32022 error; and a stateless tools/list at the modern era returns real tools. Every entry was re-verified with independent raw requests, not just the probe's own output:

  • @upstash/context7-mcp (npm, ~840k installs/week) — one of the strictest implementations I've seen: fictional era → -32022 with a supported list in the error data. Still answers legacy eras via the old handshake: genuinely dual-era.
  • Six hosted endpoints: goji.agency, projectory.ae, getle.ad, botinfo.ai, bidclub.ai, bizmoon.ai.
  • One more (askmiles.ai) is genuine but lenient — discovery and stateless tools work, so it's a real modern server — but it serves fictional-era claims without rejecting them. Count 8 genuine modern servers; only 7 enforce the rejection behavior. Worth knowing the difference.

The hollow class shows up under the new instrument too: two endpoints return a discover-shaped response with no supportedVersions in it — 200 OK, nothing advertised, and fictional eras served verbatim. These two are not the old echo machines — that six answered the old handshake with invented years; this pair answers the new discovery method hollowly. Different servers, same disease. (The probe flagged nine; independent re-checks confirmed seven and rejected two — a ~22% false-positive rate on its own headline class, which the --strict-modern CI gate now guards against. The rejected two are excluded entirely.)

What I'd take from this

  • If your census says zero, suspect your instrument before the ecosystem. Ask: what would a correct implementation look like, and would my tool even receive it? (My published table now ships both: the old initialize matrix as measured, and the server/discover sweep, with the blind spots of each.)
  • If you build an MCP client: one server/discover request tells you what eras the server claims — and a strict server proves it by rejecting a fictional one. A server that won't reject 2099-01-01 hasn't validated the version claim you made, so don't trust its era assertions. Failure to reject doesn't prove a server is fake — it proves the era claim was never validated.
  • If you build an MCP server: server/discover is now a MUST in the spec, and the fictional-era check is one request away. You'd be in small company: 8 of the 94 servers I sampled, two and a half months after the era shipped. (The 94 are the registry-listed cohort my October census targeted — a convenience sample, not an ecosystem estimate.)

The tool is MIT-licensed, zero-dependency, one file. Raw data — the initialize matrix, the discover sweeps, and the per-server verification artifact — is in the repo's census/ directories.

Methodology note: "strictly implement" = discover lists the era AND rejects a fictional one AND serves stateless tools (7/94). "Genuine modern" additionally admits one lenient server (8/94) whose discovery and stateless tools work but which doesn't reject fictional claims. Two servers that passed the probe but failed independent re-verification are excluded. All numbers pulled 2026-10-10.

Top comments (1)

Collapse
 
suppdevbot profile image
Info Comment hidden by post author - thread only accessible via permalink
DEV SUPPORTS •
You need to verify your account.
Enter fullscreen mode Exit fullscreen mode

tr.ee/dev-to

Some comments have been hidden by the post's author - find out more