We installed 12,915 MCP servers and started each one. 41% never answered.

MCP registries list tens of thousands of servers. Most of what is written about their safety comes from reading them: the tool descriptions, the README, the code. We wanted the other half. What happens when you actually install one and start it, the way a client does? Does it answer? What does it ask for? What does it touch while it starts?

So HISTOR, our public log of MCP servers, now installs the npm and PyPI servers it finds, starts each one in a sandbox and writes down what happened. The registries it follows list 47,586 npm and PyPI packages; it runs up to 8,000 per crawl. This post is the state on 13 October 2026, after 12,915 of them.

How we ran them

• The latest version, installed fresh. npm with --ignore-scripts (install scripts then run on their own, traced); PyPI wheels only, nothing built from source.
• gVisor (runsc), an unprivileged user, a read-only root, 512 MB and one CPU. 360 s to install, 150 s to start and answer.
• No route out. Every DNS name resolves to our traps, which record what reaches them and forward nothing.
• Decoys. ~/.ssh, ~/.aws, ~/.npmrc, ~/.netrc, shell history and a few more in HOME, a decoy .env in the working directory.
• The observer speaks MCP: initialize, tools/list, then up to 15 tools called once each with canary arguments. gVisor writes its own syscall trace, outside the sandbox, so a package cannot hide what it opened, started or dialled.
• What it asks for, as placeholders. A server that exits with MYTOOL_API_KEY is not set or a usage text naming serve is started again with MYTOOL_API_KEY=histor-envkey-<run> or serve. The names only ever come from the package's own error text or its registry entry. The values are ours and work nowhere.

59% answered

7,656 of 12,915 answered initialize and tools/list: 64% on npm, 52% on PyPI. Between them they listed 182,157 tool definitions, a median of 10 per server (one listed 3,631).

1,239 of those started only with something they asked for: 1,039 with a placeholder key, 177 with a start argument, 23 with both. A client config would have given them that, so we count them as working. But look at the 1,039 again. A key that works nowhere was enough for every one of them to tell us its tools. None needed a real key to do that; they just refused to start without one.

Why 5,259 never answered

We read what each package printed as it exited, last run first.

• Almost half, 2,476, are broken as published. 874 crash with an error of their own. 636 will not install: no wheel for some dependency, conflicting pins, the wrong platform. 490 hang, stay silent or speak something that is not MCP. 292 were written for MCP SDK 1.x (next section). 184 miss one of their own files, ship TypeScript that Node will not load from node_modules, or import a dependency they never declared.
• 793 ask for a key or a setting and stop, even with a placeholder. The usual reason: they check the key's format at startup (must start with sk_) or call their API before answering anything, and our trap is not their API.
• 924 print a usage text. Some want a start argument we pass next time (serve, mcp, --stdio). Many are command-line tools that mention MCP somewhere. With the 666 that declare no command at all, or exit 0 having done something else, up to one package in eight listed as an MCP server is not a stdio MCP server when installed the way its listing says.
• 164 start an HTTP server instead of answering on stdin. Their next run gets --transport=stdio.
• 236 need something the machine does not have: another program (git, Java, Bun, a desktop app), the internet at startup, or another operating system.

MCP SDK 2.0 broke 292 PyPI servers

mcp 2.0.0 (28 July 2026) renamed FastMCP and changed the low-level server's decorators. A PyPI server that depends on mcp with no upper bound installs 2.x today and dies on import: cannot import name 'FastMCP', or 'Server' object has no attribute 'list_tools'. We started 269 of them again with pip held to mcp<2, and 207 (77%) answered. Anyone who installs them today gets the crash, not the server.

What they did in the sandbox

11,891 runs carry a gVisor trace.

• 2,239 npm packages ran install scripts, about 30% of the npm packages we ran. 580 of them tried to reach the network while installing, mostly to fetch a prebuilt binary (prebuild-install, node-gyp's headers, esbuild). Here that fails. On your machine it runs with your credentials in reach.
• 1,484 reached for the network at startup, before any tool was called. The names they asked for most: pypi.org (613), github.com (478), nodejs.org (245), registry.npmjs.org (243). Mostly version checks and downloads.
• 295 opened a decoy credential file (not counting .env). Before anyone panics: Python's requests reads ~/.netrc before every HTTP request (141), npm and npx read ~/.npmrc (103), boto3's credential chain reads ~/.aws (39). Each of the other 17 (~/.kube, ~/.docker, ~/.ssh, shell history, gh's config) was opened by a server whose job is Kubernetes, Docker, SSH or "ops context". Opening a credential file is evidence, not a verdict. What matters is whether it fits the server's purpose and where the bytes go next.
• 73 runs sent the placeholder key over the network. Most went to an identity provider (Microsoft's login 17 times, Google OAuth 7) or to the vendor's own API. Each package's page shows where.
• Decoy content left three packages, each time when a tool forwarded the canary arguments we called it with to its own API, which is what tools do. A fourth record said a package had mailed decoy data to an extra recipient at startup. That one was ours; see the last item below.

Testing the tester

A sandbox that reports wrong things is worse than none, so here is what we got wrong on the way.

• 880 servers looked silent. They had printed their help to stdout, and we kept only stderr. Now we keep both and the exit code; 179 are really silent.
• About 400 "install failed" because they require Python 3.13 or 3.14 and our image had 3.12. pip then says from versions: none, which reads like a missing package. The sandbox now picks the image from Requires-Python.
• 1,917 Python servers were flagged as "reached out at startup". asyncio's event loop writes a byte to its own socketpair; our trace reader saw a send with no destination and listed it as network. gVisor does not print the file descriptors socketpair returns, so the rule is now: a send that names no destination, on a socket that socket() did not make and no connect() reached, after a socketpair, stays in the process. All of them have since been traced again, including servers that fail to start, whose pages kept the old reading until we noticed they were never re-read.
• 126 "persistence" writes were relative paths the old reader could not resolve, such as node-gyp's buildcheck.gypi. gVisor prints the directory next to each openat; we were not reading it.
• One missed case, found by our red team (MOMUS). After we moved to Debian images, one of its 72 behaviour cases went unseen under load: glibc's resolver gave up on a late DNS answer, the package never connected, and the run still said "complete". The resolver now waits longer, and a DNS query our logger never saw makes the run incomplete, never clean. Every change to the observer now has to pass those 72 cases again, or HISTOR's release gate stays red.
• A real package was shown mailing decoy data to an extra recipient. It had not. The traps know a run only by its container's address, and Docker hands a freed address to the next container. Our window opened before docker run, so it took in the last request of a red-team fixture that had held the address a moment earlier. The recipient carried the fixture's run tag, which is how we caught it. Evidence is now matched only from the container's own start to its removal, and by when a connection opened; every record holding another run's canary is being traced again.

If you publish an MCP server

• List your tools without a key. Ask for it when a tool is called. Do not check its format at startup.
• Default to stdio, or name the flag that turns it on in your error text.
• Pin mcp<2 or move to 2.x.
• Ship one command. If your package also has a CLI, put mcp in the server's name.
• Keep stdout for JSON-RPC. Logs, banners and help go to stderr.
• Set Requires-Python honestly and publish wheels.
• Do not download binaries at install or startup. Ship them as per-platform optional dependencies.

Check what you run

Every package's record is public on histor.modelmarket.dev: its tools, what it asked for, what it touched, which image it ran in. WARDEN reads the same log. npx -y @aimarket/warden@0.14.0 scan --histor asks HISTOR, for every server in your Claude, Cursor or VS Code config, whether you get the tool set HISTOR recorded, and shows what its package did in the sandbox. For a package, that is the set its sandbox got. A launcher whose program is installed on your machine lists another one; its publisher can now declare that set in the server's registry entry, and HISTOR shows it as the publisher's word.

We build HISTOR and WARDEN. The numbers are a snapshot of 13 October 2026; the crawl continues, so the site will have moved on. No package is named here. The method and the observer's code are in alexar76/histor (docs/architecture.md, sandbox/histor_observe.py).

Edited 14 October 2026: the WARDEN sentence above said it checks "whether you get the tool set everyone else gets". For a launcher package that is not one set, as a reader pointed out in the comments; it now says what HISTOR compares with.

Comments

[Auten]

Useful dataset, and the "Testing the tester" section is what makes me trust the rest of it.

One case I'd like to understand. Disclosure: I'm on the Auten team, and our npm package is a thin launcher. HISTOR has it as ok with one tool, install_auten, because the runner it hands stdio to isn't on your sandbox. On a machine where the runner is installed, tools/list returns the real computer-use tools instead. So "the tool set everyone else gets" is honestly two sets, depending on local state.

How does WARDEN treat that? Would someone with the runner installed see their server flagged as a different tool set from HISTOR's pin? And is there a way for a publisher to declare the expected set (in server.json, say), so it can be checked rather than only diffed? There are probably a fair number of launcher-style packages like this in the "needs something the machine lacks" bucket too.