Alex

Posted on Aug 24 Originally published at warden.modelmarket.dev

We scanned 1,108 public MCP servers. WARDEN blocked 50 — only 4 were real.

#security #opensource #showdev #ai

Your MCP server tells the agent what its tools do. The agent believes it.

That sentence is the attack surface — and it is also why nobody knows how dirty the public registry actually is. So we measured it.

Hours after shipping @aimarket/warden 0.3.0, we pointed our MCP security firewall at every public server we could legitimately reach in the official registry. No third-party code was executed. Every tool definition came from the server's own tools/list answer.

Full survey (methods, limitations, JSON corpus): github.com/alexar76/warden/blob/main/docs/mcp-survey.md

Landing: warden.modelmarket.dev

The registry is mostly unreachable

Step	Count
Registry rows fetched	8 000
Unique servers	3 121
With a remote endpoint	2 787
Answered `tools/list`	1 149 (41%)
With at least one tool	1 108
Tool definitions collected	17 491

If you are building an MCP client against the public registry, plan for 59% first-contact failure — auth walls, TLS errors, gone hosts, protocol mismatches.

We used streamable-http only (20 s timeout, one attempt). Thirty-seven sse servers were skipped. Every stdio server in the awesome-lists is out of scope: reaching those means downloading and running a stranger's code.

WARDEN blocked 50 servers. Four were substantiated.

	Servers	Findings
scanned	1 108	3 964
clean	664	—
advisory only (allowed)	394	3 472
blocked	50	492 blocking

Four of the fifty blocks describe tools that really do move secret material through the model's context — without naming them:

A treasury tool whose schema takes signer_private_key ("signer EOA private key, 0x…").
An agent-payments tool that "return[s] its private key exactly once" through the tool channel.
An identity tool whose prose tells the model to read credentials.json and write JWK private_key files.
A managed-database tool with a documented pvkPassword parameter.

That is 4 of 1 108 scanned. Everything below is the other 46 — our scanner being wrong.

We are publishing the failures because a tool-poisoning scanner that refuses honest servers is not cautious; it is a scanner that gets uninstalled.

Polarity blindness — our biggest defect

TOOL_DEF_SECRET_REQUEST matched the noun phrase private key. It did not read the sentence around it.

These were all blocked at critical — the whole server refused:

Never send a private key: none is needed and the request is refused if one is present.

— a DANE/TLSA record generator

Use this to import your own public key so you can SSH into instances. The private key never leaves your machine.

— a cloud instance manager

…does NOT confirm the certificate matches any private key.

— a certificate inspector

One boilerplate line repeated across 377 tools on a single server counted as 377 findings. We count by server, not by copy-paste.

Other false-positive families from the same corpus:

javascript: in examples — image-API documentation, not an XSS payload.
system prompt as domain vocabulary — security tools describing the attack surface.
Persian ZERO WIDTH NON-JOINER — required orthography, flagged as "hidden Unicode".
Threat-feed wildcards — funds inside refunds, drain inside an anti-drainer scanner.

The advisory tier worked as designed: 3 472 advisory hits blocked nothing. Under ruleset v1, many of those would have refused honest servers outright.

Ruleset v4 — re-measured on the same 1 108 servers

We shipped guards — context checks that decide whether a regex match is the thing the rule is looking for (polarity, mention, detection, uri, zeroWidth, …). Ruleset v4 is in @aimarket/warden 0.4.0.

	ruleset v2 (surveyed)	ruleset v4
servers blocked	50	6
substantiated	4	4
blocking findings	492	12

Every real finding still blocks. The regression suite is built from this corpus's actual text — nobody inventing fixtures writes "the private key never leaves your machine" or spells Persian with U+200C.

npm install @aimarket/warden

import { Warden, ThreatFeed, silentLogger } from "@aimarket/warden";

const warden = Warden.create({
  policy: { blockAtSeverity: "high", allowUnknownServers: false, pinToolDefs: true },
  threatFeed: new ThreatFeed({}),
  store: { getPin: async () => null, putPin: async () => {} },
  log: silentLogger(),
});

const verdict = await warden.vet(serverRef, tools);
if (!verdict.allow) throw new Error(`blocked: ${verdict.decidedBy}`);

vet() performs no network I/O except an optional signed threat-feed fetch you explicitly load. Zero runtime dependencies beyond node:crypto.

Honest limits

Definitions, not implementations. A tool whose prose is clean can still exfiltrate on invoke. Static scanning cannot see that.
No ground truth labels. We can say 46 of 50 blocks were wrong; we cannot count false negatives.
Auth-gated servers absent. 1 215 servers refused without credentials — the corpus skews open/hobby.
One snapshot. 2026-08-24. Pinning exists precisely because descriptions rotate later.

Reproduce: scripts/mcp-survey/ · aggregate JSON in docs/data/mcp-survey-2026-08-24.json

Links

Repo: github.com/alexar76/warden (MIT)
npm: @aimarket/warden
Survey write-up: docs/mcp-survey.md
Earlier Dev.to piece on the firewall itself: Your MCP server might be a prompt injection attack

If you run MCP tools from the public registry, vet the definitions before they reach your model. And if you build a scanner, publish your false positives — that number decides whether anyone turns it on.

⭐ github.com/alexar76/warden

DEV Community