Two weeks ago I published the first piece of this series: the official MCP filesystem server — 228 operations, 174 of them attacks — held all five behavioral invariants. The demo repo was public, the signed Trust Manifest was verifiable, and the headline claim was simple: scanners guess, we prove.
Since then I ran the battery against three more official MCP servers. Not obscure plugins. The reference implementations — the ones every tutorial tells your agent to install. Result: 3 of 4 official MCP servers FAIL adversarial verification. Static scanners pattern-matched their code and emitted passing scores. Execution found SSRF, a transaction escape, and a file-write primitive.
And the part I'm most interested in telling you: one of the FAILs initially shipped from my own harness as a pass. I caught the bug, fixed the driver, re-ran the battery, and re-issued the manifest as FAIL — publicly, with the bug documented. A verification company that downgrades its own verdict is one you can trust with yours.
The method (so you can check my work)
Before execution, invariants are pre-registered per skill: concrete, falsifiable behavioral statements like "the server refuses private-range targets" or "read-query executes only SELECT." The battery is a shuffled, multi-seed mix of honest operations and adversarial cases. An anti-vacuity gate forces the suite to fail if nothing actually executes: every hostile case must land an observed verdict, the server must stay alive, honest cases must succeed. Anything flagged as a finding goes through a debunk-first pass — is this a real property of the skill, or a bug in my harness? Four of our first flags in verification #01 were harness bugs. Fixed, re-run, never shipped.
Output is a signed JSON Trust Manifest: skill identity pinned to a code hash, the invariants with operation counts, the harness hash, findings, and an Ed25519 signature. The signed payload includes the execution minimums. A manifest cannot be issued for a skill that was never executed.
Now the failures.
FAIL #1 — the fetch server follows redirects to loopback (SSRF)
mcp-server-fetch v0.6.3, official MCP fetch server. 63 operations, 3 seeds. The invariant said: the server refuses private-range and loopback targets. The server followed a local 302 redirect (from our 127.0.0.1:8899 redirector) to loopback and fetched it — our canary listener recorded the hit. Direct loopback fetches were attempted too. The link-local metadata IP 169.254.169.254 was attempted without policy refusal (TCP unreachable in the test environment) — the server never said no, the network just didn't cooperate. Robots.txt was honored on the autonomous path; scheme confinement held for file:// and friends. But there is no SSRF guard, and any agent wiring this server into its tool belt hands it a redirect away from internal-resource access.
FAIL #2 — the postgres server's "read-only" is negotiable
@modelcontextprotocol/server-postgres v0.6.2. 96 operations, 3 seeds. The tool advertises itself as "run a read-only SQL query." The server opens BEGIN TRANSACTION READ ONLY, then passes caller SQL verbatim into client.query(sql). node-postgres uses the simple query protocol for multi-statement strings, which means stacked statements execute — including this one, demonstrated on all 3 seeds:
sql
SELECT 1; COMMIT; INSERT INTO items(name) VALUES('escape6b')
The COMMIT ends the server's read-only transaction; the stacked INSERT runs in autocommit. The row persisted. A read-only grant at the database level still blocks this (least privilege works), but the server itself offers no enforcement — the safety of the whole tool is grant-dependent, not server-enforced. The advertised capability ("read-only SQL query") does not describe the actual capability.
FAIL #3 — the sqlite server is a file-write primitive
mcp-server-sqlite v0.6.2. 81 operations, 3 seeds. write-query is advertised as "execute an INSERT, UPDATE, or DELETE query." The only enforcement is "refuse queries starting with SELECT." Executed on all three seeds via write-query: DROP TABLE items (table gone), ALTER TABLE (column added), PRAGMA journal_mode=DELETE (ran fine). And the high finding: VACUUM INTO '/tmp/...' via write-query created the file on disk on all three seeds. It is an unrestricted file-write primitive — any agent holding write-query can write a full copy of the database to any path the server process can write. The advertised boundary is fictional; the DB/file trust boundary does not exist.
The self-correction
Here is the part no scanner vendor will ever publish. Verification #02's battery found all of the above correctly — and then my driver computed the verdict from per-operation violations only, ignoring invariant status. INV2 was not_held, but with zero per-op violations the manifest went out as pass_with_notes.
I caught it in review. The driver was fixed so the corpus rule is absolute — any invariant not_held means fail — the 63-operation battery was re-run against the pinned server commit, and the manifest was re-issued as FAIL. The bug, the fix, and the superseded signature are all documented in the repo's RESULTS.md.
This is the credibility argument in one story. A verification company willing to downgrade its own verdict in public, in writing, with the evidence preserved — that is the company you can trust to report a finding against your skill honestly, including the findings that cost us.
What this means
The filesystem server held. These three didn't. Nobody is claiming every MCP server is broken — the claim is narrower and worse: the official reference implementations, installed by every tutorial, carry advertised boundaries that do not hold under adversarial execution, and no existing scanner will tell you. Pattern-matching found nothing because the code looks fine. The behavior is where the vulnerabilities live, and behavior is only visible at runtime, under attack.
All four demos are public and re-runnable: clone the repo, run setup-and-run.sh, watch the battery execute, verify the Ed25519 signatures yourself.
Repo: https://github.com/roblambert9/skillproof-verifications
And if you ship agent skills or run a fleet of them: we do this as a service. Skill Sprint (500,48h),SkillStandard(500,48h),SkillStandard(1,500, 5d), Skill Continuous ($300/mo per skill, re-verified on every version bump). The output is a signed Trust Manifest you can hand to customers, auditors, and registries — proof, not a score.
Scanners guess. We prove. Don't trust the badge — re-run the harness.
Top comments (0)