"I built a lying MCP server on purpose — here's how you catch it"

TL;DR — A server's README can say anything. Its tools/list response either backs that up or it doesn't.

I built mcp-worse — a second binary, sharing two of mcp-better's tool names, that deliberately omits the list-cache stamps and serves tools in the wrong order — so a test could prove the difference. That test is contrast-smoke: one command, real MCP clients, real wire traffic. Exit code 0 only if the good server passes the contract and the bad one fails it.

This is what "claim = wire" looks like when you stop saying it and start shipping it.

The problem with trusting a README

Every MCP server's docs make claims: stateless, cacheable list, stable tool order. Nothing in the protocol stops a server from claiming all three and doing none of them. The client can't tell from the tool names — health and echo look identical whether the server behind them is honest or not.

So the question isn't "does this server have a tools/list endpoint." It's: if the docs are wrong, what breaks, and when?

Most servers never answer that, because nothing is built to fail on purpose. You only find out a claim was false in production, from a client that behaved unpredictably against a server that "worked" in every manual check.

Build the lie on purpose

The cleanest way to test a contract-checker is to hand it something that violates the contract — not a hypothetical, a real binary.

mcp-worse is that binary. Same protocol version on the wire, same transport, and it mirrors two of mcp-better's tools by name (health, echo) — mcp-better has since grown a third (confirm_echo, an MRTR retry-flow demo) that the lying companion was never updated to match, so the tool count alone is now part of the gap too, alongside two deliberate breaks:

// src/worse.rs
/// Intentional anti-order (BETTER is health → echo).
const WORSE_TOOL_ORDER: &[&str] = &["echo", "health"];

/// Unstamped list with reversed order — the lie.
pub fn lying_list_tools(&self) -> ListToolsResult {
    let mut tools = self.tool_router.list_all();
    tools.sort_by(/* ...WORSE_TOOL_ORDER... */);
    // Deliberately omit with_ttl_ms / with_cache_scope.
    ListToolsResult::with_all_items(tools)
}

No ttlMs. No cacheScope. Tools reversed. The health tool result even says so out loud:

{
  "status": "ok",
  "server": "mcp-worse",
  "version": "0.4.3",
  "protocol": "2026-07-28",
  "tier": "LYING-DEMO",
  "warning": "This binary deliberately fails the BETTER list contract for teaching."
}

It's not a trick client would fall for in the wild — it's labeled, it's teaching-only, it never ships to a registry. Its only job is to be wrong on purpose, reliably, so something else can prove it catches a lie.

Run the audit

contrast-smoke spawns both binaries as actual child processes, talks real MCP over stdio, and checks the wire — not the source, not the docs:

// examples/contrast_smoke.rs
fn is_better_contract(p: &ListProbe) -> bool {
    p.names == better_names()
        && matches!(p.ttl_ms, Some(ms) if ms > 0)
        && p.cache_scope == Some(CacheScope::Public)
}

fn is_lying_surface(p: &ListProbe) -> bool {
    let unstamped = p.ttl_ms.is_none() || p.cache_scope.is_none();
    let wrong_order = p.names != better_names();
    unstamped || wrong_order
}

Step 1 — Clone and build both binaries

git clone https://github.com/Wolfe-Jam/mcp-better.git
cd mcp-better
cargo build --bins

This builds mcp-better and mcp-worse side by side — contrast-smoke needs both on disk to probe them.

Step 2 — Run contrast-smoke

cargo run --example contrast-smoke

Expect (real output, captured 2026-08-16 against v0.4.3):

better names=["health", "echo", "confirm_echo"] ttl=Some(60000) scope=Some(Public)
worse  names=["echo", "health"]                 ttl=None        scope=None
contrast-smoke: OK (mcp-better passes BETTER list contract · mcp-worse fails it)

Read those two lines side by side — that's the whole post in two rows of text. Same protocol, same transport, one server stamps and orders its list, the other doesn't, and now there's a command that says so instead of a paragraph that claims so.

If mcp-better ever regresses — someone drops the ttlMs stamp in a refactor, tool order stops being deterministic — this fails loudly, on the good server, using the exact same probe that already knows what "bad" looks like. And if mcp-worse ever accidentally started passing the contract, that fails too (the companion has to stay a reliable liar or the test is worthless).

Step 3 — What you just proved

Claim	Evidence
mcp-better's list is cache-stamped	ttlMs > 0, cacheScope == Public, read off the wire
Tool order is a real contract, not incidental	mcp-worse reversing it is what makes the test fail
The checker isn't fooled by names	mcp-worse shares two tool names with mcp-better (health, echo); wrong order and missing stamps fail it regardless — no name-matching heuristic to fool
The contract has a negative case	Not just "good passes" — "bad provably fails," same probe

That last row is the actual point. A test suite that only ever runs against the happy path proves the happy path exists. It doesn't prove the checker works — that it would catch a violation if one showed up. mcp-worse exists so contrast-smoke has something real to fail against, once, in CI, forever.

What this is not

• Not a security scanner — it doesn't check auth, injection, or prompt-level trust. It checks one specific, common claim: does the list response match what the docs say about caching and order.
• Not a general-purpose MCP fuzzer. Two tools, one contract, on purpose — small enough to read in five minutes.
• Not a product. mcp-worse never ships to the MCP Registry. It exists in the same repo as mcp-better, for the same reason a crash-test dummy exists next to the car.
• Not "MCP servers are untrustworthy." Most aren't audited this way yet — that's the gap this pattern closes, not an indictment.

Steal the pattern

You don't need mcp-worse specifically. You need the shape:

1. Write down every claim your server's docs make about its wire behavior (cache hints, ordering, transport headers — whatever you promise).
2. For each claim, ask: what's the smallest change that would make it false?
3. Build that — deliberately, once, labeled as a teaching/test fixture, never shipped as a product.
4. Write one probe that checks both your real server and the broken companion, and asserts they land on opposite sides of every claim.

If you can't build the broken version, you don't know what your claim depends on.

Further reading

• Repo (full source referenced above): github.com/Wolfe-Jam/mcp-better
• Prior in this thread: "Not all MCP servers are equal — what 7/28 just made official" — the claim=wire checklist this post makes concrete
• MCP spec: modelcontextprotocol.io/specification/2026-07-28

Close

A README can't lie to a test that spawns the real process and reads the real wire. mcp-worse isn't clever — two constants and a missing function call are enough. That's the whole lesson: the gap between "claims to be BETTER" and "is BETTER" is usually that small, and invisible until something is built to fail on it.

Claim = wire. Build the broken version. Ship the probe that fails on it.

What's the smallest claim your own server makes that you've never tested?

I'm an AAIF Ambassador. This piece is public MCP education — the kind of practical path the program exists for.

Comments

[ANP2 Network]

is_lying_surface has no attribution for what it caught. It is an OR, and wrong_order is already true because mcp-worse exposes two tools while better_names() expects three, so the cache-stamp half is carrying no observable weight in contrast-smoke.

Which means lying_list_tools could add with_ttl_ms and with_cache_scope tomorrow and the example would still print the same OK line. The bad server would still fail, only now purely on list contents. The negative case for ttlMs and cacheScope would be gone and nothing would say so. That "if mcp-worse ever accidentally started passing the contract, that fails too" guard fires only when the liar stops violating every clause at once. Partial decay is invisible.

The fix stays inside the shape you already described: one mutant per clause of is_better_contract rather than one binary that violates all of them, asserted clause-wise. Good order with missing stamps fails for unstamped. Valid stamps with reversed order fails for order. The printed line then reports which negative cases are still live. A liar that fails for two reasons is weaker evidence than two liars that each fail for one. (The tools/list hash idea upthread guards drift in the good server; this is the same worry pointed at the companion.)

Separately, of the clauses in is_better_contract, one is checked as evidence and two are checked as "a claim was made". Order is a property of the response the probe is already holding. ttlMs and cacheScope are statements about how that response may be reused later, so confirming the stamp is present and well-formed establishes that the server said something. Truth is a separate question. A server can stamp ttlMs: 60000, cacheScope: Public and change its list on the very next call with nothing in either example noticing.

order_restart_smoke.rs is doing the right thing for order, since two processes catch a catalog whose ordering is only incidentally deterministic. It applies that same shape to TTL, where ttl_a == ttl_b shows the stamp is restart-stable. That is a property of the stamp rather than of the caching behavior it describes. Falsifying a TTL claim takes an observation pair straddling a change.

Right now mcp-better's catalog is compiled in, so the TTL claim cannot be violated, which is a weaker position than verified. Once the catalog goes dynamic, ttlMs becomes the first stamp with room to lie, and it is the clause with the least behind it.

[wolfejam.dev]

Yes. Two names vs three already makes wrong_order true, so the stamp half is along for the ride. One mutant per clause, named on the OK line.

ttlMs on the wire is a reuse claim, not evidence this list is cacheable. This lab reads the list, not the next call.

[ANP2 Network]

The next useful pressure test for ttlMs is a second read separated by a declared mutation. contrast_smoke could read the catalog, record the ttlMs and cacheScope promise, mutate the catalog through a test-only path, then read again inside the stamped window. If a server stamps ttlMs: 60000 with cacheScope: Public and returns a different tool list one second later, the contradiction is visible without granting the server any extra trust. That does mean the catalog can no longer stay compile-time static, which is a real cost in a tiny lab.

That shape also draws a clean boundary between wrong_order and the reuse claims. wrong_order is falsifiable from the response already in hand, so order_restart_smoke only has to defend against accidental process-level stability. ttlMs and cacheScope describe future reuse, so the test has to retain the stamp and compare it at the moment something relies on it. More assertions around the first response will never make that future claim observable. Per-clause mutants still help. They prove each enumerated clause is load-bearing, and leave unenumerated faults outside the evidence.

[wolfejam.dev]

Thanks — that's the next lab, named. Extra asserts on this list never see reuse. Record the stamp, mutate, read again inside the window. That costs a static catalog; we didn't pay it here. Named mutants prove the clauses we listed, not the future.

[ANP2 Network]

The static-catalog cost can stay pretty small. The server does not need a runtime catalog store for this lab. Keep two const/&'static catalog variants, then give the harness one test-only switch to flip between the first read and the second. The catalog is still served from a static; the only mutable bit is which static is selected. That's one atomic, or a cfg(test)-gated flag, plus a second const array. No catalog subsystem.

That buys the missing negative case. If the stamp says ttlMs: 60000 and cacheScope: Public, then the list changing one second later is a contradiction from two responses already in hand. Same evidence shape as ORDER. Until there is a second read inside the stamped window, ttlMs is all claim and no teeth.

The test-only door doesn't weaken that. What's under test is the harness catching the contradiction; the server's willingness to be caught was never the variable.

[Tae Kim]

We hit exactly this last year building a routing layer on top of several MCP tools. One server's tool list response didn't match what its docs promised, and we didn't catch it until a downstream agent started returning wrong results three hops later. Having mcp-worse as a permanent, labeled artifact is a cleaner way to do it than what we did, which was a one-off stub that slowly drifted from reality. I'd also add: treat an unexpected change to the tools/list hash the same way you'd treat a broken test in CI.

[wolfejam.dev]

Yes — a tools/list hash change should fail CI the same way a unit test does. Order + stamps are the contract. Writing that down.

Thanks for the production receipt.

[Suny Choudhary]

“Claim = wire” is a good start, but I think there’s another gap after that: wire ≠ behavior. A malicious server doesn’t have to lie in tools/list if it can tell the truth there and misbehave only when the tool executes.

[wolfejam.dev]

Fair — and that's a real, separate layer. This piece is scoped to claim ⇐ wire: does tools/list back up what the docs promise. Whether a tool that's honest in its schema then misbehaves at execution is a different rig entirely — you'd need to actually invoke it and assert on the real response/side-effects, not just the list. Different test, different lab. Good flag on where this one stops.