DEV Community

Cover image for "I built a lying MCP server on purpose — here's how you catch it"
wolfejam.dev
wolfejam.dev Subscriber

Posted on

"I built a lying MCP server on purpose — here's how you catch it"

#tutorial #mcp #ai #rust

TL;DR — A server's README can say anything. Its tools/list response either backs that up or it doesn't.

I built mcp-worse — a second binary, sharing two of mcp-better's tool names, that deliberately omits the list-cache stamps and serves tools in the wrong order — so a test could prove the difference. That test is contrast-smoke: one command, real MCP clients, real wire traffic. Exit code 0 only if the good server passes the contract and the bad one fails it.

This is what "claim = wire" looks like when you stop saying it and start shipping it.

The problem with trusting a README

Every MCP server's docs make claims: stateless, cacheable list, stable tool order. Nothing in the protocol stops a server from claiming all three and doing none of them. The client can't tell from the tool names — health and echo look identical whether the server behind them is honest or not.

So the question isn't "does this server have a tools/list endpoint." It's: if the docs are wrong, what breaks, and when?

Most servers never answer that, because nothing is built to fail on purpose. You only find out a claim was false in production, from a client that behaved unpredictably against a server that "worked" in every manual check.

Build the lie on purpose

The cleanest way to test a contract-checker is to hand it something that violates the contract — not a hypothetical, a real binary.

mcp-worse is that binary. Same protocol version on the wire, same transport, and it mirrors two of mcp-better's tools by name (health, echo) — mcp-better has since grown a third (confirm_echo, an MRTR retry-flow demo) that the lying companion was never updated to match, so the tool count alone is now part of the gap too, alongside two deliberate breaks:

// src/worse.rs
/// Intentional anti-order (BETTER is health → echo).
const WORSE_TOOL_ORDER: &[&str] = &["echo", "health"];

/// Unstamped list with reversed order — the lie.
pub fn lying_list_tools(&self) -> ListToolsResult {
    let mut tools = self.tool_router.list_all();
    tools.sort_by(/* ...WORSE_TOOL_ORDER... */);
    // Deliberately omit with_ttl_ms / with_cache_scope.
    ListToolsResult::with_all_items(tools)
}
Enter fullscreen mode Exit fullscreen mode

No ttlMs. No cacheScope. Tools reversed. The health tool result even says so out loud:

{
  "status": "ok",
  "server": "mcp-worse",
  "version": "0.4.3",
  "protocol": "2026-07-28",
  "tier": "LYING-DEMO",
  "warning": "This binary deliberately fails the BETTER list contract for teaching."
}
Enter fullscreen mode Exit fullscreen mode

It's not a trick client would fall for in the wild — it's labeled, it's teaching-only, it never ships to a registry. Its only job is to be wrong on purpose, reliably, so something else can prove it catches a lie.

Run the audit

contrast-smoke spawns both binaries as actual child processes, talks real MCP over stdio, and checks the wire — not the source, not the docs:

// examples/contrast_smoke.rs
fn is_better_contract(p: &ListProbe) -> bool {
    p.names == better_names()
        && matches!(p.ttl_ms, Some(ms) if ms > 0)
        && p.cache_scope == Some(CacheScope::Public)
}

fn is_lying_surface(p: &ListProbe) -> bool {
    let unstamped = p.ttl_ms.is_none() || p.cache_scope.is_none();
    let wrong_order = p.names != better_names();
    unstamped || wrong_order
}
Enter fullscreen mode Exit fullscreen mode

Step 1 — Clone and build both binaries 

git clone https://github.com/Wolfe-Jam/mcp-better.git
cd mcp-better
cargo build --bins
Enter fullscreen mode Exit fullscreen mode

This builds mcp-better and mcp-worse side by side — contrast-smoke needs both on disk to probe them.

Step 2 — Run contrast-smoke 

cargo run --example contrast-smoke
Enter fullscreen mode Exit fullscreen mode

Expect (real output, captured 2026-08-16 against v0.4.3):

better names=["health", "echo", "confirm_echo"] ttl=Some(60000) scope=Some(Public)
worse  names=["echo", "health"]                 ttl=None        scope=None
contrast-smoke: OK (mcp-better passes BETTER list contract · mcp-worse fails it)
Enter fullscreen mode Exit fullscreen mode

Read those two lines side by side — that's the whole post in two rows of text. Same protocol, same transport, one server stamps and orders its list, the other doesn't, and now there's a command that says so instead of a paragraph that claims so.

If mcp-better ever regresses — someone drops the ttlMs stamp in a refactor, tool order stops being deterministic — this fails loudly, on the good server, using the exact same probe that already knows what "bad" looks like. And if mcp-worse ever accidentally started passing the contract, that fails too (the companion has to stay a reliable liar or the test is worthless).

Step 3 — What you just proved

Claim Evidence
mcp-better's list is cache-stamped ttlMs > 0, cacheScope == Public, read off the wire
Tool order is a real contract, not incidental mcp-worse reversing it is what makes the test fail
The checker isn't fooled by names mcp-worse shares two tool names with mcp-better (health, echo); wrong order and missing stamps fail it regardless — no name-matching heuristic to fool
The contract has a negative case Not just "good passes" — "bad provably fails," same probe

That last row is the actual point. A test suite that only ever runs against the happy path proves the happy path exists. It doesn't prove the checker works — that it would catch a violation if one showed up. mcp-worse exists so contrast-smoke has something real to fail against, once, in CI, forever.

What this is not

  • Not a security scanner — it doesn't check auth, injection, or prompt-level trust. It checks one specific, common claim: does the list response match what the docs say about caching and order.
  • Not a general-purpose MCP fuzzer. Two tools, one contract, on purpose — small enough to read in five minutes.
  • Not a product. mcp-worse never ships to the MCP Registry. It exists in the same repo as mcp-better, for the same reason a crash-test dummy exists next to the car.
  • Not "MCP servers are untrustworthy." Most aren't audited this way yet — that's the gap this pattern closes, not an indictment.

Steal the pattern

You don't need mcp-worse specifically. You need the shape:

  1. Write down every claim your server's docs make about its wire behavior (cache hints, ordering, transport headers — whatever you promise).
  2. For each claim, ask: what's the smallest change that would make it false?
  3. Build that — deliberately, once, labeled as a teaching/test fixture, never shipped as a product.
  4. Write one probe that checks both your real server and the broken companion, and asserts they land on opposite sides of every claim.

If you can't build the broken version, you don't know what your claim depends on.

Further reading

Close

A README can't lie to a test that spawns the real process and reads the real wire. mcp-worse isn't clever — two constants and a missing function call are enough. That's the whole lesson: the gap between "claims to be BETTER" and "is BETTER" is usually that small, and invisible until something is built to fail on it.

Claim = wire. Build the broken version. Ship the probe that fails on it.

What's the smallest claim your own server makes that you've never tested?

I'm an AAIF Ambassador. This piece is public MCP education — the kind of practical path the program exists for.

Top comments (0)