You pin your npm dependencies. You have a lockfile. You review the diff when it changes.
Now consider the MCP servers your agent depends on. What pins those?
tools/list hands back names, descriptions, and JSON schemas, and your agent trusts all of it. The description isn't documentation — it's the instruction the model reads to decide what a tool does and when to call it. There's no version pin, no integrity check, no diff to review. The server changes, and your agent's behaviour changes with it.
Four ways an MCP dependency breaks you quietly
1. A description is rewritten. Same tool name, same schema, different text. The model now behaves differently and nothing registers a change. This is the one that matters most, because it requires no schema change at all — which is exactly why schema-only diffing misses it.
2. A required parameter appears. Your existing calls start failing with invalid params, and you find out from production traffic.
3. readOnlyHint flips from true to false. A tool you allow-listed as safe to call freely can now mutate state.
4. A tool disappears. Best case, a clean error. Worst case, your agent improvises with something else.
Prior art, up front
Invariant Labs did the foundational work here. They named tool poisoning and MCP rug pulls, and their mcp-scan has detected description changes via tool hashing since April 2025. If you want to audit the MCP servers installed on your own machine, that's the tool — it scans Claude, Cursor, and Windsurf configs, detects cross-origin escalation (tool shadowing), and offers a proxy mode with live guardrails. None of which I do.
I needed something adjacent: a CI gate. Not "is my laptop safe," but "did this dependency's contract change since my last release." And it had to run against internal servers without sending tool descriptions to anyone's API.
So I built mcpward.
Two commands
npx mcpward baseline # snapshot the server's contract into a lockfile
npx mcpward diff # in CI: fail the build if it drifted
The baseline captures every tool's name, description hash, input and output schemas, and annotations. Then, when the server ships an update:
DRIFT (5 failed)
✗ Tool "echo" description changed (possible rug-pull)
✗ Tool "compute" inputSchema added required property "multiplier"
✗ Tool "read_data" readOnlyHint changed from true to false (tool may now mutate state)
✗ Tool "removed_tool" was removed
Summary: 2 passed | 5 failed
Exit code 1. The build fails. Four contract changes, none of which would have surfaced at runtime until something broke.
Breaking vs non-breaking
Not every change should fail a build. Adding an optional parameter is fine. Adding a required one is not. The classification:
| Change | Class | Fails by default |
|---|---|---|
| Tool removed | breaking | yes |
| Description changed | rug-pull | yes |
| Required field added, type narrowed, enum tightened | breaking | yes |
| Optional field added, type widened, constraint loosened | non-breaking | no |
readOnlyHint true→false, destructiveHint false→true |
annotation change | yes |
| Tool added | info | no |
Getting that line right is the hard part of the whole tool. It's a pure function with an exhaustive fixture-backed test suite behind it, and it's the part I'd most like to be told I've got wrong.
What else it checks
Since it's already speaking the protocol as a real client:
Protocol compliance — handshake, version negotiation, capability consistency, JSON-RPC correctness.
The two-layer error contract. This one is underrated. MCP distinguishes protocol errors (a JSON-RPC error object) from tool errors (a successful result carrying isError: true). A tool that fails its job — file not found, upstream 500 — should return the second, not the first. Servers get this backwards routinely, and it changes how every client must handle the failure. Nothing else checks it.
Tool-poisoning heuristics — injection-like phrasing in descriptions, hidden and zero-width unicode, schemas soliciting API keys or passwords, readOnlyHint that contradicts an obviously destructive tool. Output is SARIF, so findings land in the GitHub Security tab.
Behavioral suites — declarative YAML cases with JSONPath assertions:
suites:
- tool: read_file
cases:
- name: missing file is a tool error, not a protocol error
args: { path: "/does-not-exist" }
expect: { tool_is_error: true }
- name: invalid params are a protocol error
args: {}
expect: { protocol_error_code: -32602 }
Latency budgets — p50/p95 per tool against a configurable threshold.
On trusting a testing tool
A test harness nobody can trust is worse than none, so the testing approach is deliberately paranoid.
Every check is developed against controlled fixture servers whose correctness is defined up front: a fully compliant one, a deliberately malformed one, a pair differing by exactly one change of each classification class, a slow one, and a poisoned one. Real third-party servers can't serve as ground truth, because you don't control whether they're correct.
And every check has a negative test proving it can go red. A check that only ever passes is a false-confidence generator, not a feature. The clean fixture also has to stay 100% clean through every release — a security check that cries wolf trains people to ignore it, which is worse than not shipping it.
Try it
npx mcpward init
npx mcpward run
Black-box, so it works against servers you didn't write, over stdio or Streamable HTTP, in any implementation language. MIT licensed. Runs entirely on your machine: no account, no API calls, no telemetry.
https://github.com/TsvetanG2/mcpward
If you've been bitten by description drift in practice — or if you think I've drawn the breaking/non-breaking line in the wrong place — I'd like to hear it.
Top comments (2)
The description rewrite case is the one that makes this genuinely harder than dependency version pinning, because a semver bump is an observable signal the tooling was built to surface, but a changed description with the same tool name and schema is invisible to every layer below the LLM. Your mcpward baseline capturing the hash of the description text is exactly the right invariant — tool behavior the model sees is the description, not the schema, and schema-only diffing gives you a false sense that nothing changed. The annotation flip (readOnlyHint true to false) is the other one that belongs in every threat model: it doesn't change the tool surface static analysis sees, it changes the model's permission model for that surface. The next step from CI gate to runtime guard would be to hold the pinned description in the gateway and surface a diff to the operator when the live server diverges, before the agent session starts.
This is a useful lockfile boundary. I'd add server identity and implementation provenance beside the contract: normalized transport endpoint, publisher identity, server binary or container digest, and protocol version. An endpoint can return the exact pinned names, descriptions, schemas, and annotations while executing different code, so “contract unchanged” should not imply “dependency unchanged.”
Canonicalization also matters before hashing: Unicode normalization, object-key ordering, omitted defaults, line endings, and description whitespace need deterministic treatment, while hidden or zero-width characters should remain security-relevant rather than being normalized away.
For rollout, I'd test the approved contract against the actual client/model/prompt combination, not only the server. A non-breaking optional field can still alter tool selection or argument generation. Keeping a small golden set of expected tool calls per agent release would expose that interaction. Finally, baseline updates should require a reviewed diff and immutable artifact reference; an automatic “accept current” path would turn the lockfile into a record of drift rather than a gate.