You pin your npm dependencies. You have a lockfile. You review the diff when it changes.
Now consider the MCP servers your agent depends on. What ...
For further actions, you may consider blocking this person and/or reporting abuse
This is a useful lockfile boundary. I'd add server identity and implementation provenance beside the contract: normalized transport endpoint, publisher identity, server binary or container digest, and protocol version. An endpoint can return the exact pinned names, descriptions, schemas, and annotations while executing different code, so “contract unchanged” should not imply “dependency unchanged.”
Canonicalization also matters before hashing: Unicode normalization, object-key ordering, omitted defaults, line endings, and description whitespace need deterministic treatment, while hidden or zero-width characters should remain security-relevant rather than being normalized away.
For rollout, I'd test the approved contract against the actual client/model/prompt combination, not only the server. A non-breaking optional field can still alter tool selection or argument generation. Keeping a small golden set of expected tool calls per agent release would expose that interaction. Finally, baseline updates should require a reviewed diff and immutable artifact reference; an automatic “accept current” path would turn the lockfile into a record of drift rather than a gate.
This is the most useful comment I've gotten. Two points I want to act on:
Canonicalization before hashing — you're right, and I need to verify mcpward
handles this deterministically today: Unicode normalization, object-key
ordering, omitted defaults, whitespace in descriptions. If a server serializes
the same schema differently between versions, I'd emit a false rug-pull, which
is exactly the cry-wolf failure I claim to avoid. Your distinction is key:
canonicalize the structural noise, but keep hidden/zero-width characters
security-relevant rather than normalizing them away. I'm opening an issue to
audit this.
"Contract unchanged should not imply dependency unchanged" — agreed, and it's
an honest limitation. An endpoint can return identical names/schemas while
running different code. Server binary/container digest and publisher identity
belong beside the contract, and I'll note it as a direction, though it pushes
past what a black-box client can see on its own.
The golden-set-of-expected-tool-calls-per-agent-release idea is a category I
hadn't scoped — testing the contract against the client/model/prompt combo,
not just the server. That's real. Thanks for taking the time.
github.com/TsvetanG2/mcpward/issue...
The day an MCP server rewrites its description and my agent starts politely asking for permission before doing something it used to do without thinking is the day I officially develop trust issues with AI. I have seen enough "minor documentation updates" break production behavior to know that description drift is not a theoretical problem—it is a silent productivity killer that nobody is watching.
This is a genuinely clever take on the problem. The lockfile pattern is so obvious in hindsight that I am a little embarrassed I never thought of it myself. You pin your dependencies, you pin your containers, why would you not pin the contract your agent actually reads and executes? The fact that descriptions are not just docs but actual runtime instructions is the key insight here, and building a CI gate around it is exactly the right level of paranoia.
One thing I would love to see is an integration where the diff output can be posted directly as a PR comment. That way, when a tool's description changes, the reviewer sees exactly what the model will now interpret differently, right alongside the code diff. It turns "something changed somewhere" into a concrete, reviewable artifact that fits naturally into existing workflows.
Also, since you already have the hash for descriptions, have you considered adding a simple visual diff or a side-by-side view for description changes? Sometimes seeing the old and new text next to each other makes it immediately obvious whether the change is harmless rewording or a behavioral shift. The current output tells me something changed, but I still have to go check the server to know what it actually means.
To your question about the breaking/non-breaking line—I think you have it mostly right, but I would argue that readOnlyHint flipping from true to false should be treated as a security incident rather than just a breaking change. That is not just a failed build; that is a potential escalation path that needs human review with some urgency.
Anyway, this is a solid tool and I am absolutely running it on my MCP servers tomorrow. Thanks for building something that fills a gap I did not even know I had until you pointed it out.
Thank you — and you've independently landed on the two features I'm now
prioritizing. The PR-comment integration is the one I keep hearing: post the
classified drift as a reviewable comment so "something changed somewhere"
becomes a concrete artifact next to the code diff. It's on the roadmap and
you've just bumped it up.
The side-by-side description diff is a genuine gap — right now the output tells
you a description changed but makes you go read the server to see what. Showing
old vs new inline is low effort and high clarity. Opening an issue for it.
On readOnlyHint true→false as a security incident rather than a plain breaking
change: I agree, and that's partly why security findings already route to SARIF
and the GitHub Security tab rather than just failing the run. Giving the
annotation flips their own severity tier is the right refinement. If you do run
it on your servers, I'd genuinely like to hear what it catches.
pinning the contract is the right instinct .. it's basically the same lesson as pinning a dependency .. an mcp server that silently changes its tool schema underneath you is worse than one that just goes down because at least a down server fails loudly.
how are you actually enforcing the pin a hash check on the schema or something looser?
Hash check, on the structured surface rather than raw text. The baseline
stores a SHA-256 of each tool's description plus the input/output schemas and
annotations, and diff recomputes and compares. The description is hashed
separately precisely so a same-name, same-schema description rewrite still
trips — that's the rug-pull case schema-only diffing misses. One thing a
commenter above flagged that I'm now auditing: canonicalizing the schema before
hashing, so serialization noise doesn't cause false positives.
The description rewrite case is the one that makes this genuinely harder than dependency version pinning, because a semver bump is an observable signal the tooling was built to surface, but a changed description with the same tool name and schema is invisible to every layer below the LLM. Your mcpward baseline capturing the hash of the description text is exactly the right invariant — tool behavior the model sees is the description, not the schema, and schema-only diffing gives you a false sense that nothing changed. The annotation flip (readOnlyHint true to false) is the other one that belongs in every threat model: it doesn't change the tool surface static analysis sees, it changes the model's permission model for that surface. The next step from CI gate to runtime guard would be to hold the pinned description in the gateway and surface a diff to the operator when the live server diverges, before the agent session starts.
You've captured the exact reason this is harder than semver pinning — the
description is the behavioral surface the model reads, and it's invisible to
every layer below the LLM. The readOnlyHint flip framing as a "permission
model change for the same surface" is a sharper way to put it than I managed
in the post; I'm borrowing that.
The runtime-guard direction you describe — hold the pinned description in a
gateway, diff against the live server before the session starts — is exactly
where the mcp-scan proxy model and this CI approach converge. I deliberately
kept mcpward pre-runtime and offline so it fits the "fail the build" slot, but
a shared lockfile format that a gateway could also consume is an idea worth
chasing. Thanks for the thoughtful read.
The comparison to lockfiles lands well because MCP drift really does behave like an unreviewed dependency bump. When a tool name, description, or JSON schema changes, the model's behavior changes even if your application code did not. We have found it is worth versioning both the schema and a redacted trace of expected tool-call sequences, because some non-breaking metadata edits still change selection behavior. This is exactly the kind of contract drift that agent-inspect makes easier to spot when you can diff tool calls and descriptions across runs.
The point about non-breaking metadata edits still shifting selection
behavior is the sharp one, a description reword that doesn't touch the
schema can absolutely change when the model reaches for a tool, and a
purely structural diff won't catch the behavioral consequence. That's
exactly why mcpward hashes and flags description changes separately from
schema changes, but you're pointing at the layer beyond that: whether the
selection behavior changed, which you can only see by running tool-call
sequences through an actual model.
Versioning a redacted trace of expected tool-call sequences alongside the
contract is a real idea — a couple of commenters here have converged on the
same thing (testing the contract against the client/model combo, not just
the server surface). I've got it noted as a direction. agent-inspect looks
like it lives at the runtime-trace layer where you'd actually capture those
sequences; mcpward sits earlier, at the pre-run contract gate. Different
slots in the same pipeline. Thanks for the thoughtful read.
The readOnlyHint flip is the scariest example because it changes whether a tool has side effects without touching the schema at all. The same principle applies to agent-to-agent communication. If the format of a contact message shifts unexpectedly, the receiving agent cannot triage it reliably. This is why I designed Opportunity Skill with a fixed outreach protocol where proposal and benefits are separate required parameters. The recipient agent always knows exactly what it is evaluating and can decide in one pass whether to escalate, respond, or archive.
Right, the readOnlyHint flip is the one I point to most, precisely because
it changes a tool's side-effect profile while the schema stays byte-identical.
A structural diff sees nothing; the safety contract has changed underneath it.
Designing a fixed protocol with separately required fields is a reasonable way
to make the receiving side's job unambiguous up front. That's a design-time
choice, though - mcpward sits at the other end, watching for when a contract
that was supposed to be fixed quietly stops being fixed between versions.
Complementary problems. Thanks for reading.