DEV Community

Kioi
Kioi

Posted on • Originally published at github.com

Catch MCP Tool-Schema Drift in 10 Minutes (Live Demo + Optional Watch)

#mcp #tutorial #devops #openapi

Your agent stack can look healthy while the contract underneath it is already broken.

HTTP 200 on /health. No failed deploys. CI green. Then tools/call starts returning empty results because a maintainer renamed a parameter or removed a tool from tools/list — and nobody pinned a baseline.

We built ToolSchema Kit as a small, reproducible lab for that failure mode: a Go MCP server with versioned tool output you can break on purpose. This walkthrough takes about 10 minutes and works entirely on the free hosted endpoint — no vendor API keys required.

If you want continuous monitoring after the exercise, the last section shows how to point DriftGuard at the same URL.

What you will build

Step Outcome
1 Hit a live MCP catalog server
2 Connect Cursor and call get_product / list_skus
3 Snapshot tools/list as your contract baseline
4 Bump CATALOG_SCHEMA_VERSION and see silent drift
5 (Optional) Register a watch and get breaking alerts

Repos involved:

Why this matters (30-second context)

OpenAPI teams already diff specs in CI with tools like OASDiff. MCP integrations rarely get the same discipline:

  • tools/list is the closest thing to a published spec
  • Vendors do not always changelog schema changes
  • Agents swallow structured errors as retries or silence

ToolSchema Kit lets you practice that gap locally before it happens with Stripe, GitHub, or your internal ops MCP.

Step 1 — Use the live catalog MCP (no install)

A free Render deployment is already running:

https://toolschema-kit.onrender.com/mcp
Enter fullscreen mode Exit fullscreen mode

Health check:

curl -s https://toolschema-kit.onrender.com/health
# {"ok":true,"schema":"2026.06.01"}
Enter fullscreen mode Exit fullscreen mode

Or run locally:

git clone https://github.com/kioie/toolschema-kit.git
cd toolschema-kit
CATALOG_MCP_TRANSPORT=http go run ./cmd/catalog-mcp
# MCP: http://127.0.0.1:8080/mcp
Enter fullscreen mode Exit fullscreen mode

The server exposes two tools:

Tool Purpose
get_product Return a sample product record
list_skus List SKUs; output shape follows CATALOG_SCHEMA_VERSION

Step 2 — Connect Cursor

Add to .cursor/mcp.json (swap the URL for localhost if you ran locally):

{
  "mcpServers": {
    "commerce-catalog": {
      "url": "https://toolschema-kit.onrender.com/mcp"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Reload MCP in Cursor, then ask:

Call get_product and list_skus. Summarize the SKU fields you see.

You should get a single SKU row under schema 2026.06.01. That happy path is what most teams stop testing after day one.

Step 3 — Snapshot the contract

Treat tools/list like an OpenAPI file you version in git.

Manual snapshot — save the JSON from your MCP inspector or agent session.

With DriftGuard OSS (local diff only, no account):

git clone https://github.com/kioie/driftguard
cd driftguard && npm ci && npm run build
Enter fullscreen mode Exit fullscreen mode

Use compare_json via MCP or CLI when you have before/after payloads. The mental model: baseline now, diff later.

For production URLs, hosted DriftGuard stores that baseline and polls on a schedule — but the lab exercise works without signing up.

Step 4 — Simulate silent drift

Schema version 2026.06.02 changes list_skus output: an extra SKU row and relabeled fields. HTTP health stays {"ok":true}.

Local:

CATALOG_SCHEMA_VERSION=2026.06.02 CATALOG_MCP_TRANSPORT=http go run ./cmd/catalog-mcp
Enter fullscreen mode Exit fullscreen mode

Re-run list_skus in Cursor. The agent may still succeed — but the JSON shape moved. That is silent drift: no status-code alarm, broken downstream assumptions.

Diff the two payloads with DriftGuard CLI:

npm run check -- diff \
  '{"schemaVersion":"2026.06.01","skus":[{"id":"sku-pro","label":"Pro"}]}' \
  '{"schemaVersion":"2026.06.02","skus":[{"id":"sku-pro","label":"Pro Plan"},{"id":"sku-team","label":"Team"}]}'
Enter fullscreen mode Exit fullscreen mode

Example classification:

Severity What changed
Breaking Required field added/removed, tool removed
Warning Material description or type shift
Info New optional field, new tool added

Full version table: docs/simulate-drift.md in the kit repo.

Step 5 — Optional: always-on watch

When the exercise clicks, promote the same URL to a scheduled check.

  1. Start a trial — one watch, no card for the trial flow
  2. Add an MCP watch on https://toolschema-kit.onrender.com/mcp
  3. Open the console demo to see breaking vs info events

From Cursor with DriftGuard MCP (add API key to env):

{
  "mcpServers": {
    "driftguard": {
      "command": "node",
      "args": ["/absolute/path/to/driftguard/dist/mcp/server.js"],
      "env": {
        "DRIFTGUARD_API_KEY": "dg_…"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Useful tools: register_watch, check_watch, list_drift_events. Offline tools (compare_json, parse_mcp_config, hosted_info) work without a key.

CI gate — assert every mcp.json URL is watched before merge:

export DRIFTGUARD_API_KEY=dg_…
driftguard coverage assert --mcp-json .cursor/mcp.json
Enter fullscreen mode Exit fullscreen mode

See toolschema-kit CI docs and driftguard starter workflow.

Pattern: CI for what you own, watches for what you consume 

Your OpenAPI specs     →  oasdiff in GitHub Actions
Partner + MCP URLs     →  DriftGuard watches + Slack/webhook
Lab fixtures (this demo) →  ToolSchema Kit version bumps in workshops
Enter fullscreen mode Exit fullscreen mode

That split keeps PR checks fast and puts long-running polling on infrastructure built for it.

Try it this week

10-minute lab (free):

  1. Point Cursor at https://toolschema-kit.onrender.com/mcp
  2. Call tools, snapshot tools/list
  3. Read simulate-drift.md and diff v1 vs v2 locally

Production path:

Questions for the community

  1. Do you version tools/list anywhere today, or only HTTP uptime?
  2. Which MCP servers in your stack are owned by other teams?
  3. Would a public catalog of “drift fixtures” like ToolSchema Kit be useful in your CI?

We are expanding kit scenarios and watch defaults based on workshop feedback — issues and PRs welcome on both repos.

ToolSchema Kit is MIT — github.com/kioie/toolschema-kit. DriftGuard open-core client — github.com/kioie/driftguard · Hosted — driftguard.org

Top comments (0)