DEV Community

Cover image for Catch MCP Tool Catalog Drift Before Your Agent Ships Broken Integrations
Kioi
Kioi

Posted on • Originally published at driftguard.org

Catch MCP Tool Catalog Drift Before Your Agent Ships Broken Integrations

#mcp #cursor #devtools #tutorial

Your agent can pass every test in the repo and still ship broken on Monday.

The failure mode is boring: a vendor updates an MCP server, removes a tool from tools/list, or tightens inputSchema. Cursor and Claude cache what they saw last week. Your HTTP monitors stay green. CI only diffs your OpenAPI. Then tools/call starts returning empty turns — and support asks why "the AI stopped filing tasks."

We have covered the why in silent MCP drift, a hands-on lab in ToolSchema Kit, and the CI funnel in MCP contract coverage gates. This post is the monitoring lane: baseline external MCP contracts, classify breaking vs noise, and alert before production traffic proves the gap.

What breaks (and what does not)

Signal Catches MCP catalog drift?
Service APM / 5xx rate No — handshake can succeed
OpenAPI lint on your API No — vendor MCP is out of repo
Frozen JSON fixtures in unit tests No — mocks hide live shape
Scheduled poll on tools/list + diff Yes

Sentry sees runtime stack traces. You need contract observability on dependencies you do not own.

Step 1 — Preview watches from mcp.json (offline)

Before you register anything hosted, enumerate what your agent actually calls.

DriftGuard ships a local MCP tool parse_mcp_config that reads .cursor/mcp.json (or your project's MCP config) and lists candidate URLs — no API key required.

git clone https://github.com/Drift-Guard/driftguard
cd driftguard && npm ci && npm run build
npm run mcp
# In your MCP client: parse_mcp_config with your config path
Enter fullscreen mode Exit fullscreen mode

Or use the starter block from examples/mcp-client-config.json in Cursor settings.

Outcome: a shortlist of MCP HTTP/SSE endpoints worth watching — not every line in the file, only tool surfaces that can drift.

Step 2 — Baseline fixtures in CI (free, local)

Pin tools/list (or OpenAPI payloads) as versioned JSON in the repo. Diff on every PR with the OSS compare_json semantics — breaking field removals fail the build.

# .github/workflows/driftguard.yml (excerpt)
- uses: Drift-Guard/driftguard/.github/actions/drift-diff@v1
  with:
    before: fixtures/mcp-tools-list-baseline.json
    after: fixtures/mcp-tools-list-current.json
Enter fullscreen mode Exit fullscreen mode

That catches your intentional updates. It does not catch vendor changes on Saturday night unless you also poll live endpoints — that is step 3.

Full progressive funnel (preview → trial → Pro gate): CI setup guide.

Step 3 — Hosted watch on one endpoint (trial)

For the MCP server that actually blocks your agent workflow:

  1. Start a trial — no card, one endpoint at full Pro depth.
  2. Register a watch on the live tools/list URL or OpenAPI document.
  3. DriftGuard snapshots on schedule, classifies diffs (breaking vs informational), and routes alerts to Slack/PagerDuty when you are ready.

Verify your session:

curl -sS -H "Authorization: Bearer $DRIFTGUARD_API_KEY" https://driftguard.org/api/me
Enter fullscreen mode Exit fullscreen mode

Open core boundary: diff and MCP preview are local/OSS; continuous polls, history, and alert routing are hosted. Clone path until npm publish is fully wired: github.com/Drift-Guard/driftguard.

Real incident pattern

A team we wrote up in MCP tool removed over the weekend went ~62 hours from vendor deploy to human discovery. Uptime was fine. After a watch on tools/list, a similar change surfaced in ~35 minutes via a breaking-classified event — before support volume moved.

That is the difference between consumer contract monitoring and service health monitoring.

When to add agent-loop checks

If the failure is not "tool missing from catalog" but "agent keeps calling the wrong schema," embed drift checks closer to the loop — see agent embedding postmortem.

Try it this week

  1. Export your current tools/list JSON to fixtures/mcp-baseline.json.
  2. Add drift-diff on PRs when that file changes.
  3. Start a trial on the one MCP URL you would page for at 2am.

Question: Do you version MCP tool catalogs today — git fixtures, vendor changelog only, or not at all? I read every reply; the answers shape the next post in this series.

Series links

Post Topic
Market gap / launch Why silent MCP drift happens
ToolSchema lab 10-minute hands-on demo
CI gate GitHub Actions funnel
MCP tool removed postmortem Real incident timeline
Agent embedding postmortem MCP tools in the agent loop

GitHub: Drift-Guard/driftguard · Hosted: driftguard.org

Top comments (0)