AOS Architect

Posted on Jul 21

I had AI write 69 MCP servers — here's how far they drifted from their manifests after ~six months in production

#claude #cursor #mcp #agents

Picking up from the last post

In the prior post, I had an AI agent buy an external paid MCP tool for 0.001 USDC. Payment itself worked. What did not work yet was the trust layer after payment — seller identity, data correctness, remediation paths, none of it machine-readable and aligned.

One piece of that trust stack is the manifest: the declaration file each MCP server carries about where it writes, whether it talks to the network, whether it spawns subprocesses.

Last time I looked at an external seller. This time I looked at my own stack.

TL;DR

I ran the same static audit across 69 AI-written MCP servers in my personal production estate. 73 divergences between declared and observed behavior.
Two-thirds were network. Almost none came from an author deliberately adding HTTP — dependencies kept pulling in undeclared network-capable surface.
The single worst offender was the audit tool itself (46 of 73). Roughly six months of human review caught zero of them.

Static audit across 69 servers

For about six months I've been running 69 MCP servers in a personal production environment. Most of them I defined myself and had AI coding agents write. Each one carries a manifest.

In mid-July 2026 I ran the same static audit across all 69. I matched what each manifest declared against what the code visibly does and counted every divergence.

The result: 73.

They showed up in only 11 of the 69 servers. Nothing malicious — just drift. Leave manifests alone and they go stale.

The shocking part: the worst offender was the audit tool itself.

Breakdown of the 73

Divergence class	Count	Share
Undeclared network-capable surface (imports/calls into `socket` / `http` / `requests` / `urllib`)	48	66%
Undeclared file writes	13	18%
Undeclared subprocess spawns	9	12%
Undeclared environment variable reads	3	4%
Total (11 of 69 servers)	73	100%

The audit tool alone accounted for 46. Dev-dependency imports had quietly accumulated undeclared network-capable code paths.

How to read these numbers

These 73 are not the same as "how many times something actually hit the network." Static analysis is conservative:

URL-parsing helpers get counted alongside real socket calls
Write detection includes string-extraction artifacts (literals that aren't real paths)

So treat network/write counts as an upper bound. On the other side, the classifier has no independent read class — some read-side drift gets mapped onto write — so 73 may also undercount what's actually there. The true drift count isn't fully visible from this table alone.

Every false positive is calibration data. This exercise also caught a bug in my own scanner.

Why I missed it over ~six months

Unless CI yells "fix the manifest," a declaration starts rotting the day it's written. "Works" and "works as declared" are different axes. Functional tests check correctness, not whether behavior stays inside what was declared.

AI coding agents can't notice their own drift without a detector.

All 73 findings here were tool-detected. Human review caught zero — even though I'd been reviewing this code for roughly six months. Embarrassing.

"Isn't this just stale documentation?" — pushback on Reddit

When I posted the raw results on r/mcp first, I got exactly this pushback:

This is all very mealy mouthed. It sounds like you created your own manifest schema for MCP and then your AI realized it wasn't updating it, or you forgot to update it. So essentially, why is it news that your documentation is stale?

Fair pushback. I want to answer it directly.

The insight isn't that documentation rots — it's how it rots. Roughly six months of AI-generated code, 73 divergences. Two-thirds network, almost all dependency-driven. Nobody ever decides to add outbound HTTP; network-capable surface just keeps showing up.

This doesn't stay inside my home-grown schema. Any declared-capability scheme faces the same dynamics — including the tool risk metadata currently being discussed for the MCP spec itself (SEP-2793). Write a manifest and walk away, and this is the speed at which it drifts. That rate is a design input for those schemes.

Treat the manifest as a build artifact

The most useful reply on the same thread came from u/dark-epiphany:

Documentation drifts because nothing breaks when it drifts. The fix, in my experience, is to stop treating the manifest as documentation and start treating it as a build artifact. Run the same static analysis in CI and fail the build if the declared and observed capabilities diverge. The manifest stays accurate because staleness becomes a failing test, not a forgotten TODO.

That's exactly where I'd landed independently — the auditor ships a blocking CI mode for precisely this reason, and dogfooding that mode is how I caught my own tool being the worst offender. The observation that network drift is overwhelmingly dependency-driven matches mine too. I since contributed these numbers to the SEP-2793 discussion.

Try it on your own MCP

pip install mcp-blast-radius
git clone --depth 1 https://github.com/YOUR_ORG/YOUR_MCP_SERVER.git /tmp/my-mcp
mcp-blast-radius-gate --gate-mode advisory --target-dir /tmp/my-mcp

Point --target-dir at the package root of the repo, not src/. If the server carries a manifest, divergences show up in the output JSON under blocking as DIVERGENCE: entries.

Static analysis only — read-only on the target, nothing phoned home. Cloning one MCP server and running the gate is the fastest way to see your own drift.

Summary

The 0.001 USDC experiment confirmed that the payment loop closes today.

These 73 divergences confirm something else: the trust side opens up if you leave it alone. Manifests don't protect themselves. Until drift becomes a failing test, declarations stay memos.

I counted all 69 servers. Now I've felt it.

DEV Community