A Number We Withdrew, and What Still Stands: 12 AI Frameworks, 87 Vulnerabilities

The Number We Withdrew, and What Still Stands: 12 AI Frameworks, 87 Vulnerabilities

We recently pulled a statistic from our public articles because we could not re-derive it from an auditable record. Rather than quietly edit, we appended a correction notice. This post is the full story: what we withdrew, why, and the numbers that still stand because they are measured and re-verifiable.

The number we withdrew

In our article on AI agent security audits we stated that a large share of MCP servers — "60%+" — lack basic access control on tool execution. A reader at Humanbound cited the figure in a comment as the fact that resonated most with them. That is exactly the moment a bad number does its damage: it gets repeated in good faith.

When we went to re-derive the claim, we could not. The scan runs, datasets, and reports we hold do not produce that number by any auditable path. Ecosystem-wide prevalence claims require a defensible sampling method; ours did not have one. So per our data-integrity policy, the claim was withdrawn, and the article now carries a correction notice marked "Withdrawn — Pending independent verification" until it can be re-published from a re-derived, auditable report.

Why transparency about a bad number matters more than the number

Two things made the withdrawal easy to decide. First, our published claims are the standing basis of our credibility with the readers who matter — if one number is quietly wrong, everything else becomes negotiable. Second, security is a field where unverified statistics get weaponized: a plausible percentage becomes a quote, the quote becomes a dashboard, and the dashboard becomes a procurement criterion.

We had a choice: leave the figure up because it was rhetorically convenient, or pull it because it was not verifiable. We chose the second. The correction notice is now part of the record, and this post is the explicit part of that record.

What still stands — measured, not estimated

The withdrawal did not touch the claims that are directly re-verifiable. These are the numbers we will stand behind, because each one traces to a concrete record:

Notice what the two groups have in common and what they do not. The claims that stand are the ones where we can point to a record we produced: a scanner run, a trace dataset, a benchmark harness. The claim that fell was the one where we repeated an ecosystem statistic without owning the measurement. That asymmetry is not an accident — it is the difference between reporting what we did and guessing about a population we never sampled.

• 12 AI frameworks, 87 vulnerabilities. From the CCS v4.2 scan campaign across frameworks, an actual scan of 12 mainstream frameworks. The most common finding was a protocol-level gap: the MCP readOnlyHint signal is defined but not enforced, and every framework built on the shared SDK inherits it.
• 1,730+ verified findings in the CCS v4.2 corpus — the empirically verified count from those scans.
• Three real MCP CVEs, each on NVD: CVE-2026-2287 (CrewAI, CVSS 9.8), CVE-2026-42271 (LiteLLM, CVSS 8.7, actively exploited and on CISA's KEV list), and CVE-2026-12957 (Amazon Q Developer, CVSS 8.5). We wrote a walkthrough of all three with a live detection demo.
• More than 80,000 API traces collected from real provider calls across 13 LLM providers, with a 20,000-trace subset published alongside the benchmark dataset.
• CCS verifier performance of P50 below 10µs and P99 below 25µs, benchmarked over 50,000 iterations per scenario.

The distinction that keeps our claims honest

The lesson we are holding ourselves to is a separation between two kinds of statements. A measured finding — "we scanned these repositories and found these vulnerabilities" — is re-verifiable by anyone who runs the same scanner. An ecosystem prevalence estimate — "x% of all MCP servers" — requires a sampling methodology and an explicit error budget, and it should be labeled as an estimate, not a fact.

We under-built the second category and overstated it as the first. That is the mistake the correction is about.

We have since added a step to our publishing workflow: any number that enters a public post must carry a source line naming the record it came from, and ecosystem estimates get labeled as estimates with their basis stated. It is slower to write and reads less dramatic, and that is the point. If you are evaluating any security vendor's claims about the AI ecosystem, ask for the re-derivation: show me the scan, the dataset, or the method that produced this number. A vendor who cannot show it should be treated the way we treat ourselves.

For teams adopting MCP servers, the practical takeaway is simpler than it sounds: treat any ecosystem statistic you hear as a question, not an answer. Ask where it came from, how the sample was drawn, and whether you could re-derive it before your next security review depends on it. The habits that catch a bad number in someone else's slide deck are the same habits that catch one in your own.

Where the re-verifiable work lives

The checks that underpin the measured numbers are the runtime verification layer formalized in draft-correctover-ccs-02, which defines how an agent produces tamper-evident evidence of the checks that ran on each call. The CCS scanner that produced the 12-framework campaign runs as a pay-per-use check at ¥0.7/scan (the ¥7 bundle covers 10 scans). If you run it against a server you already trust, you get findings you can verify — which is the only kind we will publish.