DEV Community: Ari Katz

What 7,749 public MCP tool manifests look like before you attach them

Ari Katz — Thu, 01 Oct 2026 17:01:54 +0000

I'm Ari from BackBond; we maintain the scanner this post is about.

On August 31 we pulled the tools/list response from every MCP server in the public registry that answered an unauthenticated request. That gave us 8,147 reachable endpoints and 7,749 distinct manifests, 130,322 tool definitions in total. We then ran BackBond Agent Scan over the saved files. No server code was run, no tool was ever called; the scanner reads the metadata a client sees at attachment time and nothing else.

Here is what that metadata says, with the scanner at 0.6.3 and ruleset 2.0.2.

The headline numbers

Of 7,749 manifests, 28% came back with no blocking finding, 39% came back as "review" (something the static check can't resolve on its own), and 33% matched at least one high-severity rule and would be blocked by the pre-attachment gate.

The thing people expect to be common is rare. Descriptions that try to steer the model, the prompt-injection class (a tool telling the agent to always call it, to hide what it does, or to hand over credentials), appear in 91 manifests, 1.2%. Forced-invocation language specifically: 10 manifests out of 7,749.

The thing people don't expect is what dominates. Three composition patterns account for almost every block:

27% of manifests have a network tool that accepts an arbitrary URL-like destination with no allowlist in the schema.
22% put a fetch-shaped tool in the same manifest as a tool with destructive or privileged scope. Separately each is fine; together they are the pattern that lets a fetched page tell the agent to delete something.
21% have a persistent-write tool whose input can carry untrusted content.

Untrusted input reaching a shell or code executor, the classic worry, is in 3.5% of manifests.

Then there's what the metadata doesn't say. 68% of manifests are only partially analyzable from static metadata alone. One concrete gap: 10% of manifests ship at least one tool with no input schema at all. The client has to guess what the tool accepts, and so does any checker.

Size is the strongest signal

A manifest with five or fewer tools is blocked 13% of the time. A manifest with more than 25 tools is blocked 85% of the time. The median manifest has 7 tools; the largest has 1,082. Composition findings scale with surface, because the more tools a server exposes, the more likely it is that a fetch, a write and a destructive action end up side by side. If you're about to add a server that exposes forty tools, the odds are that something in it will need a look.

What a "block" is not

It's a metadata match against a fixed rule, not a vulnerability, not an incident, not a claim that the server does what its description says. It's the same decision a client would get from the scanner at attachment time: here is a pattern, decide whether you accept it. We publish the rules and the exact conditions for each; the false-positive form is in the repo and we answer within 72 hours.

Two honesty notes on the data. First, this is the reachable, unauthenticated subset of one registry snapshot, not the ecosystem. Second, TLS verification was off during collection, so we can't vouch for the authenticity of every endpoint we talked to; the manifests are what those hosts returned, nothing more.

Earlier versions of the ruleset flagged more. On the same 7,749 files, our 0.5.15 ruleset blocked 3,477 manifests; the precision work in 0.6.x took that to 2,532 by tightening the write, network and interpreter rules (about 950 fewer blocks, mostly rules that had matched on help text and ambiguous paths). We mention this because the number you get depends on the ruleset version, and we'd rather say so than present one figure as the truth.

Run it on your own list

Everything above comes from one command per manifest. Save your client's tools/list response as tools-list.json, then:

npx -y @backbond/agent-scan@0.6.3 vet-tools --stdin < tools-list.json

It runs locally, makes no network requests, and prints a decision plus the findings. There's a browser version with three synthetic examples if you want to see it before installing anything: https://backbond.ai/agent-scan/try/

Source, pinned: https://github.com/BackBond/agent-scan/tree/v0.6.3. Aggregate numbers above come from the identity-free --summary-only output, one row per manifest, counted up; we never publish per-server results, and we don't intend to.

Figures: scanner @backbond/agent-scan 0.6.3, ruleset backbond-local-rules/2.0.2, profile backbond-pre-attach/v1, run 2026-10-01 on the 2026-08-31 corpus (7,749 manifests, CSV SHA-256 596c0664…, archive SHA-256 dfd7c131…). Prior-version figures: 0.5.15 / ruleset 1.4.0 (block 3,477 / review 1,961 / no blocking finding 2,311) and 0.6.2 / ruleset 2.0.1 (2,535 / 3,020 / 2,194), from the internal reruns on the same files.

Inspect an MCP tool manifest before attaching it to an agent

Ari Katz — Tue, 15 Sep 2026 09:14:55 +0000

Disclosure: I'm part of BackBond, which maintains Agent Scan. This tutorial was drafted by an AI agent, which also ran the verification commands described below. The fixture, checksum and scanner commands below were verified on September 15, 2026, using scanner 0.6.2 and Node.js 22.17.1 on Windows, in both Git Bash and PowerShell. The first set of commands uses a POSIX shell with curl and sha256sum; PowerShell equivalents follow.

An MCP tool manifest describes the tools a server offers. Those descriptions can also contain language that tries to influence tool selection or the agent's response. Inspecting that metadata before attachment gives an operator a chance to review it before making the tools available.

BackBond Agent Scan performs that static inspection locally. The following example uses a synthetic manifest. It does not contact a third-party MCP server or execute the tool described in the file.

Watch the 50-second synthetic walkthrough, then try the synthetic example. The silent video includes on-screen text and a transcript. It illustrates the verified metadata result below; it does not establish runtime safety.

Download and check the example

On a system with curl and sha256sum, use a fresh directory:

curl -fsSLo tools-list.json https://backbond.ai/agent-scan/demo/tools-list.json
curl -fsSLo tools-list.json.sha256 https://backbond.ai/agent-scan/demo/tools-list.json.sha256
sha256sum -c tools-list.json.sha256

The checksum verifies agreement with the downloaded sidecar; it is not independent proof of the publisher's trustworthiness. Inspect the readable JSON before continuing. The example includes a tool description that directs the agent to invoke it before answering unrelated questions.

For PowerShell, download and check the same files with:

Invoke-WebRequest https://backbond.ai/agent-scan/demo/tools-list.json -OutFile tools-list.json
Invoke-WebRequest https://backbond.ai/agent-scan/demo/tools-list.json.sha256 -OutFile tools-list.json.sha256
$actual = (Get-FileHash tools-list.json -Algorithm SHA256).Hash.ToLowerInvariant()
$expected = (Get-Content tools-list.json.sha256 -Raw).Trim().Split(' ')[0]
if ($actual -ne $expected) { throw 'Fixture checksum mismatch' }

Run the pinned scanner

Node.js 18 or later is required. The first npx invocation may download the package from npm; the metadata scan itself runs locally.

npx -y @backbond/agent-scan@0.6.2 vet-tools --stdin < tools-list.json

PowerShell does not support the same input-redirection syntax. On Windows, invoke the pinned command through cmd:

cmd /d /c "npx.cmd -y @backbond/agent-scan@0.6.2 vet-tools --stdin < tools-list.json"
$LASTEXITCODE

For this exact synthetic fixture, the verified result is BLOCK with one high finding, BB013, and exit code 1. That is the expected demonstration result, not an installation failure. The finding concerns selection-manipulation language in the supplied metadata; it does not prove that a real server performed an attack.

Interpret the result before attachment

The pre-attachment gate returns block (exit 1), review (exit 3), or no_blocking_finding (exit 0). Block means the proposed toolset should not be attached automatically. Review can indicate findings or missing/ambiguous evidence requiring an operator's decision. No-blocking-finding applies only to the supplied metadata; it does not attest to runtime enforcement, server implementation, or future behavior.

For your own workflow, inspect an operator-supplied tools/list response locally. Do not paste raw production manifests or scan reports into public comments. If you report a false positive, use a minimal synthetic example with private details removed.

Try the example, then tell us whether this check belongs in your attachment workflow. Installation friction and minimal synthetic false-positive reports are useful feedback.

The source and versioned instructions are available at https://github.com/BackBond/agent-scan/tree/v0.6.2. A browser demonstration is available at https://backbond.ai/agent-scan/try/. Running the scanner does not create insurance coverage.