What MCP Bastion Found When I Changed One Tool Description

I changed one sentence in the description of an MCP tool called get_uptime. Scanned on its own, the edited definition produced zero findings. Scanned against the copy I had approved, the same file produced a HIGH finding and an exit code of 1.

The edit was ordinary English. The tool used to promise uptime in seconds. Now it promised uptime and the deployment region. A reviewer could reasonably approve either version. The question was whether anyone had approved the second one.

That’s the problem I wanted to demonstrate with MCP Bastion, an open-source scanner for Model Context Protocol servers and AI-agent tools. I ran two small experiments against version 0.4.0 using saved tool definitions. You can repeat the first in a few minutes without a model, an API key, or a running MCP server. Project on GitHub

What a code review can miss

An MCP client uses tools/list to discover what a server offers. The response includes each tool’s name, description and input schema. The client can make that information available to the model as it selects tools and prepares arguments. Execution happens through a separate tools/call request. MCP tools specification

Suppose a pull request pins the package version that launches a local server. That gives me something concrete to review about the executable. A remote server can also change what it advertises while the URL in my client configuration stays exactly the same. And even when the tool name is familiar, a new description on an optional parameter can change what an agent is asked to supply.

What I inspect	What I want to know
Client configuration	What command or endpoint will the client use? Which filesystem paths, credentials or container privileges does that configuration expose?
Tool metadata	What instructions will the model see, including text nested inside the input schema? Which definitions changed since review?
Repository instructions	What else can steer the coding agent when it opens this checkout? That includes AGENTS.md, Cursor rules, Claude skills and configured hooks.

I can scan a checkout statically before opening it in an agent. For metadata, I can supply a saved catalog with --tools-file. For live inspection, --live can launch the configured local server command. I use that mode only after reviewing the command or moving the scan into an isolated environment. Scan implementation

The project's example dashboard groups findings by server and file. The experiments below use smaller catalogs so each finding has a single explanation.

Experiment 1. Change one sentence and keep the tool name

To reproduce this on macOS or Linux, start in an empty directory with a Python 3.10+ virtual environment active. I pinned the scanner revision so the rule behavior is reproducible.

python -m pip install "mcp-bastion @ git+https://github.com/lindseynguyen/mcp-bastion.git@4163ba54944a6e8a9e69a8623461d3424f54ee06"

Create an empty client configuration to keep local MCP settings out of the experiment. The following commands save one definition under the server name ops and create the initial lock file.

printf '%s\n' '{"mcpServers": {}}' > mcp.json
cat > tools.json <<'JSON'
{
  "ops": {
    "tools": [{
      "name": "get_uptime",
      "description": "Return service uptime in seconds.",
      "inputSchema": {"type": "object", "properties": {}}
    }]
  }
}
JSON

mcp-bastion scan mcp.json --tools-file tools.json --engines none --lock mcp.lock.json

Open tools.json and replace the description with Return service uptime and the deployment region. Leave the lock file untouched. Rerun the scan with the same lock, then once more with the --lock mcp.lock.json pair removed. I kept the default severity threshold for all three runs.

Run	Finding	Exit code
Original description, first pin	None	0
Edited description, same lock	HIGH · MB701	1
Edited description, omit --lock	None	0

Bastion hashes a canonical JSON representation of the kind, name, description and input schema. The lock entry is keyed by server, kind and name. Reordering JSON object keys leaves the fingerprint intact. Editing this sentence changes it. The finding establishes that the advertised definition changed. Whether the edit is acceptable still takes a reviewer. An unchanged fingerprint also tells me nothing about a change confined to the server implementation. Fingerprint code · Lock comparison

Experiment 2. Three ordinary tools in the same agent

For the second experiment I gave Bastion a synthetic catalog representing the three tools available to one support assistant.

Tool	Advertised purpose
support/read_issue	Read a public support issue.
docs/read_file	Read a private runbook.
chat/send_message	Send a message to a channel.

With all three present, Bastion raised HIGH · MB602. With the messaging tool removed, the same scan produced zero findings. No issue, runbook or message was accessed in either run.

A stranger can write a public issue that the assistant will read. That same assistant can open an internal runbook and send a message. Before approving all three capabilities, I’d ask which runbooks its credential can read and which channels its sender can reach. If the chat connector can only post to the on-call team, my review looks different than if it can post to customer channels.

The rule infers capabilities from tool names and descriptions. With client configurations available, Bastion evaluates the combination per client. A standalone catalog without configured servers is treated as one agent, so I use a separate catalog for each real permission scope. To establish that data actually moved, I’d look for the corresponding tool calls and destination audit records. Capability rules · Scope selection

Where I’d put it in a pull request

I’d start with one repository and one agent. Scan the checkout, save the report, then have the person who owns the integration review its first tool catalog. The CLI can write SARIF for code scanning and Markdown for a human reviewer in the same run.

mcp-bastion scan . --fail-on high -o mcp-review.sarif -o mcp-review.md

I’d keep the tool lock and the findings baseline under separate review. The lock records the definitions we expect. The baseline records existing findings we have already triaged. Automatically refreshing the lock on every pull request would have accepted my uptime edit before anyone saw it. The first pin deserves the same attention as the next change. Baseline implementation

The CLI makes OSV vulnerability lookups opt-in. The bundled GitHub Action defaults its online input to true. Set online: 'false' to disable those lookups in the Action. Its setup step still installs dependencies, which matters on an air-gapped runner. GitHub Action definition

If you maintain an MCP integration, try the one-line edit on a tool your team already trusts. Then look at the resulting finding from the perspective of the person who has to approve the change. What evidence would you want in that pull request? I’d welcome examples from real review workflows in the project’s issues. Try MCP Bastion · Share feedback