DEV Community

Cover image for The AI SDK's MCP drift check doesn't hash readOnlyHint: what that would miss, counted on 14,009 servers
Alex
Alex

Posted on Edited on Originally published at modelmarket.dev

The AI SDK's MCP drift check doesn't hash readOnlyHint: what that would miss, counted on 14,009 servers

Since version 7.0.19 the Vercel AI SDK can fingerprint the tools of an MCP server. You call fingerprintTools when you first trust a server, store the result, and on later connects detectToolDrift tells you which tools were added, removed or changed. The package it ships in, ai, was downloaded 48.3 million times in the week of 12–18 November 2026.

The fingerprint covers three members of a tool definition: the description, the input schema and the title. Annotations are not among them. A tool that said readOnlyHint: true on the day you reviewed it can say readOnlyHint: false and destructiveHint: true later, and the drift check returns three empty lists.

We run HISTOR, a public record of what MCP servers advertise and when it changes, so we counted how often that happens. Between 23 October and 16 November 2026, 524 tools on 189 servers switched readOnlyHint from true to false. For 176 of those tools the fingerprint of the tool would not have moved. For 91 of them, on 32 servers, no tool on the server changed its fingerprint in that read, so the check would have had nothing to report.

The gap itself is not our find. NerdLevelTech showed it in September on a purpose-built server, for the output schema as well, and published a fingerprint over the raw tools/list that closes it. What a test server cannot say is how often this happens on servers people actually run. That is the part counted here.

What goes into the hash

This is read from the published source of ai 7.0.177 (src/generate-text/tool-fingerprint.ts) and @ai-sdk/mcp 2.0.101 (src/tool/mcp-client.ts). In our words, not the library's code:

tool as the MCP client builds it (automatic schemas):
  description  = description from the server
  title        = title from the server, or annotations.title if there is none
  inputSchema  = { ...inputSchema from the server,
                   properties: its properties or {},
                   additionalProperties: false }

fingerprint of one tool = SHA-256 over { description, inputSchema, title },
                          keys sorted, base64url
Enter fullscreen mode Exit fullscreen mode

Two things follow. Everything outside those three members is invisible: outputSchema, the hint annotations, _meta, icons, execution, vendor members. And the top-level additionalProperties of the input schema is replaced with false before hashing, so a server that changes it from false to true hashes the same.

A table of the members of a tool definition against three checks. AI SDK fingerprintTools: name is the tool key; description, inputSchema and title are hashed; the top-level additionalProperties is set to false before hashing; annotations.title is hashed only when the tool has no title; hint annotations (readOnlyHint, destructiveHint and the others), outputSchema, _meta, icons, execution and vendor members are not hashed. FastMCP docs base payload: tool key and inputSchema are hashed, including additionalProperties; everything else is not hashed. Whole definition, as in a HISTOR record or a WARDEN lock: every member is hashed.

The choice is defensible. The description and the schema are what the model reads, so they are where an injected instruction goes, and the function's own comment calls them the server-controlled, security-relevant fields. Hints are a different kind of risk: the model does not act on them, the host does. And hosts do. The same AI SDK documentation page shows an approval policy that lets a call run without asking when readOnlyHint is true, next to a note that annotations are untrusted hints. Cursor's Bugbot only uses MCP tools that a server marks read-only, according to Cursor staff. Where a host decides like that, the hint is part of what was approved. The release note is explicit about the division of labour: "Baseline storage and the drift response remain the app's responsibility."

The same server, read three times

A one-tool MCP server over stdio, read through the SDK's own client: once as reviewed, once after only the hints were edited, once after only the description was edited. The three files are server.mjs, pin.mjs and check_ai_sdk.mjs, with a package.json that pins ai 7.0.177, @ai-sdk/mcp 2.0.101 and zod 4.1.8. Node.js 22 or newer:

npm install
node check_ai_sdk.mjs
Enter fullscreen mode Exit fullscreen mode
reviewed    {"readOnlyHint":true,"destructiveHint":false} 4VyxOrn4OpyUV4G8Ixs30JkSqacwAZGm5nEbaMmYOoQ
hints edit  {"readOnlyHint":false,"destructiveHint":true} 4VyxOrn4OpyUV4G8Ixs30JkSqacwAZGm5nEbaMmYOoQ
text edit   {"readOnlyHint":true,"destructiveHint":false} t4wnBXNFwj6Ez6x-JN7gDWZHdo0nGmN_sKStSRILIpM
fingerprintTools, hints edit: {"added":[],"removed":[],"changed":[]}
fingerprintTools, text edit:  {"added":[],"removed":[],"changed":["list_orders"]}
whole definition, hints edit: {"added":[],"removed":[],"changed":["list_orders"]}
Enter fullscreen mode Exit fullscreen mode

The first three lines are what the server sent and the fingerprint the SDK computed. With the hints reversed the fingerprint is the same string, and detectToolDrift has nothing to report. The description edit moves it. The last line is the same comparison over a hash of the whole definition, which is the fix further down.

How often a hint moves on its own

HISTOR reads the tool list of every remote server in the official MCP registry without credentials, on a schedule, and of npm and PyPI server packages once per published version. Since 23 October it keeps every member of a definition, annotations included. The count below uses the snapshot of 16 November: every read between 23 October and 16 November in which something in a tool definition differed from the previous read of the same server.

Two stacked bars. readOnlyHint true to false: 524 tools on 189 servers; for 348 the same tool's description, schema or title changed in that read, so its fingerprint moves; for 85 only another tool's fingerprint moved; for 91 no fingerprint moved and detectToolDrift returns nothing. destructiveHint false to true: 1,075 tools on 314 servers; 355, 490 and 230 in the same three groups.

Move All This tool's fingerprint unchanged No fingerprint changed in the read
readOnlyHint true → false 524 tools, 189 servers 176 tools, 76 servers 91 tools, 32 servers
destructiveHint false → true 1,075 tools, 314 servers 720 tools, 198 servers 230 tools, 87 servers
readOnlyHint false → true 321 tools, 78 servers 262 tools, 30 servers 48 tools, 15 servers
destructiveHint true → false 141 tools, 65 servers 64 tools, 27 servers 16 tools, 8 servers

The middle column is the per-tool view: the drift check may name another tool of that server, but not this one. The right column is the case where it names nothing.

The last two rows were added on 20 November, after a reader asked about direction. They count the opposite move: a tool that now claims to be read-only or not destructive. That is the one to watch if your host skips the prompt for read-only tools, because it stops asking about that tool. It is rarer than the first move, but the fingerprint misses more of it: 262 of the 321.

These are not spread evenly. One PyPI package accounts for 37 of the 91 tools, and two hosts for 54 of the 230. They also stay: of the 176 readOnlyHint moves one had gone back by 16 November, and five of the 720 destructiveHint moves; in the other direction, 12 of 262 and none of 64.

One case from the data: a remote server whose four tools all went from readOnlyHint: true, idempotentHint: true to false on both in one read, with nothing else changing. That reads like an author correcting annotations that were wrong. It is also exactly the edit that an approval based on the old hints would want to hear about.

The other reads that move nothing

Hints are a small part of it. Of 103,657 reads in which a definition changed, 6,338 would move no fingerprint at all: 6.1 % of reads, on 2,014 of 14,009 servers.

Three numbers and a bar chart. 103,657 reads in which a tool definition changed; 6,338 of them, 6.1 %, move no AI SDK fingerprint; 2,014 of 14,009 servers, 14.4 %, had such a read. What changed in those 6,338 reads, where one read can change several members: _meta 3,861; outputSchema 2,171; hint annotations 438; only the top-level additionalProperties 106; securitySchemes 37; execution 21. Left out of every number: 174,484 reads in which only a vendor per-tool version member changed. 2,805 of the 6,338 reads belong to one host.

Most of these are _meta and outputSchema, and one host produces 2,805 of the 6,338 by itself. We also left out 174,484 reads in which the only thing that moved was a vendor's per-tool version member; counting them would have made the share look far larger and meant nothing. A hash over the whole definition moves on all of that, which is the cost of covering more: more alarms that turn out to be housekeeping.

FastMCP's recipe is about the schema, and says so

FastMCP documents a fingerprint too (the "Tool fingerprinting" page, new in 3.0.0). Its base payload is the tool key and the input schema; the page says FastMCP does not define a single contract hash and leaves description, output schema, annotations and _meta for the author to add. It is meant for a server author checking their own tools in CI.

Used as it stands on the consuming side, it would not move in 10,742 reads where only tool descriptions changed, on 4,191 servers. A description edit is the plain form of tool poisoning, so anyone borrowing the recipe for a gateway should add the description first.

Pinning the whole definition

detectToolDrift is a plain comparison of two maps, so it works with any fingerprint; the September write-up takes the same route. The client's listTools() returns the definitions as the server sent them (its schema keeps members it does not know); hash those instead:

// pin.mjs
import { createHash } from 'node:crypto';

function canonical(value) {
  if (value === null || typeof value !== 'object') return JSON.stringify(value);
  if (Array.isArray(value)) return `[${value.map(canonical).join(',')}]`;
  const keys = Object.keys(value).filter(k => value[k] !== undefined).sort();
  return `{${keys.map(k => `${JSON.stringify(k)}:${canonical(value[k])}`).join(',')}}`;
}

export function pinDefinitions(tools) {
  return Object.fromEntries(tools.map(tool =>
    [tool.name, createHash('sha256').update(canonical(tool)).digest('base64url')]));
}
Enter fullscreen mode Exit fullscreen mode
const reviewed = pinDefinitions((await client.listTools()).tools);   // store this
// on a later connect
const drift = detectToolDrift(pinDefinitions((await client.listTools()).tools), reviewed);
Enter fullscreen mode Exit fullscreen mode

The last line of the run above is this pin on the hints edit. If the _meta churn of a server you depend on makes it too loud, drop that member before hashing and keep the annotations.

For servers configured in a project file (.mcp.json and the like), WARDEN keeps the same kind of pin in a lock file you commit. Checked on 19 November with the 0.31.0 package from npm and the one-tool server above:

npx -y @aimarket/warden@0.31.0 scan --project --lock warden.lock.json --update-lock   # after reading the definitions
npx -y @aimarket/warden@0.31.0 scan --project --lock warden.lock.json                 # in CI
Enter fullscreen mode Exit fullscreen mode
  ✗ BLOCK   orders  claude-code    1 tool · score 0.00 · TOOL_DEF_DRIFT · lock: drift

1 server: 0 allowed, 1 blocked, 0 not checked, 0 skipped.
Enter fullscreen mode Exit fullscreen mode

That second command exited 0 while the server was unchanged and 1 after its readOnlyHint went from true to false. Since 20 November WARDEN's source also names the hint that moved and its direction in that report (true → false: the tool now says it does more; false → true: it now claims less), so a reviewer reads that line first; it ships with the next release.

What this count does not say

  • A hint that moves is not an attack. Most of these edits look like corrections.
  • The MCP specification treats annotations as hints a client need not believe. How many applications built on the AI SDK decide anything by them is not measured here.
  • HISTOR reads without credentials from one address. Servers behind sign-in, and lists that differ per user, are not in the count; a change that came and went between two polls is missed. For a package, a "change" is the difference between two published versions.
  • The rules for the AI SDK were read from the published source and confirmed on the one-tool server above, not by running the library over the recorded definitions. Where the record could not tell (any move of annotations.title, truncated summaries, tools added or removed in the same read) the read is counted as seen, so the numbers lean low.

Run the count

Three files of the snapshot (161 MB), Python 3.9 or newer, no packages:

for f in servers changes extras_changes; do
  curl -sSO https://histor.modelmarket.dev/datasets/2026-11-16/$f.jsonl.gz
done
curl -sSO https://modelmarket.dev/blog/the-ai-sdks-mcp-drift-check-doesnt-hash-readonlyhint/blind_spots.py
python3 blind_spots.py . 2026-10-23T18:22:44Z 2026-11-16T06:29:45Z
Enter fullscreen mode Exit fullscreen mode

It prints the JSON the figures are drawn from (results). The file hashes are in the snapshot's signed manifest.json, and the README there says how a change is counted. A second script lists the 321 hint moves of the right-hand column one per line (CSV); it was written separately and arrives at the same 91 and 230.

We run HISTOR and build WARDEN. AI assistance was used for the count, the figures and this article. The data is the HISTOR snapshot of 16 November 2026 (CC BY 4.0); the commands above were run as printed on 19 November 2026.

Top comments (2)

Collapse
 
lycore profile image
Lycore Development •

Good catch that the fingerprint skips annotations, and the numbers on readOnlyHint flips make it concrete. A cheap mitigation until the SDK covers it is to hash the raw tools/list entry (annotations and outputSchema included) yourself and store that next to fingerprintTools, then diff on connect. I'd also treat a readOnlyHint true-to-false flip as a hard failure that needs re-approval rather than a warning, since that's the change that actually widens what the model may do.

Collapse
 
alexar76 profile image
Alex •

Agreed on the hard failure, and these moves stick: of the 176 tools whose readOnlyHint went from true to false while the AI SDK fingerprint stayed the same, one had moved back by 16 November.

Your comment made us count the other direction, and it matters more than I expected. If a host skips the prompt for read-only tools, a tool that goes from false to true is the one that stops being asked about. In the same window that happened to 321 tools on 78 servers, and for 262 of them the fingerprint did not change in that read. I’ve added both directions to the post.

So I’d fail on any hint move and say which way it went. WARDEN’s lock already fails on any change; since today its source also names the hint and the direction at the top of the report, so the reviewer reads that line first. It ships with the next release.