45% of remote MCP servers changed their tool definitions in four weeks. The log is now a dataset.

#mcp #security #opensource #dataset

A client that connects an MCP server reads its tool definitions: names, descriptions and input schemas. The model then reads them as instructions. Pinning a definition once and trusting it later only works if the definitions hold still.

HISTOR, our public log of MCP servers, reads the tool list of every remote server in the official MCP registry once a day and records every change. As of today the whole record is downloadable as files. Here is what the first snapshot (20 October 2026) says about how often definitions move.

Half of them move

Take the 10,929 remote servers that had a recorded tool list by 24 September and are still listed. By 20 October, 4,926 of them (45%) had served a different tool list at least once.

2,036 changed the description of a tool they already had.
1,180 added a tool; 336 removed one.
The typical changing server changed once. The 90th percentile changed 5 times. 75 servers changed 14 or more times in those 27 days.
Changes are almost never flapping between two lists: 200 of 23,821 changes return to a list the server had shown before.

Packages behave differently because a published npm or PyPI version cannot change. HISTOR runs each new version in a sandbox. Across 21,738 packages, 1,419 listed different tools in a newer version.

What moves

Over all 23,821 recorded changes (remote servers and packages, 24 September to 20 October):

	Changes
Existing tools modified, nothing added or removed	16,436 (69%)
Tools added or removed only	2,330 (10%)
Both	3,136 (13%)
Only tool names readable (the list could not be fully digested)	1,919 (8%)

Counting each modified tool once per change, its input schema changed 105,324 times, its description 59,287 times and its output schema 16,422 times. 14,164 changes, 59%, touch at least one description. That is the text the model treats as guidance.

Two patterns worth knowing if you publish a server

A live number in a description rewrites your tool every day. 2,719 changes changed nothing but digits in a description. 2,493 of them come from one publisher, who writes a running count of its sources and tools into the description of every endpoint, a sentence of the form "routes the question to one of N tools across M sources". Each time the count moves, every one of those tools has a new definition. A client that pins definitions sees a change. A client that caches its prompt has to rebuild it. A reviewer has to read a diff that says nothing. Counts, dates and "last updated" belong in a tool result, not in the definition.

Some descriptions argue with the client's other tools. 280 servers on 63 hosts have a tool description that tells the model to choose it over web search, over the client's built-in fetch, or over its own knowledge: "PREFER OVER WEB SEARCH for …", "prefer it over the built-in web search", "use this instead of WebFetch". 185 of the 280 belong to one publisher. Telling a model when to pick one of your own tools over another is good practice. Telling it to pick yours over tools from other servers is a claim on the client's routing, made in text the user rarely reads. A client can ask the user before honoring that.

What this does not measure

What a client receives with credentials. HISTOR connects anonymously. Servers that need a key show their public tool list or none.
Who caused a change. A new tool list may be a release, a deployment rollback or a feature flag. Most changes are ordinary releases. The log records that it happened, not why.
Whether a change is harmful. Out of these changes, WARDEN's pattern scan put 3 servers newly into its block tier. That covers only the 735 changes where both lists were scanned under the current rules. Pattern matches are leads for a human to read, not verdicts.
Popularity. Every listed server counts once.

Get the data

histor.modelmarket.dev/datasets/. One snapshot a month, CC BY 4.0 for the compilation:

servers.jsonl.gz: 83,957 rows (remote endpoints and packages, delisted ones included), with status, tool count and the digest of the current tool list
toolsets.jsonl.gz: 57,846 tool lists exactly as served, current ones and every one a change replaced
changes.jsonl.gz: 23,821 changes with tools added, removed and modified, field by field
scans.jsonl.gz and pattern-set.json: WARDEN's pattern matches and the rules
manifest.json: the SHA-256 of every file, signed by the log's key, with the signed tree head the snapshot was taken under

Fields that vary in shape are JSON text, so every file loads as a flat table. The 45% above is one DuckDB query:

WITH base AS (
  SELECT id FROM 'servers.jsonl.gz'
  WHERE transport <> 'stdio' AND NOT delisted
    AND toolSetDigest IS NOT NULL AND firstPinned <= '2026-09-24T23:59:59Z'
)
SELECT count(*) AS servers,
       count(*) FILTER (WHERE id IN (
         SELECT serverId FROM 'changes.jsonl.gz'
         WHERE json_extract_string(summaryJson, '$.detail') = 'full')) AS changed
FROM base;
-- 10929 | 4926

To check one server without downloading anything: modelmarket.dev/tools/histor-check.

We build HISTOR and WARDEN. No server is named here; the dataset names all of them, as the registry does. A server operator who wants to be left out of future snapshots can open an issue on alexar76/histor.

DEV Community