When an MCP server drops a parameter or removes a tool, nothing errors at install time. There's no changelog in the protocol. The agent just starts failing later, in a way that's hard to trace.
So I started checking. Every 15 minutes, a small runner connects to about 2,000 remote servers from the official MCP registry the way a client does: handshake, then tools/list. It saves the tool list and diffs it against the last one.
Here's what one week looked like (Oct 19 to 26):
- 849 servers answered without a key. Another 896 endpoints were up but asked for a login, which is a working server, not a dead one.
- 307 of the 849 changed their tool list at least once.
- 59 made a change that can break a caller: 37 removed a tool, others removed a parameter or added a required one.
- 251 reworded a tool description. That's not breaking, but descriptions are what the model reads when it picks a tool, so a rewrite can change behavior.
- Median answer time was 221 ms; the slowest 10% took over 700 ms.
Most of these are small servers moving fast, which is fine. The problem is that nothing in the protocol tells the client a tool changed after you approved it.
That's what Toolweather is: the data from those probes, exposed as an MCP server your agent can ask before it calls something. check_mcp_server_health says whether a server is answering and whether its tools changed recently, find_alternative_mcp_servers suggests others that do the same job, and watch_server tracks a few you depend on. It's free to try with no signup (50 calls a day, or 1,000 with a free key).
Add it with one URL: https://toolweather.toolweather.workers.dev/mcp
Status page and leaderboard: https://toolweather.pages.dev
Things I haven't solved: a schema diff can't see a tool that keeps the same schema but starts returning different data, and probing from one location means a server that blocks datacenter IPs can look down when it isn't. If you run an MCP server and think yours is classified wrong, tell me and I'll check the raw response.
Top comments (0)