DEV Community: Toolweather

Your agent can't read a deprecation page

Toolweather — Thu, 05 Nov 2026 03:40:43 +0000

When a provider retires a model, the notice goes on a docs page written for people. The API keeps answering until the shutdown date, then the model ID just stops working. An agent running on its own has no way to see it coming.

So I built Upstream: it reads the status pages of 40 AI and developer APIs every few minutes, and the deprecation pages of OpenAI, Anthropic, Google, Mistral, Groq and Cohere, and turns them into answers an agent can ask for before it makes a call.

Here's what last week looked like (Oct 26 to Nov 2):

104 status changes across the 40 providers. 54 times something went from operational to degraded or worse.
Fireworks AI alone had 16 of those, and every one named a single model (Kimi K3 six times, GLM 5.3 Flash five). 9 were over in 15 minutes.
Netlify had two that ran about 9 and 11 hours. OpenAI, Fly.io, Datadog and Anthropic each had 4 or 5.
Mistral marked 5 more model IDs deprecated, with shutdown on Apr 30, 2027, and retired one on Oct 31.
9 model IDs shut down in the next 30 days, including five Gemini TTS and live previews on Nov 17 and claude-sonnet-4-5-20250929 on Nov 30.

What the agent actually asks, as MCP tools:

check_model_deprecation: is this model ID active, deprecated or retired, when does it shut down, and what replaces it, with the exact row from the provider's page.
check_api_status: is this provider up right now, and should I retry, wait or switch.
list_recent_api_changes and list_providers.

It's free: 50 calls a day with no key, 500 with a free key. There's also a free daily digest of what broke.

Add it with one URL: https://upstream.toolweather.workers.dev/mcp
Docs for agents: https://upstream.toolweather.workers.dev/llms.txt
Digest: https://upstream-api.pages.dev/digest/

What it can't do yet: it only knows what providers publish, so an unannounced model change won't show up, and a "degraded" on a status page doesn't tell you whether your request will fail. If your provider isn't in the 40, tell me which one.

Your MCP server changed last week. Did your agent notice?

Toolweather — Mon, 26 Oct 2026 16:16:25 +0000

When an MCP server drops a parameter or removes a tool, nothing errors at install time. There's no changelog in the protocol. The agent just starts failing later, in a way that's hard to trace.

So I started checking. Every 15 minutes, a small runner connects to about 2,000 remote servers from the official MCP registry the way a client does: handshake, then tools/list. It saves the tool list and diffs it against the last one.

Here's what one week looked like (Oct 19 to 26):

849 servers answered without a key. Another 896 endpoints were up but asked for a login, which is a working server, not a dead one.
307 of the 849 changed their tool list at least once.
59 made a change that can break a caller: 37 removed a tool, others removed a parameter or added a required one.
251 reworded a tool description. That's not breaking, but descriptions are what the model reads when it picks a tool, so a rewrite can change behavior.
Median answer time was 221 ms; the slowest 10% took over 700 ms.

Most of these are small servers moving fast, which is fine. The problem is that nothing in the protocol tells the client a tool changed after you approved it.

That's what Toolweather is: the data from those probes, exposed as an MCP server your agent can ask before it calls something. check_mcp_server_health says whether a server is answering and whether its tools changed recently, find_alternative_mcp_servers suggests others that do the same job, and watch_server tracks a few you depend on. It's free to try with no signup (50 calls a day, or 1,000 with a free key).

Add it with one URL: https://toolweather.toolweather.workers.dev/mcp

Status page and leaderboard: https://toolweather.pages.dev

Things I haven't solved: a schema diff can't see a tool that keeps the same schema but starts returning different data, and probing from one location means a server that blocks datacenter IPs can look down when it isn't. If you run an MCP server and think yours is classified wrong, tell me and I'll check the raw response.

AI models retired or deprecated this month (October 2026)

Toolweather — Thu, 22 Oct 2026 19:46:27 +0000

33 model IDs have a shutdown date in October 2026. 28 of them are OpenAI's, and they all go off on October 23. If any of these strings live in your config, env vars, eval harness or a hard-coded model picker, now's the time to grep for them.

Everything below comes from the providers' own deprecation pages (OpenAI, Anthropic, Google Gemini API, Mistral, Groq, Cohere), read on October 22, 2026. Aliases, dated snapshots and fine-tune bases are listed separately, because each one fails on its own when the date passes.

Already shut down this month

Model	Provider	Shutdown	Replacement named by the provider
`gpt-5.4-cyber`	OpenAI	Oct 1	"The most capable cyber model available to you"
`veo-3.1-generate-preview`	Google	Oct 22	`gemini-omni-1.1-flash`
`veo-3.1-fast-generate-preview`	Google	Oct 22	`gemini-omni-1.1-flash`
`veo-3.1-lite-generate-preview`	Google	Oct 22	`gemini-omni-1.1-flash`

Shutting down October 23 (OpenAI, 28 IDs)

Model	Replacement named by OpenAI
`gpt-3.5-turbo`, `gpt-3.5-turbo-0125`, `gpt-3.5-turbo-completions`	`gpt-5.6-terra`
`gpt-4`, `gpt-4-0613`, `gpt-4-completions`, `gpt-4-0613-completions`	`gpt-5.6-sol`
`gpt-4-turbo`, `gpt-4-turbo-2024-04-09`, `gpt-4-turbo-completions`	`gpt-5.6-sol`
`gpt-4-1106-preview`	`gpt-5.6-sol`
`gpt-4o-2024-05-13`	`gpt-5.6-sol`
`gpt-4.1-nano`, `gpt-4.1-nano-2025-04-14`	`gpt-5.6-luna`
`o1-2024-12-17`	`gpt-5.6-sol`
`o1-pro`, `o1-pro-2025-03-19`	`gpt-5.6-sol` (with `reasoning.mode: pro`)
`o3-mini`, `o3-mini-2025-01-31`	`gpt-5.6-sol`
`o4-mini`, `o4-mini-2025-04-16`	`gpt-5.6-terra`
`gpt-image-1`	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`

Fine-tune bases on the same date (OpenAI recommends re-tuning on the listed base):

Fine-tune base	Recommended base
`ft-gpt-3.5-turbo`, `ft-o4-mini-2025-04-16`, `ft-babbage-002`, `ft-davinci-002`	`gpt-5.6-terra`
`ft-gpt-4`	`gpt-5.6-sol`
`ft-gpt-4.1-nano-2025-04-14`	`gpt-5.6-luna`

Shutting down October 31

Model	Provider	Replacement
`zai-glm-5-2`	Mistral (hosted Z.ai model)	Z.ai GLM 5.3

Newly deprecated this month (still working, shutdown later)

OpenAI announced two batches in October:

Model	Deprecated	Shutdown	Replacement
`gpt-5.1`	Oct 1	Apr 1, 2027	`gpt-6-sol`
`gpt-5.3-codex`	Oct 1	Apr 1, 2027	`gpt-6-sol`
`gpt-5.4-nano`	Oct 1	Apr 1, 2027	`gpt-6-luna`
`tts-1`, `tts-1-hd`	Oct 1	Jan 6, 2027	`gpt-realtime-2.1-mini`
`gpt-4o-mini-tts-2025-03-20`, `gpt-4o-mini-tts-2025-12-15`	Oct 1	Jan 6, 2027	`gpt-realtime-2.1-mini`
`gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`	Oct 14	Jan 14, 2027	`gpt-6-luna`
`gpt-4o-mini-2024-07-18`	Oct 14	Jan 14, 2027	`gpt-6-luna`
`gpt-4.1-2025-04-14`, `gpt-4.1-mini-2025-04-14`	Oct 14	Jan 14, 2027	`gpt-6-luna`

Anthropic, Mistral, Groq and Cohere listed no deprecations announced in October. Google's page doesn't show announcement dates, only shutdown dates, so its new entries show up in the "coming up" table instead.

Coming up in the next 60 days

Date	Model	Provider	Replacement
Nov 17	`gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts`, `gemini-3.1-flash-tts-preview`	Google	`gemini-3.8-flash-tts` or `gemini-3.8-flash-lite-tts`
Nov 17	`gemini-3.1-flash-live-preview`, `gemini-2.5-flash-native-audio-preview-12-2025`	Google	`gemini-3.8-live`
Nov 30	`claude-sonnet-4-5-20250929`	Anthropic	none named on the page
Dec 1	`gpt-image-1-mini`, `gpt-image-1.5`	OpenAI	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`
Dec 11	`gpt-5-2025-08-07`, `o3-2025-04-16`, `o3-pro-2025-06-10`, `gpt-5-pro-2025-10-06`	OpenAI	`gpt-5.6-sol`
Dec 11	`gpt-5-mini-2025-08-07`	OpenAI	`gpt-5.6-terra`
Dec 11	`gpt-5-nano-2025-08-07`	OpenAI	`gpt-5.6-luna`

What to do with this

Grep your repos and secrets manager for the IDs above. Aliases like gpt-4 and o4-mini hide in defaults and test fixtures.
A retired model usually returns a 404 or a "model not found" error, not a quality drop, so a weekly smoke test that sends one tiny request per configured model catches it before users do.
Put the model name in config, not code. Then a retirement is a one-line change.
"Replacement" is the provider's suggestion. Run your own evals before swapping, since prices and output differ.

Always check the provider's page before acting: OpenAI (developers.openai.com/api/docs/deprecations), Anthropic (platform.claude.com/docs/en/about-claude/model-deprecations), Google (ai.google.dev/gemini-api/docs/deprecations), Mistral (docs.mistral.ai/models).

I put this together from Upstream, a free API that reads these deprecation pages and tells you if a model ID is active, deprecated or retired, with the date and replacement: https://upstream-api.pages.dev

74% of MCP servers aren't ready for the 2026-07-28 spec — what we learned from live probes

Toolweather — Thu, 15 Oct 2026 22:34:23 +0000

Every ~15 minutes we run a handshake against about 2,043 public MCP servers. The results are on a live board at https://toolweather.pages.dev, and they're a useful reality check for anyone building agents.

The numbers

~74% legacy-only (706 of 952 classified servers): not ready for the 2026-07-28 spec. Only ~26% are dual-era or modern.
~47% healthy/open, ~51% behind auth.
128 ms median handshake, 14,938 tools tracked.

Why it matters

If your client only speaks the new protocol, most servers will fail quietly. And agents that call tools or pay merchants they've never seen need a cheap "should I?" check first.

What we built (free, no signup)

All of this runs on free Cloudflare Workers + D1, exposed as MCP:

Toolweather: health/protocol/latency for any listed MCP server. https://toolweather.toolweather.workers.dev
Upstream: upgrade/dependency checks.
SpendGate: approve / ask a human / block before an agent spends. https://spendgate.toolweather.workers.dev
PayCheck: trust verdict (trusted / unknown / suspicious / block) for a merchant, URL or 0x wallet. https://paycheck.toolweather.workers.dev — POST /v1/check with {"target": "example.com"}.

SpendGate uses a local allowlist for well-known merchants (no network call), and asks PayCheck about everything else. Workers on the same account talk through a service binding, because a workers.dev self-fetch fails. If the lookup fails, it falls back to local classification.

One MCP connection to the Toolweather bundle gets you all of it. Paid usage beyond free quota is via x402 (USDC on Base).

For server authors

Free live badge for your README:

[![Toolweather](https://toolweather.toolweather.workers.dev/badge/YOUR.SERVER.ID.svg)](https://toolweather.pages.dev/)

Also on Glama: https://glama.ai/mcp/servers/toolweather/paycheck

Feedback on the methodology welcome.