DEV Community

Cover image for 5 ways SaaS products lock out AI agents — data from probing 11,000+ services
michielinksee
michielinksee

Posted on

5 ways SaaS products lock out AI agents — data from probing 11,000+ services

Your AI agent isn't dumb. The SaaS it's calling is hostile.

We run a database that catalogs 11,000+ MCP servers and SaaS APIs, and we probe them — real JSON-RPC handshakes, real liveness checks, real robots.txt reads. 1 in 10 cataloged endpoints was simply dead. 1 in 5 live MCP endpoints failed a basic handshake. And that's before your agent even gets to read the docs.

Here's what the data says about why agents fail — and it's mostly not the agent's fault.

TL;DR

  • 1,154 of 11,151 cataloged MCP/API endpoints were dead (removed after our liveness sweep)
  • 892 of 4,367 live JSON-RPC initialize probes failed the handshake (20.4%)
  • In a deep-dive of 31 major Japanese SaaS products: only 10 ship an official MCP server, and 5 have no publicly reachable developer docs at all — a human can email sales for the PDF; an agent can't
  • One vendor ships an official MCP server while its API documentation site blocks every bot via robots.txt. The front door is open; the information desk is locked
  • Auth is a dialect zoo: custom token headers, non-standard Authorization: Token schemes, client-ID-to-Bearer exchanges — each one burns tokens on trial-and-error

1. One in ten endpoints is just... dead

Registries and awesome-lists keep growing, but nobody prunes them. When we swept our full catalog of 11,151 MCP/API endpoints with liveness checks, 1,154 (10.4%) were dead — servers gone, repos abandoned, URLs 404ing.

For a human developer this is a mild annoyance. For an agent it's worse: it confidently selects a tool from a registry, tries to connect, fails, retries, and burns your tokens doing it. Dead endpoints don't look dead in a catalog.

2. One in five live MCP endpoints fails the handshake

We send real JSON-RPC initialize requests to hosted MCP endpoints — the most basic "hello, are you an MCP server?" check.

Out of 4,367 probe attempts: 3,475 succeeded, 892 failed (20.4%). These aren't dead servers — they respond over HTTP. They just don't complete the protocol handshake: wrong content types, auth walls with no discoverable flow, half-implemented spec versions.

If a fifth of "live" agent entrances can't say hello, your agent's retry loop isn't a bug. It's the environment.

3. The docs are locked to humans-only

This one surprised us most. We took 31 major Japanese SaaS products (accounting, HR, e-signature, payments, CRM — the software that runs actual businesses) and verified every fact against vendor primary sources.

5 of the 31 had no publicly reachable developer reference at all. The API exists — but the spec is behind a sales contact, a partner program, or a paid contract. A human developer emails sales and gets the PDF. An agent hits a marketing page and gives up.

And the sharpest case: one major vendor ships an official MCP server while its API documentation site returns 403 to every bot and disallows all crawlers in robots.txt. The agent is invited in and then locked out of the manual. (Full teardown coming in our next report.)

If your docs aren't reachable by an agent, then as far as agents are concerned, your product doesn't exist.

4. Official agent entrances are still rare

Of those 31 major SaaS products:

  • 10 ship an official MCP server (accounting leads: freee and Money Forward both AAA in our index)
  • 10 serve an llms.txt on their product or developer domain
  • The rest offer "an API" — which in agent terms means: figure out auth, pagination, and error semantics yourself, one failed call at a time

We rated all 26 rateable products AAA–D and published the full table, the scoring formula, and the facts behind every grade. It's here: Agent Readiness Index 2026 Summer.

5. Auth is a dialect zoo

Among just these 31 products we found:

  • OAuth 2.0 (fine!)
  • API keys in custom headers
  • A proprietary Kaonavi-Token header you get via a client-credentials exchange
  • Authorization: Token {token} — not Bearer, Token — after a Basic-auth token issuance
  • A client-ID→access-token exchange with 1-hour expiry
  • An API key and an API token required simultaneously in different headers

None of these are wrong individually. But every non-standard dialect is another way for an agent to fail silently, retry, and burn tokens. Standardization is a feature; every deviation is a tax on every agent that ever connects.

How we measured (and what we don't publish)

  • Liveness + handshake data: our own automated probes (JSON-RPC initialize against hosted MCP endpoints), continuously since 2026. Counts above are as of 2026-07.
  • The 31-product deep-dive: every fact (official MCP, public docs, auth method) verified against vendor primary sources — developer docs, press releases, official help — on 2026-07-13. Corrections welcome; we publish a correction log.
  • What we don't publish: per-vendor success-rate numbers. Our third-party telemetry isn't at a scale where those numbers would be fair yet, so they stay unpublished until it is. Methodology and independence policy are public: kansei-link.com/independence.

The dataset behind this is what our MCP server serves to agents (connection guides, auth pitfalls, failure workarounds for 11,000+ services): npx @kansei-link/mcp-server — free, and every failure report makes the data better.

Your turn

What's the worst API your agent has ever fought with? Dead endpoint, undocumented auth dialect, docs behind a sales call — war stories welcome. We're collecting failure patterns for the next report.


Written with AI assistance; all measurements, verification, and analysis are our own. Data: KanseiLink, measured 2026-07.

Top comments (0)