Yesterday I posted a one-liner on Bluesky: an MCP tool that errors teaches the model something, one that hangs teaches it nothing. A reply pointed ...
For further actions, you may consider blocking this person and/or reporting abuse
The three-state distinction is the key: completed with rows, completed with zero rows, and not completed. I’d make it structural rather than relying only on prose—e.g. status, matched_count, scanned/covered sources, normalized filters, observed_at, truncated, and retryable—then generate the human-readable text from that envelope. “0 rows” is still unsafe if one partition timed out or a replica was stale, so coverage and freshness should be independent from count. I’d also bind retries to one logical operation ID and report attempt count separately; otherwise a client can mistake repeated execution for multiple user actions. Contract tests should force timeout-before-query, timeout-after-partial-read, cancellation, stale replica, and genuine zero matches, then verify the client never collapses them into the same empty array.
Agreed on making it structural — prose drifts, an envelope can't. And the coverage/freshness split catches the case our three-state framing genuinely can't see: a clean zero from a stale replica still reads as a real answer. Binding retries to one logical operation ID is the piece I hadn't considered — is that from a system you've built?
The empty-success case is the one I keep seeing cause real damage, because it looks clean in both places people check first. The tool log says 200, and the model has no sharp error token to route around.
One thing I would add for MCP tools is an explicit result-shape contract for "nothing found". Empty array plus a reason code is much easier to recover from than empty array as success. It gives the agent something concrete to question before it builds the next three steps on sand.
Testability is the part I underweighted: a reason code is something CI can assert on, where a sentence only works if the model reads it the way I intended. The bit I'd expect to go wrong is the code list drifting per tool, until callers stop switching on it and just check for emptiness again. You've moved where I'd put the boundary — I wrote the post as if the result string were the whole contract.
This is the kind of MCP testing people skip because the happy path feels more interesting. Errors, hangs, and empty results each teach the agent a different lesson. I like treating them as contract cases, not just bugs, because the caller behavior is part of the tool API too.
Contract cases is a better frame than the one I had. That third mode only reached the post because someone replied and named it — enumerating bugs never tells you when the list is done. Do you keep those caller-behavior cases in the tool's docs, or in tests?
I would keep them in both places, but with different jobs. The docs should name the behavior contract in human language: timeout, empty result, malformed result, partial side effect. The tests should pin the actual caller response. If it only lives in tests, future tool authors miss the design intent; if it only lives in docs, it rots.
The 'future tool authors miss the design intent' half is the whole argument for me — I've been that future author, squinting at a mock and wondering what behavior it was protecting.
That is exactly why intent needs a home near the behavior it protects. A future maintainer should not have to reconstruct the rule from a screenshot, a commit message, and a broken test at the same time.
That is the split we landed on too: the guard carries its reason as a comment right beside it, and the test title states the situation it protects, so the intent survives in two places a maintainer will actually open. The part I have not solved is drift between the two — the comment gets edited, the test does not. How do you notice when they have come apart?
The empty success case is the one that actually burns review time. When a tool returns [] with no isError, the agent happily invents the next step on top of a void, and the damage only shows up later as a weird hunk in the diff: a duplicate user, a rewritten config, a default that looked intentional. Loud errors are annoying but cheap. Silent empties are expensive because nothing in the transcript flags the turn that lied. Making the server say "0 rows matched this exact query" is the difference between a recoverable miss and a six turn hallucination you only catch by reading the patch.
We ended up applying your rule to our own pipeline's no-ops: every skip returns a named reason ("no-stock", "already-published-today") rather than a bare success, because a bare success today is an unexplainable diff next week. Naming the query does something to the record too — an empty that says what it searched turns a later "why did it do this" into a lookup instead of archaeology. Linea puts the diff and the PR right next to the pane, so I'm curious whether that catches the silent-empty turns in practice, or whether they still only surface once someone reads the patch.
The empty success case is the one that has burned me most with Claude Code MCP servers. An error with a clear retry shape usually gets fixed on the next turn. A hang at least stops the session. A 200 with empty content lets the model invent a story about what the tool returned and then spend six more tool calls defending that story. Writing the isError text for the model, not for a human log, is the fix people skip, and it is cheaper than any timeout tuning.
Agreed on the cost order, though the two are different kinds of work. A timeout is one knob per server; the isError text has to be written per tool, and it goes stale quietly when that tool's failure modes change. I don't have a way to notice that staleness yet, short of hitting it.