MCP defines two ways for a tool call to fail. There is a third, no protocol can catch it, and it cost me twelve articles.
On 25 July 2026 I audited every entry in my publishing calendar that carried a published URL — twenty-five of them. Twelve live posts on Medium were truncated: between 41% and 72% of the original text had survived, and some had lost their code blocks entirely. A thirteenth URL answered with HTTP 410. Every one had been reported as a success, with a valid URL, weeks earlier.
Here is the mechanism. My publishing server puts drafts on Medium by driving a signed-in browser, because Medium's write API has been left to rot. The adapter waited for the address bar to change to the new draft's edit URL, treated that as done, and closed the tab. Medium, meanwhile, saves the editor's contents incrementally, over background requests that had not finished. Everything that had not reached the server by the time the tab closed was lost, silently, and the adapter returned ok. There was no failed request to catch: the page that would have made it no longer existed.
Now the part I got wrong. For weeks I read this as "browser automation is unreliable and APIs are reliable". That is not what the data says. dev.to came through completely intact — and so did two other platforms that I drive through exactly the same browser, over exactly the same debugging protocol, in the same run. The dividing line was never browser against API. It was one submission against many: those platforms hand over the whole text in a single POST with nothing left to arrive afterwards, while Medium accepted it in instalments and my adapter stopped watching after the first one. The same fact explains the survivors: short announcements of around 1,500 characters finished saving before the tab closed. It was a race, and short texts won it.
And the reason nobody noticed for weeks is the most ordinary reason there is. The check that existed asked whether the URL was alive. It was. A truncated article is a perfectly healthy web page: 200, renders, title, hero image, three paragraphs, and then it simply stops.
The second bug, which is better than the first
The adapter also had to notice when Medium's editor failed to save, so it searched the page text for the phrases Medium shows on failure — one of which is "something is wrong". It searched document.body.innerText, which on that page includes the article being published.
One of the articles contained the sentence "if something is wrong you flip back".
A successful run was declared a failure, the retry ran, and a duplicate draft appeared. The rule generalises well past browsers: never read status out of prose. If the only way to learn an outcome is to hunt for words in running text, you have guessed it rather than learned it, and prose will eventually contain your keyword by accident.
Why the protocol cannot help
MCP's two levels are a good pair. A protocol error means nothing ran — no such tool, arguments that failed the schema. An execution error means the tool ran and the news is bad, and it comes back as a successful response carrying a flag, in text, because the recipient is a model in the middle of a task and can act on it.
Neither level sees this. From the protocol's point of view the call went perfectly: request sent, well-formed response, no error flag, a URL inside. The flag that reports failure is set by the same code that was wrong about what happened. A protocol can carry a verdict; it cannot audit one.
There is a bill attached, too. On that request the model paid for twenty tool descriptions, paid again for the call, received ok, and moved on. The spend was real. The work was not done.
What replaced the check
The check that the work happened has to be separate from the tool that did it, and it has to ask a different question: not "did the call return" but "is what is now on the server the same as what I sent".
Mine reads every published article back and diffs it against the source, rendering each URL in the signed-in browser that owns it, because Medium blocks anonymous fetches and drafts are private to their account. It compares two signals: the number of code blocks, and whether the text of the last link is present — in these articles the last link sits in the closing "further reading" block, which makes it the first casualty of truncation. It has a second mode that opens everything in a profile signed in to nothing, because an author always sees his own article in full, so a paywall or a login wall is invisible from the account that published it.
The full audit table, the script, and the fix that now refuses to return a URL if less than 98% of the text landed on the server are on my own site: https://dorokhovich.com/blog/mcp-false-success — and the third article of the series, on what an MCP server is made of, is at https://dorokhovich.com/blog/how-mcp-works
Source: my own publishing server and its runbook, audited 25 July 2026. One server, one platform, one bug, a sample of twenty-five. It is not evidence about AI agents in general and I have no money figure to give you, because I never calculated one.
One real question, and I want the disagreement rather than the agreement: does your post-deploy check ask whether the thing is reachable, or whether it is correct? If you think the distinction does not apply to what you run, tell me what your check compares against, because mine compared against nothing and looked green for weeks.
Top comments (0)