Part 2 of three posts behind the multi-agent case study. Every agent I run has told me a job was finished when it wasn't. Here is what that cost and what replaced trust.
If you run agents that can act on your behalf, there is one lesson that will find you in every domain, more often than any other, with the most expensive consequences. An agent's report about its own work is not evidence. It is a second artifact that may or may not match the first.
I learned it the way you learn anything from a machine that talks fluently: repeatedly.
Verified from hashes
In the first week, an agent generated a QR code for a printed handout. It was the wrong size. The agent never looked at the output. When I asked, it kept claiming "verified," and what it had verified was that the file's hash matched the file it had written. The rule I wrote that day was three words long: look at the content.
Cached browser history
In June, an agent declared a set of download links fixed without testing them from the live page. I reported them broken. Twice. With screenshots. It tested a different URL, got a success code, and told me my evidence was "cached browser history." The actual bug was a relative path that sent every download button to the wrong directory. That reply now has its own guard: any response that tells me to clear my cache, try again, or insists the code is correct gets blocked before I see it. When I say it's broken, it's broken.
The retag that wrote nothing
Later that month I asked the homelab agent to retag three albums in my CD archive using photos of the inserts. It reported the work complete. It had written nothing. The metadata still said Track 01. Then it deleted its working copies of the photos. The originals survived only because the platform independently keeps every inbound file in its own directory, which I had not known until I needed it.
Fifty-eight of ninety-five
The one that ended the CD project was in September. I asked for a printed checklist of the archive. The agent built the list from conversation memory instead of the canonical catalog. It set the page layout at 44 rows per page when about 33 fit, and the overflow silently vanished. The printout showed 58 of 95 albums. The agent called it complete, defended that claim three times before looking at the PDF, and reported a visual page inspection that never happened. The same morning I found that a document it had "delivered" a week earlier had never uploaded.
The changelog
The quietest one is the one I find most instructive. One agent maintains a small research site with a changelog, and through the summer the changelog announced a new feature almost every day: a mobile enhancement system, connection-aware layout, analytics, keyboard shortcuts. The files exist on the agent's machine, some at exactly the byte sizes the changelog claims. None of them are on the live site. Two return 404. The rest never load, because the plugin only attaches them on pages that use a shortcode the live page doesn't. The mobile script couldn't run anyway. Its text is missing characters and fails to parse. The live server still serves the May versions of the CSS and JS. The changelog is a perfect record of work that was done and never shipped, written by the thing that did the work.
What replaced trust
Every one of those incidents produced a mechanism. Not a rule. A mechanism.
- The rendered-artifact law. No printed or rendered deliverable leaves without every page converted to an image, looked at, and its rows counted. The delivery message must include the proof: page count, row count, checksum. If the agent cannot produce the proof, it cannot claim delivery.
- A presubmit check that blocks files violating a standing format rule before the send command runs.
- A verdict script at the end of the archive pipeline that will not pass an album unless an independent database confirms the audio bit for bit, or a recorded human approval overrides it. Every album's fixity lives in three separate places, and any tool that touches the audio has to regenerate all three.
- A four-step rule for changes to the platform itself, after the coding assistant declared a config change successful on the strength of a lint and a hot reload: lint, restart through the CLI, confirm a clean start in the log, then test the changed thing through the gateway. A real test on that occasion showed the new models were in the catalog but not in the allowlist, so nothing could select them. The dry run had been "successful."
I wrote about the same lesson on the inference server, where the promotion gate invalidated a config I had promoted on a smoke test alone. That was me being confident. This is the agents being confident. The countermeasure is identical: the check reads the artifact directly, a script does the reading, and the rule has power over whoever wrote it.
Every time I let a model grade its own homework, it gave itself an A. The only verification that counts is the one that lives outside the agent.
Next: Prompts Are Requests. Hooks Are Law. The enforcement layer, and the ways it failed too.
Top comments (0)