How Reliable Is Your AI Agent?
For about three weeks I thought my server host was robbing me.
The agent I run, Ghost, is my own instance of Hermes, an open-source framework from Nous Research. It works on a rented box all day with me nowhere near it: research, drafts, scheduled jobs, the unattended autonomy everyone is being sold. In May it started slowing down. A job that took a minute took five. The server’s own dashboard blamed “steal time,” the polite name for a noisy neighbour on shared hardware eating the processor. So I did the normal thing. I complained to support, read forum threads about oversold hosts, priced a migration.
The neighbour was me. The failure had spent three weeks disguised as someone else’s fault, which is exactly what the dangerous ones do.
Ghost runs each of its tools inside a throwaway container and, by default, never deletes the dead ones. They piled up. When I finally ran the one command that would have told me on day one, the list of dead containers filled the screen and kept scrolling: 748 of them, all but five long dead, and the machine underneath had seized trying to keep track. The host throttled everything. Nothing was broken in the way broken usually looks: no crash, no error, no alert, only a number climbing by one, over and over, for weeks, with nobody reading it.
That counter is what running an agent on your own actually looks like. The demo writes you a poem. The real thing is a number climbing in the dark while the disk fills.
Grade it twice
The counter was one way Ghost had fooled me. There was another, and this one you can run on your own agent this week.
I ran a proper audit of its research against primary sources. Twenty-two factual claims about companies and markets, checked one at a time. Twenty came back directionally right, the right company and the right direction and the right thesis. Ninety-one percent. You could sell that number.
Then I graded the same twenty-two strictly. Was every specific right too, the exact figure, the exact date, the exact quarter. Seventeen. Seventy-seven percent.
That gap is the entire problem. The claims Ghost got strictly wrong were not inventions. They were the boring kind of miss: a price that was right last week, a date that had drifted, a version that had moved on. It had the shape of the world right and its current state wrong, and current state is the part you act on.
So an agent that is confidently, directionally right is more dangerous than one that is obviously wrong. The obviously wrong one you check. The directionally right one earns your trust and then spends it on a stale figure you paste into a memo for someone who acts on it. If you have ever pulled a number from an AI and dropped it into a deck, you have shipped one of these without knowing.
A 91% that hides a 77% is the exact accuracy at which people stop checking.
The audit takes an hour and tells you more about your own agent than any benchmark can. Take twenty of your agent’s claims and grade them twice, once for the shape and once for every specific, current as of today. The spread between the two scores is your blast radius, and it is always wider than the single number you have been quoting.
It was never the model
Once I had that shape in my eye I saw it everywhere, and never in the model. The scheduler reported success whenever Ghost replied at all, so a job could fail outright, write back that it could not fetch the data, and still get logged green because something had come back. A config change I made was silently overruled by a second file that loaded later and won, pointing the agent’s storage at the wrong place; the system did exactly what the files told it, and nothing reconciled the two. The search index ballooned overnight to a size with no relation to the data inside it, until the disk hit 100 percent, the agent started returning “no space left,” and the work stopped, with nothing watching it grow.
None of this was the model’s fault. Resource leaks, configs that override each other, success signals that lie: these are the oldest problems in running software, and intelligence buys no exemption from them. The model underneath was cheap and fast; a frontier one would have hit all of them the same. A smarter model slips less often, and it still cannot see the slip it makes, because the evidence sits outside it. Every failure started in the same place: the agent did something, and nothing outside it looked at the result. A better model changes the odds. It does not remove the need for something outside the agent to look.
That was the turn for me. I had spent weeks grading Ghost on how clever it was. What decides whether an agent is safe to leave alone is duller and far harder to fake: how much of what it does gets checked by something that is not the agent.
An agent cannot be the thing that confirms its own work.
It reasons from inside its own process, where a sandbox that cannot start a container and a whole host that is down look identical. It will hand you a confident account of which one it is, and the account is worthless, because the fact that would settle it sits on the other side of a wall it cannot see over. An agent that grades its own work can always move the grade, which is why the only gate a self-rewriting agent cannot game is time: the checks that hold are the ones it has no hands on.
Different operators, same answer
For a while I assumed this was specific to my setup. Then I read the operators who run Hermes hardest, and kept finding my own containers in their notes. None of us had compared notes; we were solving different problems, with different tools, in different words. One, tired of trusting the agent's own edits, makes every self-change a diff a human signs off before it goes live. 1 The setup guide everyone passes around spends its length on the dull perimeter: what each surface may touch, what runs sandboxed, what a human approves. 2 Others come at it from other directions, building evals to close the loop on output 3 and pruning the skills the agent writes for itself, 4 and the instinct is identical every time.
Everyone who runs one of these for real comes back having built the same thing. Not a smarter model. A verifier.
This finds you whether or not you run a server. It bites anyone who lets an AI do something they then act on: the draft you send without rereading, the figure you quote because it sounded sure, the report you stopped opening because it is always fine. An agent does not have to be autonomous to fool you. It only has to produce something you have stopped checking.
You do not need your own month of this to learn what it teaches.
Where the check has to live
Fixing all of it was the same move every time. Put something outside the agent that can see what it cannot, and have it watch what is actually running, not the agent’s account of it. The form that takes depends on the failure.
The silent pile-ups, the containers and the disk, each had a leading number, one that moves long before the crash: the count of dead containers, the size of the index, the free space left. Those you watch directly. Set the alarm well below the cliff and read it far more often than it can break, every half hour rather than every twelve hours, so the warning fires while the problem is still a number and not yet a wall.
Top comments (0)