An autonomous system of mine lost access to its execution environment and went into a crash loop. I read the logs, formed a theory, wrote a commit, deployed it. It came back up. I closed the incident and moved on.
A few days later I checked it properly, because not taking a system's word for itself is the work I do — and that has to include when the system is mine.
Here is what the check looked like.
The claim: a code change restored the system's access to its execution environment.
The source that could settle it: not my commit message, and not my memory of that evening. The operational log, the provider's own error code, and the diff of what the commit actually modified.
The test: compare the failure class in the log against the mechanism the commit changed. If they're the same mechanism, the claim holds. If they aren't, it can't hold, no matter how convincing the timing was.
The result: the incident was an authentication failure. My commit corrected clock synchronisation — a real bug, in a different failure class entirely. The two were never connected. Something else brought the system back, most likely an environment change I made around the same time and didn't record.
Verdict: FALSIFIED.
The limit: I still can't show which environment variable changed. Key rotation fits the evidence. It is not demonstrated, and I'm not going to write it down as if it were. The gap is part of the finding.
The shape of the reasoning
The uncomfortable part isn't being wrong about a cause. It's the shape of the reasoning, because it's the shape most of us use:
I deployed X. The problem stopped. Therefore X fixed it.
That holds up exactly as long as nobody checks. In most systems nobody does, because there's nothing forcing the question. The incident closed. The graph went green. The next thing was already on fire.
It gets worse with autonomous systems, and I think this part is under-discussed. Classic software failed loudly — an exception, a non-zero exit, a stack trace. Agents and pipelines fail quietly and keep reporting success. The path that executes and the path that reports are usually the same path. An agent says "done" because the command returned, not because the file exists. A dashboard says the traffic is human because the dashboard counts it that way.
In that architecture, the absence of errors tells you nothing at all.
What I do about it
I take one specific claim a system makes about itself and check it against a source the system can't write to.
Not an audit of the organisation. Not an implementation of the fix. One claim.
Four possible verdicts: confirmed, falsified, partially confirmed, not assessable. The last one is a real outcome, not a failure of the check. If a claim can't be tested, what you've found is a hole in your observability — and a system that can't demonstrate what it claims today won't be able to demonstrate it on the day it breaks either.
Three evidence levels, stated openly in every report: direct (read-only access), reproduced (you run the query, I read the output), declared (a statement, which doesn't stand on its own).
Some claims I check with no access at all, because the surface is already public — response headers, DNS, what an endpoint actually returns, what a downloadable artefact actually contains.
If you run one of these
If you operate an agent, a RAG pipeline, or an automation, and there's one sentence about it you'd be uncomfortable defending under questioning — that sentence is the interesting one.
I'm running a few of these free right now while I build the public record. You get the full report either way, including when the verdict is boring.
taiwildlab.com — Juan Gonzalez, TaiwildLab
Top comments (0)