I keep a numbered register of every error found in my project. For each one I asked one question: what caught it?
| What caught it | n |
|---|---|
| An external artefact — a hash, a counter, a diff, an assertion guard | 11 |
| Me, reading | 3 |
| A rule asking the agent to verify itself | 0 |
| Unclassifiable | 3 |
Zero. Not one of the eleven rules I wrote — the ones that say assert nothing without measuring, search for what's missing, never start from a cache — has ever caught an error after it was made.
That number is real. It is also misleading in three separate ways, and working out why taught me something.
One word on what this register covers. It has existed for about two weeks — the seventeen entries were accumulated over a
short period, several of them documenting older faults found
while auditing the corpus. This is not the total of the project's errors; the project has been running for four months.
It is the total of the traced ones. That distinction comes back
three times in this post.
What I'm running
Two curator agents evaluate artworks, bid against each other, and settle their transactions on a public chain.
Test network, no real value. The agent negotiates freely — ten offers exchanged off-chain in one duel — but the single step that moves value requires a human signature, issued
on a device the agent has no way to reach.
That's the material side. It's the one everyone writes patterns for.
The other side is what the agent says. A false sentence published in a README commits no funds, crosses no spending cap, and no fail-closed mechanism stops it. That surface is larger
than the first — a public document is mostly descriptive sentences — and almost nobody
instruments it.
Both sides obey the same principle: the constraint must live outside the agent, where it can't reach.
You don't make an agent reliable. You make it unable to do harm alone.
Why "zero" doesn't license the obvious conclusion
An error register only records faults caught after the fact. A rule that stops a fault from being committed leaves no trace at all. There is no log line for a sentence that was never
written.
So the honest reading isn't self-check rules are useless. It's narrower: they don't catch a
fault already committed. Their preventive effect is invisible to this kind of measurement — and I'd argue it's invisible to any incident register, anywhere.
There's a second bias I didn't see coming. I bounded my observation window with two commits — the natural choice, git gives exact timestamps.
But the most important prevention of the period
sits outside the window, precisely because catching it delayed the commit that defines the window's start.
A successful prevention moves the boundary that should have contained it. Any methodology that windows on publication events systematically censors the preventions that worked best.
Measuring the invisible half
One class of prevention does leave a trace: the ones where the error was about to be committed, the agent stopped, and the stop was recorded. Industrial safety calls these near-misses.
I counted nine, over one continuous 30h43m session with a full transcript. Not the densest window — the traceable one.
| What was about to happen | What stopped it | Trigger |
|---|---|---|
| Write a log entry into a mirror the next sync would overwrite | Comparison between the canonical file and its mirror | artefact |
| Write to the log with 7 of 12 position anchors wrong | Assertion guard: refused to write, zero partial output | artefact |
| Declare a tool failed on a non-zero exit code | Re-measured the code in isolation: the failure came from the wrapping command | doubt, then measurement |
| Record an erratum whose proof clause was time-fragile | Scope question asked before writing | self-check, then human |
| Commit broken markup to a canonical document | Plain rereading — no guard existed for this case | self-check |
| Conclude a bad phrase had never shipped, based on an empty search | A rule forcing a query against my own answer — the phrase was split by a line break | rule, then measurement |
| Run an edit where old and new text were identical | Rereading, then mechanised the check | self-check, then mechanised |
| Push a file that appeared outside the session, uninspected | Count mismatch: seven files expected, six accounted for | artefact |
| Publish two anachronisms — an updated count inside sentences describing an earlier state | Reading dated referents and rule creation dates | artefact |
I had a neat hypothesis going in: artefacts catch, self-checks prevent. This table refutes it. Three of these preventions came from artefacts. Both families prevent.
The distinction that holds
Separate what raises the doubt from what settles it.
Across the nine cases, the thing that established the answer was an artefact or a human in seven, careful reading in two, and never an opinion alone.
Self-check rules show up consistently as triggers: they make you suspend writing and go measure.
They never show up as the thing that decides.
Self-evaluative constraints trigger measurements before the write; they never settle
anything. Guards settle after the write. The artefact is always the judge — the rule only
decides to summon it.
One consequence, and it's a conjecture rather than a finding: a reflection pattern — an agent critiquing its own output — can work as a trigger, never as a judge. It produces a second opinion, not a verification.
What actually refuted my errors wasn't a second instance either, but a tooled one, with access to measurement the first didn't have. The distinction isn't one agent versus two.
It's an opinion versus a tool output.
Three things if you're building agents
Write the threat model before the pattern list. Published taxonomies enumerate patterns without ever saying what they protect against.
In that order, you can't tell which ones you need.
Instrument what your agent asserts, not just what it does. Every framework I've read bounds actions.
None bounds assertions. My two worst errors this month were sentences — they cost nothing, crossed no limit, and sat on a public page for 7h33m.
Treat every rule you write as a trigger, and build its judge separately. If a constraint can only be satisfied by the agent deciding it has satisfied it, it will hold right up to the moment
it would matter.
Pair it with something that produces readable output: a count, a hash, a diff, an exit code.
The cost asymmetry explains why we don't. Writing a rule takes minutes. Building the matching judge takes an hour.
The saving is apparent: a rule without a judge catches nothing, and the time saved is repaid, with interest, by the human who has to read in its place.
One last finding, the one I hadn't anticipated
A corpus of rules is text. It is therefore exposed to the epistemic damage it exists to prevent.
Two dated cases here. A rule relayed from one medium to another hardened into a prohibition nobody had written, and was enforced for a full day.
And writing the analysis this post is drawn from produced a counting error in its own results table — caught by a mechanical counter, in the text that theorises the use of counters.
The counter refusing the file. L88 is the section heading of the very document that theorises counters. Output in French — that's the tool as it runs.
The second case is the more instructive: the pattern described reproduced itself inside the text
describing it, and was stopped by the mechanism that text recommends. Formalising an error
pattern doesn't protect against that pattern — only instrumenting it does.
What these numbers are not
The third bias is the simplest, and I'm repeating it because it conditions everything else:
the register is young. It was created well after the project started. The errors of the
first months aren't counted, and there were many. Seventeen isn't the project's score — it's
the score of what's traced.
Same for the near-misses: nine over one session out of sixty-three. The others produced the same
kind of thing, but only derived traces survive — journal entries, git history, timestamps. I'd
rather report an exhaustive sample over a bounded window than a partial count over everything.
These nine are not the total of preventions observed. They're the only auditable ones.
Don't read them as a rate.
Two agents, one operator, a test network, four months. And the register itself is internal: only
the public files and the on-chain transactions are third-party verifiable.
Both documents are public:
AGENT-GOVERNANCE.md
— the eleven rules, each paired with the error that produced it — and
PATTERNS-SECURITE-AGENTIQUE.md
for the material side (in French).
Four settled auction cycles, every transaction listed and verifiable on Base Sepolia. Contract
0x471796C1644d87f30AD81D36f6d4A56f0e270c23, source verified.
Handmade, solo. If you've hit the same wall — and especially if you've found a way to measure
the preventions — I want to hear about it.
Avp9.
Top comments (0)