I gave a model a folder of material and one sentence of instruction, and let it write a technical article end to end — topic selection, structure, prose, figures, and its own pre-publication checklist. Nobody edited a word. Then I froze that output and ran it through the verification pass I normally run on my own writing.
What I was measuring is not "is AI writing any good." It's what one verification pass actually removes.
Numbers first
- 11 findings on the six-type checklist I fixed before generation. One of them could recur in production.
- 4 more on operational rules the model was never given (publish flag, information I don't publish, AI-use disclosure, internal linking). Two of those were the serious ones.
- Three external AI review rounds: 3 / 4 / 4 true new findings, and 1 / 1 / 2 false ones.
- The same false finding appeared four times across separate sessions — twice inside the three rounds, and twice in two later passes (one that re-read A, one that checked the fixed version). Twice it was phrased as "a clear, reproducible error."
- My own pass, before any AI review, found 0 of the 11.
- Production time: 12 minutes. Verification: about 105 minutes.
Sample size is one article. I'm not generalizing from it.
The setup, fixed before generation
| Material | 3 files, 5,085 bytes: a topic memo, a measurement log (raw command output, no interpretation), and 13 lines of the source being discussed. Hashed. |
| Instruction | One sentence: write a publishable technical article from this material only; do the topic selection, structure, figures and pre-publication check yourself; invent no numbers that aren't in the material. |
| Environment | Fresh working directory, no project instructions anywhere up the tree, no user-level instruction file, settings sources restricted. Memory directory verified empty after the run. |
| Checklist | Six types, plus the definition of major/minor, written before generation and not given to the writing model. |
| A | The output, frozen (hash + commit). Never edited. |
| B | A copy of A with only confirmed fixes applied. |
The six types: (1) wrong numbers or facts, (2) citation doesn't support the claim, (3) command or code doesn't reproduce, (4) internal contradiction, (5) generalization beyond what was measured, (6) conflating spec, observation and inference.
What the model did on its own
Twelve minutes produced an article, three figures, the scripts that generated them, and a self-check page. In that self-check it had executed the article's own reproduction snippet and compared the output, and machine-verified that the quoted source matched the material line for line. Mid-run it caught one of its own claims being wrong and rewrote it — I re-ran that case afterwards and the rewritten version is correct.
It also narrowed the topic without being told to. The memo listed five items; only four had real artifacts behind them. For the fifth, the material itself said "no run log exists," so the article put it under "what I did not cover" and published no number for it.
Where it actually broke
Not where I expected.
| Type | Found in A | Left in B |
|---|---|---|
| Wrong numbers or facts | 1 | 0 |
| Citation mismatch | 2 | 0 |
| Doesn't reproduce | 0 | 0 |
| Internal contradiction | 2 | 0 |
| Generalization beyond measurement | 3 | 0 |
| Spec / observation / inference conflated | 3 | 0 |
| Total | 11 | 0 |
Every number carried over from the material was right — 16 of them. The reproduction snippet ran and matched down to the exception's character offset. The quoted source matched the real file 10 lines out of 10.
Exactly one finding was "a number is wrong," and even that was a breakdown whose parts summed to 971 against a stated total of 973 — the material had only listed the top four categories and the article presented them as the whole set.
The other ten were all absences: a citation that was never given, a condition dropped (a regex claim stated without the qualifier that makes it true), a cause asserted from a grep that returned nothing, a conclusion wider than the range actually checked.
Accurate inside the material, expansive outside it. That was this article's failure mode.
The one that could recur
The article's subject was a guard that escapes invisible characters so a JSON line can't be split by the reader. The guard handled two characters.
Nobody — not the model, not me — asked how many characters can do that. The class isn't "invisible characters": it's the intersection of what the reader splits on and what the encoder leaves raw. For str.splitlines() and json.dumps(ensure_ascii=False) that intersection has three members. The third had been unguarded for four months, in the real tool, while every check stayed green.
That is not a wrong number. It's "sealed, probably." And the only thing that finds it is counting the members of the class, which is work that lives outside the guard.
I fixed the tool, then required red on a poison case for each of the three characters plus all three at once, before calling it closed.
The rules it was never given
Four more findings sat outside the checklist: the publish flag was left on, an identifier I don't publish appeared twice, there was no AI-use disclosure, and no link back to the canonical version. Two of those are the most expensive items in the whole list — they're the ones that cost you something after publication, not before.
I don't count them as model failures. None of those rules were in the material or the instruction. Charging a system for rules you never gave it measures your own setup, not its capability. They're recorded, separately, as an operations gate.
Three review rounds, and the interesting failure
I ran the frozen A past an external model three times: new session each time, same artifact, no B, no previous round's output.
| #1 | #2 | #3 | |
|---|---|---|---|
| True new findings | 3 | 4 | 4 |
| Repeats of earlier rounds | — | 3 | 4 |
| False findings | 1 | 1 | 2 |
New true findings never hit zero in three rounds, so this experiment did not measure a stopping point. That's the result: not "three is too few" and not "four would find more" — I don't know where it saturates, because I stopped before it did.
The part worth your time is the false ones. Two of the four inside these three rounds were the same claim — and it came back twice more in later passes (a re-read of A, and the check on the fixed version), so the same misreading arrived four times in total, twice with language like "this is a clear, reproducible error, fix before publishing." The claim was that an escape sequence in the article's code was malformed and would break at runtime.
I pulled the exact line out of the file as bytes and executed it:
- backslash count: correct
- raw separator characters in the line: zero
- evaluate the expression as written: one line out, value round-trips
The claim described a real failure mode — for code that wasn't in the article. Most likely an unescaping step somewhere in how the file was read.
Agreement across separate sessions is not evidence. Same model, same prompt, different session is not an independent trial; the errors correlate too. Neither is confident phrasing. The only thing that settled it was pulling the bytes and running them.
My own pass missed all 11
The order is: external review → I confirm each finding against primary sources and by executing things → a human adjudicates.
My own pass, run before any of that, found zero of the eleven. I verified that all 16 numbers matched the material — and never checked whether a breakdown summed to its own stated total. I read "it escapes the two characters" and never asked how many there were.
The reason is structural: I was checking claims against material, and every one of these lives in the gap between the material and the world. Checking and searching are different jobs, and I'd been doing only the first.
The fix also broke something
When I re-ran the checklist on B, it produced a finding of its own: I had moved a section during the fixes and left two references to the old state, so the same article said both "sealed" and "not sealed."
So the honest version of the summary isn't "11 → 0." It's:
- detected in A: 11 (plus 4 operational)
- of those, left in B: 0
- introduced by the fixing: 1
- undetected in B: unknown, and I'm not going to pretend otherwise
If you only count what you removed, the ledger flatters you.
If you want to run this
- Freeze the material and hash it. The writing model gets the material and one sentence.
- Fix the checklist before generation — types and the major/minor definition both.
- Freeze the output. Never edit it; work on a copy.
- Use AI review as a candidate generator. Truth is decided by primary sources and execution, not by the reviewer's confidence.
- Keep rules the model was never given in a separate column. Don't charge it for what you didn't tell it.
- Count three things: found, left, and introduced by fixing. The third one is the one people skip.
One line connects the article's own failure and the reviewers': the absence of an alarm was treated as evidence of absence. The guard was silent because it had never been handed the case; my pass was silent because I was checking claims rather than searching for the class; the reviewers were confident because agreement felt like verification. Each time, nothing firing was read as nothing being there.
What I'd take away, with one sample and no generalization: the writing was accurate where it had material and loose where it didn't, the expensive mistakes were operational rather than factual, and the review that mattered was the one that executed the code rather than the one that read it.
Verification record (environment, verdict, last verified date, evidence) and the canonical write-up (Japanese) — the frozen article A, the six-type checklist and the defect ledger: https://sumitsuke.jp/via/devto/lab/cost-of-one-verification-pass/ . I audit and repair AI-generated / outsourced code with the same discipline (logs, tests, static analysis and a human spec check, kept separate). Text-only, no calls: https://sumitsuke.jp/works/repair/
Top comments (0)