DEV Community: Mahiro Hirakawa

A six-line edit made fifteen units stale. The freshness check was working correctly.

Mahiro Hirakawa — Mon, 14 Sep 2026 19:32:21 +0000

Every unit of this project carries a record of its last verification, and each record pins the configuration it was checked against by hashing it. If the configuration changes, the pin no longer reproduces and the record goes stale, which is exactly what you want: a verification result that quietly outlives its inputs is worse than no result.

One afternoon I added a six-line block to that configuration file. It described a new component and touched nothing else.

Fifteen units went stale. Every one of them owed a full re-run of its checks and a fresh record appended afterwards, which is most of an hour of machine time and a boundary ceremony, for six lines that concerned one of them.

Nothing was broken. The check did precisely what it promised.

The pin was over the whole file

pin = sha256(entire configuration file)

Any byte anywhere changes that hash. So the pin does not mean "the configuration this unit depends on is unchanged". It means "nobody has edited this file at all", which is a far stronger statement and one that becomes false constantly in a file every component writes to.

The gap between what a pin means and what everyone reads it to mean is the whole bug, and it is invisible while the file is small.

what the pin says	what everyone reads it as
this file is byte-identical	my inputs are unchanged
any edit invalidates me	edits to my section invalidate me

The failure mode is people, not machines

An hour of machine time is affordable. What is not affordable is what it does to behaviour.

When correcting a six-line block costs fifteen re-runs, three things start happening. I watched all three:

edits get batched until there are enough to justify the ceremony, so the configuration is wrong for longer
the staleness line gets read past, because it is almost always red for reasons unrelated to what you are doing
someone proposes re-pinning without re-running, which is the point where the check stops meaning anything

A check whose cost is out of proportion to the change gets routed around. It does not get fixed, because it is not broken.

The repair is a declared partition

The pin became the hash of the unit's own block plus the sections genuinely shared between units, rather than the whole file:

pin = sha256(block for this unit) + sha256(shared sections)

Two properties matter more than the saving. The partition is declared, as rows in the file itself rather than inferred by a parser guessing at section boundaries, so it is readable and arguable. And it is recomputable by hand, since the recipe is printed next to the hash and anyone can rerun it with sha256sum.

Now an edit to one unit's block stales one unit. An edit to a shared section stales everything, and that is correct, because everything does depend on it.

What I would check earlier next time

There is a cheap diagnostic I did not run for months. Take the last N edits to the file and ask, for each, how many units the pin invalidated and how many actually depended on the change.

edits sampled          20
units invalidated      15 each time
units actually affected 1, 1, 1, 2, 1, 15, 1, ...

One row genuinely needed all fifteen. The rest were a partition problem visible in five minutes of arithmetic over the git log, and the ratio is the argument for fixing it.

Two things I keep

Hash granularity is an interface, not an implementation detail. Choosing the whole file is choosing "any edit invalidates every dependant", and that choice deserves the same scrutiny as any other contract, especially in a file that many components share.

"Correct but disproportionate" is a defect class of its own. It never shows up as a failing check, so nothing surfaces it. It shows up as people batching their edits, skimming past a red line, and eventually proposing to weaken the check, and by then the conversation is about the check rather than about its granularity.

The OOM killer stopped my acceptance check four times in two days. It was rebuilding the project to read a number.

Mahiro Hirakawa — Mon, 14 Sep 2026 18:58:29 +0000

There is a step at the end of each work session that decides whether each unit of the project is accepted. It gathers, for every unit, whether its test suite passed, whether its checks are current, whether its documentation was regenerated, and prints one line per unit.

It kept dying.

acceptance read: killed

Four times in two days. No error, no stack, just gone. The machine's memory limit, with nothing else running. On the fourth kill I stopped restarting it and read what it does.

It was compiling the project. To find out whether the tests passed, it ran them.

The word "read" was doing a lot of work

The step is called a read, and I had thought of it as one for months. It sounded like gathering. What it actually did, for each of fifteen units, was invoke the build and wait.

That is not a read. It is a second full build whose output nobody looks at except to extract a boolean.

The memory limit was not the problem. It was the only thing telling me anything. A build is the most expensive operation in the repository, and the acceptance step ran fifteen of them in sequence, on a machine that can comfortably run one.

Why re-running looks like the rigorous choice

Here is the argument I would have made for the old design, and it is not stupid:

A recorded result is a claim about the past. The tree may have changed since. Running it now is the only way to know it holds now.

That is correct in general and wrong here, because of what the recording already is. Every control run appends a block to a permanent record, and that block carries the commit it ran at plus its verbatim output:

control block   unit=projection  commit=26ef4cc  OK_ALL controls=13  run=2026-09-13

So the record is not someone's summary. It is the control's own print, pinned to a commit. Asking "did this pass at this commit" is answerable from it exactly, and re-running the build answers the same question by doing the same work again.

step	question	how it should answer
the control	does this hold right now	run it, append the print
the acceptance read	what do the controls say at this commit	read the record

Two steps, two questions. The acceptance step had been answering the first question while being named for the second, so it duplicated the control's work and added nothing except a way to run out of memory.

The staleness question, which is the real one

"What if the record is from an older commit?" is the objection that matters, and it is answerable without building anything:

ACCEPT_GATE  current=S  next=face  recorded=9/15  stale=0

The read compares each block's commit against the commit being accepted. If they differ, that unit is stale and says so. Staleness becomes a printed number rather than something the design tries to make impossible by brute force.

That is the useful move: the cost of re-deriving was buying a property that a comparison provides for free. Freshness is a question about two identifiers. It does not need a compiler.

A general form I keep coming back to

Any step whose name is a verb of observation, read, check, report, gather, audit, that takes minutes and consumes gigabytes, is probably not observing. Something inside it is re-deriving a fact that already exists somewhere.

The tell is resource usage that does not match the verb. An observation's cost should scale with how much there is to observe, not with how expensive the thing observed was to produce.

Two things I keep

A read that builds is not a read. Separating "produce the fact" from "collect the facts" is not tidiness; it is what makes the collecting step cheap enough to run at every boundary instead of once a day when someone is brave.

When a step keeps being killed, read it before restarting it. I restarted three times and reduced parallelism twice, both of which are treatments for a resource problem. The fourth kill was the first time I asked what the step was actually doing, and the answer took two minutes to find.

My coverage check did arithmetic instead of looking. All 23 pointers were wrong.

Mahiro Hirakawa — Mon, 14 Sep 2026 18:03:19 +0000

A table in one document points into another. Each of its 24 rows carries a source pointer saying where the thing it describes actually lives, so a reader can go and check.

The coverage check for that table reported complete, always. Here is what it did:

rows in the table        24
rows carrying a pointer  23
rows that are a vessel    1
24 - 23 - 1 = 0 uncovered  ->  complete

That is arithmetic over counts. It never opens the file the pointers point into. A pointer can name a line that does not exist, or the wrong line, or a line in a file that was deleted, and the subtraction still comes out to zero.

When something finally resolved them, all 23 were wrong.

Wrong by one or two lines, which is the informative part

Not wrong by hundreds. Every pointer was off by one or two lines against the target.

That number tells you which failure this is. A pointer that drifts as a file grows ends up off by however many lines were inserted above it, which after a year is a large and varied number. Off by one or two, uniformly, across every row, is not drift. It is an off-by-one in whatever produced them, present from the first commit.

symptom	drift	never worked
offsets	large, varied	small, uniform
when introduced	gradually	at creation
repair	re-point and add a freshness check	fix the generator, then re-point
how it reads in a log	"these have gone stale"	"these have gone stale"

The two look identical in the report. They want different repairs, and only the distribution of the offsets tells you which one you have.

The same defect in four tools in one week

While fixing it I found that four separate readers were joining two tables the same way: by row order.

# join by position: row 7 here describes row 7 there
for (i = 0; i < left.length; i++) check(left[i], right[i])

Position is a key that every insertion invalidates. Add a row to one side and every pair after it is silently wrong, which is exactly what had happened. Nothing errors, because both sides still have rows at those indices.

All four became keyed joins, matching on an identifier that travels with the row:

# join by key: an unmatched row on either side is a named finding
for (row of left) check(row, right.byId[row.id] ?? MISSING)

The difference that matters is not correctness in the happy case. It is that a keyed join can report MISSING, and a positional join has no way to express it. A positional join cannot fail; it can only be wrong.

What replaced the subtraction

Each row now carries the pointer plus a verbatim quote of what it points at. Coverage is the count of rows whose quote still matches the bytes at that location, which is a question about the world rather than about the table's own shape.

Retired rows moved out of the denominator entirely, as pointer lines that no counter reads.

The satisfying part: after the repair, no count moved. The table had said complete and, once every pointer was actually resolved, it was complete. Every one of those 23 pointers had been wrong, and the number the check reported was right. It was right for no reason.

Two things I keep

A coverage number computed from counts is a statement about your table, not about your code. If the check never opens the thing being pointed at, it cannot go red for the reason you built it, and the correct total it prints is a coincidence you should not spend.

Read the distribution of the errors, not just how many. "All 23 are stale" and "all 23 are off by one" are the same count and different bugs. One asks for a freshness check. The other asks you to fix the thing that wrote them, or you will re-point 23 rows and generate 23 more.

Every correction I made to my spec added a requirement. The retired rows were still being counted.

Mahiro Hirakawa — Mon, 14 Sep 2026 17:32:23 +0000

The specification here is append-only. You never edit a row. To change one you retire the old row, which stays in the file under a retired heading, and stand a new row beside it. The old text remains readable, so a reader six months later can see what the rule used to say and why it moved.

Two tools read that file to build the list of requirements the code must satisfy. Neither of them knew what a retired heading was.

So a retire-and-stand produced two live requirements where there had been one. Correcting a row did not fix the specification. It grew it.

The number that should have caught it moved the wrong way

The acceptance check reports how many declared requirements are backed by a test:

dod_rows=263  printed=191/263

Retire one row and stand its replacement, and both sides move:

dod_rows=264  printed=191/264

The denominator grows. The numerator does not, because the retired row names a check that nothing prints any more, which is the entire reason it was retired.

action	requirements	backed by a test	coverage
before	263	191	72.6%
retire one row, stand its replacement	264	191	72.3%
correct ten more rows	274	191	69.7%

Coverage therefore falls slightly every time anyone corrects anything. Nobody reads a one-row drop as a bug. It reads as the ordinary friction of a growing spec, and it points in a direction that discourages exactly the work you want: precision.

Why no reader noticed

Both tools locate requirements by matching a heading pattern. A retired heading matches it, because retiring changes a prefix in the text and not the shape of the line. There was no bug in either reader. There was a concept, "retired", that lived in the writing convention and had never been given to anything that reads.

That is the general shape and it is worth naming: a convention that is not in the data is not a convention, it is a habit. Every reader is free to disagree with it, and they will disagree silently, because each one is behaving exactly as written.

The fix is not a skip

The obvious repair is to teach both readers to skip the retired prefix. I did not do that, for the same reason I would not fix a duplicated constant by copying it a third time.

Instead the heading rule became data: one table saying which heading forms are live, which are retired, and what each one means, with the readers deriving their filter from it.

[[heading]] form = "## R-\d+"          status = "live"
[[heading]] form = "## retired_R-\d+"  status = "retired"

A third tool, added later, cannot get this wrong by forgetting. It can only get it wrong by not reading the table at all, which is a visible omission rather than a silent divergence.

The check that makes the property hold

The property worth testing is not "retired rows are skipped". It is that retiring is denominator-neutral:

assert: retire one row and stand its replacement
        dod_rows before == dod_rows after
        printed  before == printed  after

That check would have failed on the day the convention was invented, long before anyone accumulated corrections. It tests the operation rather than the state, which is what you want for anything append-only, because state grows and operations are the only thing you can pin down.

The same defect, one layer up

The append-only rule exists so that history is never destroyed. The retirement convention exists so that append-only does not mean accumulating contradictions. The readers were built for the first rule and nobody had told them about the second.

Whenever you add a rule that makes an earlier rule survivable, check whether everything that consumed the earlier rule knows about the new one. The second rule is usually written as prose, in a different document, months later, by someone who is thinking about the writing side and not the reading side.

Two things I keep

Retiring a row must not change any count. If correcting something moves your coverage number, the number is measuring your editing activity rather than your code, and the first thing it will teach people is to correct less.

Put the convention in a table the readers load. "We mark these as retired" is a sentence in someone's memory. A heading-form table is a thing a new tool has to read, and its absence is loud.

I checked 18 claims in my own spec. Five were false, and one tag byte was responsible for one of them.

Mahiro Hirakawa — Mon, 14 Sep 2026 17:06:02 +0000

The specification for this project is written in precise language on purpose. It says things like "this map is a homomorphism" and "these two operations form a lens" rather than "this converts X to Y", because the precise words carry obligations and the vague ones do not.

Nobody had ever checked whether the obligations were met. I built something that reads each claim, resolves the terms against the actual definitions, and returns valid, invalid or unknown.

validity table  rows=18  valid=9  invalid=5  unknown=4

Nine of eighteen. Every invalid one was the same mistake in a different costume: a precise word used at the strength of its vague synonym.

One tag byte

The clearest case. The spec said the encoder is a homomorphism into byte strings under concatenation. That is a strong claim with a single obligation:

enc(a · b) = enc(a) ++ enc(b)

Here is what the encoder does for a pair:

enc(pair(x, y)) = tag ++ enc(x) ++ enc(y)

One byte in front, to say what kind of thing follows. That byte is why the encoding is unambiguously decodable, so it is doing real work and it is not going away. It also means the equation is false for every input. Not approximately true, not true up to a constant: there is no pair for which the two sides are equal.

What the encoder actually is, is a map that is injective and prefix-free. Both of those are properties the rest of the system relies on, and neither of them is a homomorphism. The word was reached for because "structure-preserving" felt like the right shape, and nothing ever asked which structure.

Four names for four different things

Another row used DAG, strict order, covering relation and transitive closure as if they named one concept. They are related and they are not interchangeable:

term	what it is
DAG	a graph with no directed cycle
strict order	transitive, irreflexive
covering relation	the edges of the Hasse diagram, no intermediate element
transitive closure	every reachable pair, as an edge

A DAG is not transitive. A covering relation is deliberately not transitive, and taking its transitive closure gives you back the order, so the two are opposite ends of the same operation. The spec had one sentence that was true of one of these and was written as though it applied to all four, and the code that read that sentence chose a different member than the code that wrote it.

A claim its own document denies

The third kind is the one I found most uncomfortable. One row asserted the round-trip laws for a lens, which require:

get(put(v, s)) = v
put(get(s), s) = s

A different section of the same document, 130 lines away, states plainly that the two maps do not compose to the identity. Both sentences had been read many times, by me, and never on the same day.

This is the failure mode of a long specification. Contradictions are invisible when the two halves never have to be true at the same moment, and nothing in prose forces that moment to happen.

The four unknowns are not failures

Four rows came back unknown, and keeping that as a separate answer is the part I would defend hardest.

One example: a comparison is total only under a hypothesis supplied by a different layer. Whether the claim holds is not decidable where the claim is written. It is not false, and marking it false would produce a repair for a problem that may not exist. It is also not valid, and calling it valid would launder an assumption into a fact.

Collapsing unknown into invalid is the more tempting direction, because it makes the report look rigorous and gives everyone something to do. It is still wrong. The honest output is a count of decided and a named list of what a decision is waiting on.

Why prose lets this happen

None of these five claims survived contact with a checker, and all of them had survived years of reading. Prose does not force you to name the structure, so "this preserves structure" never has to answer which structure, and the reader supplies whichever one makes the sentence true.

The repair is not to write less precisely. It is to make each precise word carry its obligation in machine-readable form: when the document says homomorphism, something should be able to ask which operation on each side, and then check it.

Two things I keep

A precise word is a promise, and the cost of breaking it is paid by a reader who believed you. If you are not prepared to have "homomorphism" checked, write "encodes" and lose nothing.

Contradictions inside one document are found by evaluation, never by reading. Two statements a hundred lines apart are never in your head simultaneously, so no amount of careful reading brings them together. Something that resolves both at once does it in milliseconds.

One row in my ledger said 12 of 3. Six review rounds later, not a single verdict had changed.

Mahiro Hirakawa — Mon, 14 Sep 2026 16:58:35 +0000

There is a ledger here where each row compares our implementation against someone else's on one axis, and carries a letter: ahead, level, behind. Each letter is supported by numbers and by pointers into both codebases.

A reviewer reads rows and disagrees with them. Six rounds ran over one section of that ledger. Every round found real defects. Not one round changed a letter.

What the rounds actually found

Three kinds of thing, none of which is a wrong conclusion.

A closing sentence that was false about its own code. One row ended by saying the operation returns "not contained" for a failing name. The code returns three different outcomes: not contained, absent for a name outside the root, and unsettled for a race that ran past its bound. The sentence was not a simplification. It named one of three and implied it was the set.

Two counts glued together with a slash. One row wrote:

probes=12/3

which reads as twelve of three. There was no ratio. There were two independent facts, twelve probes run and three conditions declared, and a slash between them because the surrounding rows all had a shape like n/m and this one wanted to match.

Cited ranges that did not hold what the row claimed. A reader that resolves every citation and compares its extent against the claim found several rows off:

range_wider     row 4:  cites 9 lines, needs 7
range_narrower  row 10: cites 6 lines, needs 12

The second is the dangerous one. A citation narrower than its claim points at a fragment and asserts the whole.

kind	changes the verdict	changes whether you can check it
clause names 1 of 3 outcomes	no	yes
two counts written as a ratio	no	yes
cited range wider or narrower	no	yes

Six rounds without a letter moving is not obviously good news

From the outside this is indistinguishable from churn. Six passes, a pile of edits, the same conclusions as before. If someone asked what the review accomplished, "we corrected some numbers" is a weak answer and it is also the true one.

What changes the reading is what each round left behind. Over those six rounds the checking apparatus gained three readers it did not have: one that resolves a cited range and compares it to the claim, one that refuses a count written as a pair unless both halves come from declared sources, and one that parses the ledger's tables rather than trusting their shape.

The repaired rows show what the readers now require. A count stays a count, and a sample keeps its population beside it:

OK_CONTAIN   conditions=6
OK_ATTEST    probes=12  declared=3        was: probes=12/3
arms=9  sampled=7  ratio=7/7              a pair, with the thing it is a pair of

So the rounds were not producing opinions about the rows. They were converting a prose ledger into a machine-checkable one, one defect class at a time, and the letters were never what was in question.

The test that tells churn apart from progress

After a review round, name what now runs that did not run before. If the answer is nothing, that round produced an opinion.

An opinion is worth having. It is just not worth six rounds, and it does not accumulate, so the seventh round rediscovers what the third one found.

The three readers above will catch their defect classes in every future row without anyone reading carefully. That is the part that compounds. The corrected rows are a one-time payment.

What I would do differently

I would say out loud, after round two, that the letters look stable and the evidence does not, because that is a different activity from reviewing the claims and it deserves to be named. Rounds three through six were an audit of the citation apparatus wearing the costume of a content review, and nobody said so, so each round started by re-reading rows that were not the problem.

Two things I keep

A count and a ratio look identical and mean different things. 12/3 passed several readings because it had the shape of the rows around it. Any number written as a pair needs both halves to come from somewhere declared, and a reader that enforces that is fifteen lines.

Review rounds that never move a verdict are measuring your evidence, not your claims. That is legitimate work and it is a different job. Name it when you notice it, because the two jobs want different reviewers, different stopping conditions, and, most of all, a different definition of done.

The same one-line table-parsing bug turned up in five separate tools. I fixed it four times.

Mahiro Hirakawa — Mon, 14 Sep 2026 16:05:05 +0000

A lot of this project's specification lives in markdown tables, and several tools read them. Parsing a table row looks like a solved problem:

const cells = line.split('|').slice(1, -1).map((s) => s.trim());

That line is in more places than anyone had counted.

Two cells broke it

One spec table has a cell containing a set definition and another containing the length notation for a sequence. Written out, they carry a vertical bar inside the cell:

| E0 | the admitted set  | N_t := {x \| E(x,t) = true} |
| S0 | length            | \|·\| over a byte sequence   |

Both are correct markdown. The bar is escaped, which is exactly what the format asks for. The naive split does not care that it was escaped, so both rows fractured into the wrong number of cells.

The symptom was not "the parser is broken"

This is the part I want to keep. What the run reported was:

verified_lying: 2
premise_defs:   2

Two definitions in the specification appeared to rest on nothing. That is a genuinely alarming finding, several steps more serious than a lost cell, and it is what everyone reacted to. The first stretch of work went into the two definitions themselves, both of which turned out to be fully reproduced by a proof and a test.

A parser failure arrives dressed as a content failure. The cells shifted, so a column that should have held a reproduction pointer held something else, and every check downstream honestly reported what it was given.

what the log said	what it meant
two definitions unverified	a column moved by one
the specification has a hole	the reader has a hole

It was the fourth time, and then the fifth

The same class had already been found in a proof-declaration printer and in the map generator. Each time, the person who hit it fixed the tool in front of them and carried on, which is a completely reasonable thing to do and is why it kept happening.

A fifth instance turned up in a shared cell-splitting helper the same week.

Nothing anywhere knew how many table readers existed. There was no wrong decision to point at. There was an absent list.

The repair was the inventory

The fix that ended it was not a better regex. Every tool that parses a table became a declared row, and a check reads that list:

OK_TABLE_READERS  readers=7/7  escaped_pipe=1  lookalike=1

A new tool that reads tables has to be added to the list, and each one is exercised against a fixture containing both hazards. Adding the row is not optional in the sense that the check counts the readers it can find in the tree and compares them to the declared ones.

The second hazard is the one worth stealing. The mathematical notation does not actually want U+007C. It wants U+2223, the DIVIDES character, which looks nearly identical in a monospace font and is a completely different codepoint. A reader that learned about the escaped pipe and nothing else still gets that row wrong, silently, and the fixture now contains one of each.

Two things I keep

A bug found more than twice is not a bug. It is a missing inventory. The third occurrence is the signal to stop patching and start asking how many places have this shape, because the patch rate is clearly not keeping up with the discovery rate.

Expect a parser failure to present as a content failure. Whatever sits downstream of the parse will report faithfully on the garbage it received, and its report will be about your data rather than about your reader. When a check suddenly claims something alarming about content that was fine yesterday, suspect the thing that fed it before you suspect the content.

My guard asserted the output path was a scratch directory. The row landed in the permanent log anyway.

Mahiro Hirakawa — Mon, 14 Sep 2026 15:58:36 +0000

There is a rule here that tests never write to the real record store, and unlike most of my rules it was already a device rather than a sentence. Before its first write, any check asserts that its data root is set and points outside the real tree, and refuses otherwise.

The rule exists because of an earlier incident, where an end-to-end run appended three entries into the permanent log and nobody noticed until the numbering looked wrong. The guard was the repair.

Months later a reviewer tool minted a row into that same permanent log. The guard was green on the run that did it.

What the guard compared

Reduced to its shape:

const root = process.env.RECORD_ROOT;
if (!root) throw new Error('RECORD_ROOT not set');
if (root.startsWith(REAL_TREE)) throw new Error('refusing to write the real tree');
// ... proceed to write under root

The scratch root was a fresh directory under the temp area. The string did not start with the real tree's path. The assertion was true.

Inside that scratch tree was a junction, the Windows form of a directory symlink, created to give the tool the same layout it sees in the real checkout without copying a large tree. When the tool opened a file under that junction, the runtime followed it. The write went where the link pointed.

what the guard looked at	what the write used
the string in an environment variable	the path the filesystem resolved
the intent, before anything opened	the destination, after every link was followed

Both are called "the path". They are not the same object, and a string comparison cannot tell them apart.

The fix is two lines, and one of them is the interesting one

import { realpathSync } from 'node:fs';

const root = realpathSync(process.env.RECORD_ROOT);        // resolve, then compare
if (root.startsWith(realpathSync(REAL_TREE))) throw new Error('...');

Resolving both sides is the obvious half. The half worth keeping is the second change, which is to move the assertion from the intent to the handle. Instead of checking a path before writing, check what you actually opened:

const fh = await open(target, 'w');
const { ino, dev } = await fh.stat();
if (isUnderRealTree(dev, ino)) throw new Error('...');

That version cannot be defeated by anything the resolver does, because it is asking the thing that was actually opened rather than the request that produced it.

The general shape

A guard that validates an intent is not guarding the action. Between the value you check and the effect that happens, there is a resolver, and the resolver is a component you did not audit.

This is not specific to symlinks. The same gap is everywhere:

what you check                 what resolves it            what actually happens
a path string          ->      symlinks, junctions, ..  -> a different inode
a hostname             ->      DNS, /etc/hosts, proxy   -> a different server
a database name        ->      connection pooling, DSN  -> a different database
a branch name          ->      remote refs, worktrees   -> a different ref
a container mount      ->      bind mounts              -> the host filesystem

Every row is the same failure waiting. The check passes on the name, the effect lands on the resolution, and the distance between them is invisible in the log line that says the check was green.

Plant the resolver in your own fixtures

The lasting change was not the two lines. It was that this class became something the test fixtures deliberately produce.

A related checker in the same project had been silently skipping files it could not read, which nobody knew because no fixture had ever contained one. Its selftest now plants two: a dangling junction, and a file with permissions denied. The checker is required to report both rather than pass over them.

Once you know a resolver sits between your check and your effect, the fixture has to contain something that resolves surprisingly. Otherwise the only place that ever happens is production.

Two things I keep

Compare resolved values, or better, do not compare paths at all. Ask the opened handle where it is. A string is a request, and the answer to a request is a different thing than the request.

A rule that is already a device can still be wrong about what it measures. I had promoted this one from a sentence to a check and then stopped thinking about it, which is precisely the moment it is least examined. Being enforced is not the same as being correct.

I wrote three rules on Saturday. I broke all three on Saturday.

Mahiro Hirakawa — Mon, 14 Sep 2026 15:32:27 +0000

A long working day produced three lessons. Each got written down as one sentence in the log, the way a good lesson should be:

1. Name a task's success criterion by the line it will print, never by a phrase.
2. Regenerate the derived index before mirroring the tree, not after.
3. A bulk removal runs every reader of the thing being removed, not just the classifier.

Three clear sentences, each earned by an actual failure that cost actual hours. All three were violated again before the same day ended.

It was not a discipline problem

I want to be precise about this, because the comfortable reading is that I was tired and should have been more careful.

I knew all three rules. I had written them myself, that day, in a file I had open. At the moment each one applied I was not thinking about rules at all. I was thinking about the task, which is the correct thing to be thinking about, and that is exactly when a sentence in a log provides no resistance whatsoever.

A sentence has no enforcement surface. Nothing consults it. The gap between writing a rule and needing it is measured in hours and in context switches, and a rule that lives only in a document loses every one of those.

What survived versus what did not

The interesting number is not three violations. It is that the rules already implemented as devices were not violated, because violating them was not available.

rule	form	violations that day
name success by its print	sentence	1
index before mirror	sentence	1
removal runs every reader	sentence	1
a control must not write the real data store	device, refuses	0
a claim of having read needs a verified quote	device, refuses	0

The bottom two are not better rules. They are the same kind of rule, wired into something that stops.

Each sentence became a refusal

The repair was not to write the sentences more forcefully. Each one became a thing that declines to proceed.

$ plan --add task=rework --success="the merge is clean"
REFUSED  success token 'the merge is clean' is not printed anywhere in this tree
         success= must name a line some device actually emits

The planner now reads every print name the tree can produce and refuses a success criterion that is a phrase rather than a line. The first rule stopped being advice and became a thing you cannot type past.

The second became data. The step order moved out of my head and into a file, with a checker that reads it:

order = ["edit", "repin", "recite", "index", "mirror", "controls",
         "append", "index", "accept", "record"]

The checker refuses an index regeneration while an edit is still in flight, and refuses to commit an index without the index's own success line present in the same run. There is no longer a version of the sequence that lives in my memory and can be reordered by hurry.

The third turned a one-off script into a tool with the readers declared as rows, so "run every reader" stopped meaning "remember which tools parse this" and started meaning "iterate a list that someone has to add to".

The shape underneath all three

Every one of these was the same failure wearing a different hat: a step performed by habit where it should have been performed by a device.

That gives a usable test for any rule you are about to write down:

Name the thing that will refuse. If you cannot, you have written a note, not a rule.

A note is still worth writing. It is the record of a real failure and it is what the device gets built from later. The error is counting it as a fix and moving on, because the next violation then looks like carelessness rather than like the predictable behaviour of an unenforced sentence.

Two things I keep

A rule applied by hand is a rule you will break, and you can measure how fast. Write the sentence, then count violations for a week before believing the problem is solved. Mine had a zero percent survival rate over about eight hours, which is more informative than any amount of confidence about my own carefulness.

Knowing a rule does not change behaviour at the moment the rule applies. At that moment your attention is entirely on the task, which is where it belongs. Design for that, rather than for a version of yourself who is thinking about process while doing work.

6,718 of my 10,238 code blocks had no owner. Every single one was in the same directory.

Mahiro Hirakawa — Mon, 14 Sep 2026 15:03:19 +0000

The rule is that every block of code stands under a declared owner, so that for any function you can ask which stated responsibility it serves and get one answer. The rule had existed for a while. Nothing had ever counted it.

The first census:

kind	total	under an owner	orphan
code blocks	10,238	3,520	6,718
Rust functions	2,004	1,088	916
proof declarations	600	213	387
cross-layer edges	50	35	15

Sixty-six percent of the code blocks had no owner. That is the kind of number that ends a meeting.

The big number was one decision

The census also printed where the orphans were, and the answer was all of them in the same place:

blocks with no owner    = 6,718
of those, under tools/  = 6,718   (100%)

The instrument directory had never been declared. Every script that checks, counts or verifies something lived outside the ownership model entirely, not because anyone decided to exclude it but because the model had been written while thinking about the product and the tools had accumulated beside it.

So this was not 6,718 problems. It was one missing declaration, and the number went to zero when a layer for the instruments was declared and the devices were listed under it. The largest finding in the census was the cheapest one in it.

The small number was real work

The 916 orphan Rust functions looked smaller and were not:

orphan Rust functions = 916
top three crates      = 142, 101, 91
crates involved       = 14

Nothing groups those. They are spread across fourteen crates, and the top crate holds fifteen percent of them. There is no single row that covers this. It is fourteen passes of reading, one crate at a time, deciding for each function which stated responsibility it serves or declaring it a helper of one.

The same census produced two opposite kinds of finding

finding	count	concentration	what it is	cost
unowned blocks	6,718	100% in one directory	a missing declaration	one row
unowned functions	916	15% in the largest crate	genuine backlog	fourteen passes

The counts point the wrong way. Read as bare totals, the first looks like the catastrophe and the second like a rounding error. Grouped, they swap places.

What grouping is actually doing

It is asking whether the findings share a cause. A flat list of N findings silently asserts that there are N decisions to make, and that assertion is almost never checked, because a census feels finished the moment it produces a number.

The failure mode is not that you file 6,718 tickets. Nobody does that. What actually happens is you look at 66 percent, conclude the rule is unenforceable at this scale, and quietly stop measuring. A number too large to act on gets treated as a statement about the rule rather than a statement about the code, and the census that produced it does not get run again.

So the census prints the concentration next to every count, and a finding without a grouping column is not yet a finding.

The estimate that started it was wrong

One more thing from the same run, worth its own paragraph.

The task was written with an estimate in it: roughly 653 proof declarations. That number came from someone's memory. When the work actually counted, three different ways of counting gave three different answers:

declarations, by three greps:  554   589   600

The right move is the one that was taken: report all three, flag the disagreement, and do not correct any of them yet. Each grep encodes a different guess about what counts as a declaration, so the spread is a statement about an undefined term rather than about the code.

Choosing one, especially the one closest to the estimate in the brief, would have converted an open question into a fact by preference. Once a denominator is written down, nobody re-derives it.

Two things I keep

Print the concentration beside every count. A census that reports totals makes every finding look like the same kind of thing, and the whole value of a census is telling apart the finding that is one decision from the finding that is a month.

When two instruments disagree about a denominator, the disagreement is the finding. Not an inconvenience to resolve before reporting. The gap between 554 and 600 is the most informative output of that run, because it says the term being counted was never defined.

My coverage print read 13 of 13 on every run. It was incapable of printing anything else.

Mahiro Hirakawa — Mon, 14 Sep 2026 14:58:42 +0000

One of the crates classifies a failure into its worst matching category. The category list is an array where each entry carries a code, and a check prints how much of that list the classifier actually covers.

OK_WORST_CLASS  arms=13/13

Thirteen of thirteen, every run, for as long as the check had existed. I read that as good news for months. It was not news at all.

The numerator was built out of the denominator

Reduced to the shape, the check was this:

const ARMS: [Arm; 13] = [ /* ... */ ];

let codes: Vec<Code> = ARMS.iter().map(|a| a.code()).collect();
println!("OK_WORST_CLASS arms={}/{}", codes.len(), ARMS.len());

codes is ARMS with a function mapped over it. map does not drop elements. So codes.len() is ARMS.len() by construction, and the fraction is n/n for every possible value of n, including zero.

This is not a subtle bug in the counting. There is no input, no state, no ordering that makes this print anything except full. The check had no failure mode. A check with no failure mode is a println! wearing the costume of a check.

The first repair was also wrong

The obvious fix is to make the numerator mean something. So the numerator became "arms whose code agrees with what the classifier returns for that arm", which is a real property that can be false.

The print went to arms=13/13 again. This time it deserved the number, and it was still not measuring the thing anyone was afraid of.

The fear is not "an arm I wrote disagrees with itself". The fear is an error case with no arm at all. A new variant gets added to the error type. Nobody adds a row to ARMS, so the classifier silently files it under a neighbour.

what the fraction can see	what it cannot
an arm whose code disagrees	a case with no arm
an arm that was written wrong	a case nobody wrote
drift inside the list	absence from the list

Both halves of the fraction were still derived from ARMS. A list cannot report its own omissions. Whatever is missing from it is missing from the numerator and the denominator at the same time, which is exactly the cancellation that keeps the ratio at 1.

The denominator has to come from the other side

The repair is not in the counting. It is in where the denominator is read from.

// before: both sides from the list of arms I wrote
arms = agreeing_arms / ARMS.len()

// after: the denominator is the thing being covered
arms = agreeing_arms / ErrorKind::ALL.len()

ErrorKind::ALL is the closed set the classifier is supposed to cover. Now adding a variant without adding an arm moves the denominator and not the numerator, and the print falls to 13/14 on the commit that introduces the gap.

The general rule, which is obvious once stated and easy to violate once writing code:

A coverage number is only meaningful when its denominator comes from a different source than its numerator. Same file is a warning. Same array is a guarantee of nothing.

The class is findable by machine

The part I find most useful is that this does not require noticing. An audit can read a check's own source and ask whether the expression producing the numerator is derived from the expression producing the denominator. If it is, the check gets a named finding. "Aggregation gap" is now a category in that audit, sitting next to the other ways a check lies.

It fired on two checks the day it was written, and both had been green since the day they were merged. Neither had ever been suspected, because a green check is not something anyone goes looking at.

$ audit --class=aggregation-gap
FAIL  worst_class    numerator derived from denominator (map over the same array)
FAIL  arms_declared  denominator is a compile-time constant of the numerator's source

Two things I keep

A ratio that has never moved is not evidence of health. It is an untested hypothesis about your own arithmetic, and the longer it has held at 100% the less anyone remembers to doubt it. The question to ask a green check is not "is this right" but "what input makes this print something else".

Absence is the thing coverage is for, and absence is exactly what a self-derived denominator cannot represent. If the list defines both what you have and what you should have, you have written a tautology and given it a percent sign.

My pipeline recorded 0 of 15 into an append-only file before anyone read why. The real answer was 9.

Mahiro Hirakawa — Mon, 14 Sep 2026 14:32:30 +0000

The release boundary for this project is ten declared steps. Two of them are next to each other and are separate on purpose:

step 8   accept_read     compute the acceptance verdict for every unit and print it
step 9   record          write that verdict into the permanent record

Step 8 looks. Step 9 commits what was seen. The gap between them exists so a person can read the verdict before it becomes history.

I ran them as one command chain. Both steps are always needed and the second always follows the first, so joining them felt like removing a redundant keystroke.

What the read actually returned

Every one of the fifteen units came back red. Not one of them for a reason that had anything to do with the units:

FAIL_SEMANTIC  reason=op_unknown  op_unknown=51

A generator had started emitting a new form of annotation that morning. The checker that validates those annotations works from a closed vocabulary. That is the right design, and the 51 new names were simply not in it yet. The checker refused, correctly, the way a closed vocabulary is supposed to refuse.

Nothing was broken. One list needed 51 entries added to it.

The record step did not care

It had no reason to. Its input was a verdict and the verdict said red, so it wrote the truth it was handed:

ACCEPT_RECORDED rows=15 accepted=0/15 commit=f6d74b1

That file is append-only. Records are never edited and never removed, which is a property I chose and still want, because a record you can quietly fix is not a record. The 0/15 is permanent. It sits in the history with a later block superseding it, and anyone reading that history sees a day when everything failed followed by a correction.

Twenty minutes later, after the vocabulary was declared and the map regenerated, the read was run again and this time read first:

ACCEPT_RECORDED rows=15 accepted=9/15 commit=2536c12

Nine. The false number cost more to carry than the missing 51 entries cost to add.

`&&` protects the thing you were not worried about

The chain even looks like a safety property. "Only record if the read succeeds" sounds like exactly the guard you would want.

The read did succeed. It ran to completion, computed a verdict for all fifteen units and printed its reason, then exited 0. Exit 0 was correct. A measuring device that reports a red measurement has done its job perfectly.

what the chain checks	what it does not check
did the read process run	is the verdict worth committing
did it exit non-zero	did a human look at the reason
did it crash	is the red about the subject or about the instrument

That last row is the one that bit. op_unknown=51 is a statement about the checker's own configuration, not about the fifteen units it was pointed at. No exit code can carry that distinction, because both cases are a successful run that produced a red.

The rule I now hold

A step that observes and a step that commits the observation are never one invocation.

Not joined by &&. Not joined by ;. Not wrapped in a convenience script. Two commands, and in practice two turns, so that something slow enough to read sits between the measurement and the permanent consequence.

$ boundary --step=accept_read      # look, and stop
$ boundary --step=record           # commit, as a separate decision

This is the same shape as plan and apply, or a dry run against a real one. It is routinely collapsed for the same reason I collapsed it: the two commands always run together in the happy path, and the happy path is where the habit forms.

Two things I keep

Exit 0 means the measurement ran. It never means the measurement was good. A device that says "red" and exits 0 is a healthy device, so any automation keyed on the exit code is reading the wrong field for this question.

The cost of collapsing "look" into "act" is paid in permanent records, not in retries. If the consequence were a retry I would never have noticed, and I would have kept the shortcut. The reason this one taught me anything is that the file it wrote to cannot be edited.

DEV Community: Mahiro Hirakawa

A six-line edit made fifteen units stale. The freshness check was working correctly.

The pin was over the whole file

The failure mode is people, not machines

The repair is a declared partition

What I would check earlier next time

Two things I keep

The OOM killer stopped my acceptance check four times in two days. It was rebuilding the project to read a number.

The word "read" was doing a lot of work

Why re-running looks like the rigorous choice

The staleness question, which is the real one

A general form I keep coming back to

Two things I keep

My coverage check did arithmetic instead of looking. All 23 pointers were wrong.

Wrong by one or two lines, which is the informative part

The same defect in four tools in one week

What replaced the subtraction

Two things I keep

Every correction I made to my spec added a requirement. The retired rows were still being counted.

The number that should have caught it moved the wrong way

Why no reader noticed

The fix is not a skip

The check that makes the property hold

The same defect, one layer up

Two things I keep

I checked 18 claims in my own spec. Five were false, and one tag byte was responsible for one of them.

One tag byte

Four names for four different things

A claim its own document denies

The four unknowns are not failures

Why prose lets this happen

Two things I keep

One row in my ledger said 12 of 3. Six review rounds later, not a single verdict had changed.

What the rounds actually found

Six rounds without a letter moving is not obviously good news

The test that tells churn apart from progress

What I would do differently

Two things I keep

The same one-line table-parsing bug turned up in five separate tools. I fixed it four times.

Two cells broke it

The symptom was not "the parser is broken"

It was the fourth time, and then the fifth

The repair was the inventory

Two things I keep

My guard asserted the output path was a scratch directory. The row landed in the permanent log anyway.

What the guard compared

The fix is two lines, and one of them is the interesting one

The general shape

Plant the resolver in your own fixtures

Two things I keep

I wrote three rules on Saturday. I broke all three on Saturday.

It was not a discipline problem

What survived versus what did not

Each sentence became a refusal

The shape underneath all three

Two things I keep

6,718 of my 10,238 code blocks had no owner. Every single one was in the same directory.

The big number was one decision

The small number was real work

The same census produced two opposite kinds of finding

What grouping is actually doing

The estimate that started it was wrong

Two things I keep

My coverage print read 13 of 13 on every run. It was incapable of printing anything else.

The numerator was built out of the denominator

The first repair was also wrong

The denominator has to come from the other side

The class is findable by machine

Two things I keep

My pipeline recorded 0 of 15 into an append-only file before anyone read why. The real answer was 9.

What the read actually returned

The record step did not care

&& protects the thing you were not worried about

The rule I now hold

Two things I keep

`&&` protects the thing you were not worried about