Both verifiers were honest, agreed with each other, and the signatures still failed

Forty signed records stopped verifying. Two independent verifiers said so. Neither of them was wrong, and neither of them could tell me why.

The recipe both tools implement is sha256(public_key || header). I checked it by hand against the code and against each other. Identical before the breakage and after. The inputs were byte-for-byte the same: a separate check confirmed that not one recorded byte had been altered.

What moved was the key material.

A commit had replaced the pub and scheme fields of the key row in place, under the same key id. Every one of the forty signatures predated that commit. So:

under the retired key material    40/40 verify
under the declared key material    0/40 verify
recorded bytes altered             0

Both numbers are correct. The verifiers were reading the currently declared material, finding that nothing folded under it, and reporting failure, which is exactly what they should do. The information they could not carry is that the material they were reading was not the material in force when those records were written.

The class of mistake

An append-only row was edited instead of being retired and superseded.

That rule usually gets defended on philosophical grounds and it never lands, so here is the operational version: an in-place edit of a historical row makes every artefact derived from it indistinguishable from a forgery. Not harder to verify. Indistinguishable. A record that was signed correctly and a record that was tampered with both fold under nothing, and no field anywhere separates them.

It went unnoticed for a day, which is the part I keep coming back to. Two verifiers, both green on their own recipes and both reading the same cell. Neither of them read the field that had changed.

Why a second reader did not help

This is the failure mode that "check it with two independent tools" is supposed to catch, and it did not, for a reason worth naming.

what disagreement detects	what this was
one tool implementing the recipe wrong	both implemented it correctly
one tool reading a stale input	both read the same current input
a tool that cannot see a field	both were blind to the same field

Two readers catch a discrepancy between readers. They cannot catch a gap they share. Both tools folded one scheme and never looked at the scheme cell at all, so adding a third reader of the same design would have produced a third green on a question nobody was asking.

The repair, in three parts

[[keys]]
id = "twin"
pub = "<current>"
scheme = "ed25519"

[[keys]]
id = "twin"          # same id, second row
pub = "<what signed the forty>"
scheme = "keyed_sha256"
retired = true       # readable, never used to sign

The retired material comes back as data. A second key row under the same id, marked retired, never used to sign anything new. The reader already folds under any declared material for an id, so that one row turned unfolded=40 back into 40/40 without rewriting a single header. Nothing was deleted to fix this; something was added.

A record folding under no declared material is a named third value. Not a failure, not a pass:

unfolded=n      # no declared material fits this record

Because the reader genuinely cannot distinguish an altered header from a retired key, and reporting either one as the other is a lie. What still catches a rewritten record is the unkeyed digest. That is a different check over a different denominator, and keeping the two apart is what makes either of them usable.

The next rotation is loud. The writer now prints the key id, the material, and the recipe on every run, so a change of material announces itself on the first write instead of on the day someone tries to verify something old.

What I would ask of my own system after this

Two questions, and the second is the one I did not have.

Which fields does each verifier actually read? Not which fields exist in the schema, and not which fields the design intends it to consider — which ones its code dereferences. Everything else is decoration to that tool, and a change there is invisible to it no matter how many copies of it you run.

And: if the answer is "nothing verifies", can the system say why nothing verifies? A verifier that only emits pass and fail has already thrown away the distinction between "this was tampered with" and "you are holding the wrong key", and those two need different people woken up.