A table in one document points into another. Each of its 24 rows carries a source pointer saying where the thing it describes actually lives, so a reader can go and check.
The coverage check for that table reported complete, always. Here is what it did:
rows in the table 24
rows carrying a pointer 23
rows that are a vessel 1
24 - 23 - 1 = 0 uncovered -> complete
That is arithmetic over counts. It never opens the file the pointers point into. A pointer can name a line that does not exist, or the wrong line, or a line in a file that was deleted, and the subtraction still comes out to zero.
When something finally resolved them, all 23 were wrong.
Wrong by one or two lines, which is the informative part
Not wrong by hundreds. Every pointer was off by one or two lines against the target.
That number tells you which failure this is. A pointer that drifts as a file grows ends up off by however many lines were inserted above it, which after a year is a large and varied number. Off by one or two, uniformly, across every row, is not drift. It is an off-by-one in whatever produced them, present from the first commit.
| symptom | drift | never worked |
|---|---|---|
| offsets | large, varied | small, uniform |
| when introduced | gradually | at creation |
| repair | re-point and add a freshness check | fix the generator, then re-point |
| how it reads in a log | "these have gone stale" | "these have gone stale" |
The two look identical in the report. They want different repairs, and only the distribution of the offsets tells you which one you have.
The same defect in four tools in one week
While fixing it I found that four separate readers were joining two tables the same way: by row order.
# join by position: row 7 here describes row 7 there
for (i = 0; i < left.length; i++) check(left[i], right[i])
Position is a key that every insertion invalidates. Add a row to one side and every pair after it is silently wrong, which is exactly what had happened. Nothing errors, because both sides still have rows at those indices.
All four became keyed joins, matching on an identifier that travels with the row:
# join by key: an unmatched row on either side is a named finding
for (row of left) check(row, right.byId[row.id] ?? MISSING)
The difference that matters is not correctness in the happy case. It is that a keyed join can report MISSING, and a positional join has no way to express it. A positional join cannot fail; it can only be wrong.
What replaced the subtraction
Each row now carries the pointer plus a verbatim quote of what it points at. Coverage is the count of rows whose quote still matches the bytes at that location, which is a question about the world rather than about the table's own shape.
Retired rows moved out of the denominator entirely, as pointer lines that no counter reads.
The satisfying part: after the repair, no count moved. The table had said complete and, once every pointer was actually resolved, it was complete. Every one of those 23 pointers had been wrong, and the number the check reported was right. It was right for no reason.
Two things I keep
A coverage number computed from counts is a statement about your table, not about your code. If the check never opens the thing being pointed at, it cannot go red for the reason you built it, and the correct total it prints is a coincidence you should not spend.
Read the distribution of the errors, not just how many. "All 23 are stale" and "all 23 are off by one" are the same count and different bugs. One asks for a freshness check. The other asks you to fix the thing that wrote them, or you will re-point 23 rows and generate 23 more.
Top comments (0)