A table in my spec cited a source file by line number, 23 rows of it. Every one was off by one or two. The coverage check next to it had been green the whole time.
The check was green because of how it counted:
covered = total_rows - uncited_rows - vessel_rows
= 24 - 23 - 1
= 0 uncovered # green
Coverage was arithmetic. It counted how many rows carried a citation, never whether any citation pointed at what it claimed. A pin can be wrong in every row and this subtraction still comes out to zero.
Nothing read the pins
That is the part worth sitting with. It was not that the reader was lenient, or that the comparison had an off-by-one of its own. No device read those cells at all. They were text in a table that a human would follow by hand, and the only automated thing in the neighbourhood was counting how many of them existed.
So the failure had no way to surface. A wrong pin produces no error because nothing dereferences it. It produces no drift warning because nothing compares it. It survives every run of the check that appears to be about exactly this, because that check is about a different question wearing similar words.
| what it sounds like it checks | what it actually checks |
|---|---|
| every row cites its source | every row has a non-empty citation cell |
| the citations are correct | nothing |
| the source has not moved | nothing |
Why the pins drifted in the first place
Line numbers are a positional reference into a file that other people edit. Insert a line near the top and every citation below it is wrong, silently, in a commit that has nothing to do with the table. The citations were not written carelessly; they were correct when written and then decayed by an edit somewhere else.
Any reference whose correctness depends on a coordinate that other work moves will drift. The question is only whether anything notices, and a count of citations never will.
Content pins instead of coordinates
The repair was to change what a citation is:
src = 02:317
quote = "the verdict is admitted before the body is read"
The line number stays, because it is useful for a human opening the file. It is no longer what the reference rests on. The quote is, and a quote is checkable: read the file, find the text, report whether it is there and whether the line number still matches.
Now three distinct outcomes exist where there was one:
| what the reader finds | verdict | who acts |
|---|---|---|
| the quote at the cited line | the pin is good | nobody |
| the quote at a different line | the pin moved
|
the tool repins it |
| no quote anywhere | the source changed under the citation | a person looks |
The middle one is the one that matters. Most drift is a pure move, and a mechanism that can distinguish a move from a real change turns a manual re-audit into a regeneration.
The structural half: join by key, not by position
The count above worked positionally: row 1 of the table against row 1 of the source list, and so on. That is the same class of mistake as the line numbers, one level up: it assumes an order nobody is maintaining.
Rejoining the two tables by an explicit row id let the check say something it could not say before: which specific row is uncovered, and which specific pin moved. A positional reader can only produce a count. A keyed reader produces a name, and a name is what makes the failure actionable.
The difference shows up in the output rather than in the logic:
before rows=24 cited=23 uncovered=0
after rows=24 cited=23 uncovered=0 moved=17 missing=0 row=s12.3-r09 moved_to=02:319
The first line cannot name anything, so a bad run and a good run print the same characters.
This was the fourth positional reader in one codebase that had to become a keyed one. The others were a symbol ranking, a set of expected outputs, and a list of check targets. In all four the positional version had been green for months.
What I would look for elsewhere
Any place where documentation cites code, this pair of questions separates a real check from a decorative one.
Does anything dereference the citation, or only count it? And if the thing cited moves, does the reference report a move, a break, or nothing at all?
If the answer to the first is "only count it", the second does not matter yet. Nothing is watching.
Top comments (0)