My requirements document lists ten layers by name. My source tree has directories numbered P0 through P9. Ten and ten. Coverage was being reported as nine of twelve units, which felt like ordinary unfinished work.
Two of the ten layers had no specification whatsoever. Not incomplete: absent. And the count could not see them, because the count was taken over the directory numbers.
backend units prev=12 now=14 reason=two declared requirement layers were never in the frame
honest fraction: 9/14, with a spec owed for two
The collision
The requirement and the directory tree both use short labels, and they do not mean the same things.
| label | in the requirements | in the source tree |
|---|---|---|
P2 |
Envelope | the code layer |
P5 |
Transition | the verdict layer |
spec/P2 exists. spec/P5 exists. Both are directories with real content and real tests. So a tool asking "does P2 have a spec?" gets yes, and the Envelope layer, which is a different P2, is never asked about at all.
Two namespaces, overlapping labels, and nothing declaring the correspondence. A hole looked filled because something else was standing in its slot.
The document said so, in its own words
This is the part I keep turning over. The verdict layer's specification contains a sentence saying that receipts, escrow and offline verification are "Transition, not this layer's rows".
It knew. It said so in plain text, in a file everyone had read.
That sentence was not a signal, because nothing compared the set of things the requirements demand against the set of things some specification claims to own. A disclaimer is only information if something is collecting disclaimers.
Why the denominator was wrong rather than the numerator
The instinct on seeing 9 of 12 is to go find the missing 3. That is a numerator question, and it has an answer, and the answer is wrong.
12 = number of directories that exist, minus some bookkeeping
14 = number of layers the requirements actually declare
A denominator derived from what exists cannot count what is missing. It is the same shape as measuring test coverage over the files you remembered to include: the number is real, the set is not the set you meant, and nothing in the output says which set was used.
What went in
A namespace map, as data. One row per requirement layer, naming the directory that owns it:
[[layer_map]]
req = "Transition"
spec = "P11"
and a check that prints the join:
REQ_LAYERS covered=n/n missing=<names|0>
A requirement layer with no owner is now a count with a name in it. Not a footnote, not a sentence in a synthesis document that someone has to remember.
Declare the things that do not exist yet. The two missing layers got declaration blocks with a status of declared and no implementation behind them. That sounds like paperwork and it is the load-bearing part: a declared-but-absent thing renders red. An undeclared absent thing renders as nothing at all, which is indistinguishable from fine.
Three rules
Check what your denominator is derived from. If it comes from the implementation, it measures the implementation's opinion of its own scope. Derive it from the requirement, the schema, the spec, the contract: something written before the code and not by it.
Two namespaces with overlapping labels need a declared mapping, and the mapping has to be data. Kept in anyone's head, the collision is invisible precisely where it matters, because both sides resolve to something.
Make absence declarable. The difference between "we have not built this" and "this never occurred to anyone" is whether there is a row for it. Only one of those two states can go red.
Top comments (0)