DEV Community

Mahiro Hirakawa
Mahiro Hirakawa

Posted on

My check ran green every time and its result could never reach the thing that needed it

An acceptance report reads one cell per unit. For weeks it read the same thing on every single unit:

accept: map=0  reproduced=absent    (every unit, every run)
Enter fullscreen mode Exit fullscreen mode

absent is not a failure. It means the acceptance step looked for a result and found nothing there. Meanwhile the check that produces that result was running, and passing:

OK_REPRODUCED independent=agree
Enter fullscreen mode Exit fullscreen mode

Green in one place, absent in another, permanently. Two separate causes, and neither is in the check itself.

Cause one: the result had no route

The acceptance step does not call the check. It reads the check's result out of the permanent record, by rule, because the whole point is to read what was written rather than what someone claims.

Writing into that record requires being on a list of devices whose output is routed there. This check was not on the list.

devices routed to the record:  [ ... 28 entries ... ]
reproduce_check:               not present
Enter fullscreen mode Exit fullscreen mode

So the check ran, printed a correct result, and the result went nowhere the acceptance step could see it. Not a bug in the check. Not a bug in the acceptance step. A missing edge between them, and neither end had any reason to complain about it.

question answer
did the check run? yes
did it pass? yes
is its result in the record? no
does anything report that gap? no, until you ask for the cell

Cause two: the checker pinned a file that grows

The other half was a mismatch that appeared, disappeared, and reappeared with no pattern I could see:

AUDIT_ROW id=9  MISMATCH  printed=57f54773a953  recomputed=e7b5a84af32e
Enter fullscreen mode Exit fullscreen mode

A map is built over 358 input files, and its freshness is a digest over all of them. Among those 358 was the append-only record itself.

The record is only read by the map. But it is in the pinned input set, so every append to the record changes the map's digest, and an append happens on every run. The map was stale the instant anything was written down, including the writing-down of the map's own run.

  inputs = [
    ...source files...,
-   record_band/*,          // read-only, but pinned: every append invalidates
  ]
Enter fullscreen mode Exit fullscreen mode

The rule that came out of it: a record you only read must not be in your pinned input set. Pin what you are derived from. A log you consult is not an input in that sense, and treating it as one makes your freshness a function of how often other people write things down.

Why both of these hide so well

Neither shows up as a red. The first produces absent, which reads as "not applicable yet". The second produces an intermittent mismatch that always has a plausible local explanation, because something genuinely did change.

And both are invisible from inside either component. The check is correct. The acceptance step is correct. The map is correct. The defect lives in the relationship, and the relationship is not owned by anything that runs.

The rule

"It ran and passed" and "its result is readable by whatever depends on it" are different claims, and only the second one is worth anything.

The first is a property of the check. The second is a property of the wiring, and the wiring is the part with no tests, because a missing edge has nothing to attach a test to.

If a component reports its results into a registry, the thing worth asserting is not that the component works. It is that a fresh result from that component appears in the registry within one run. That assertion fails on day one when you add a new component and forget to register it, which is exactly when you want to hear about it rather than weeks later when someone asks why a column is empty.

Top comments (1)

Collapse
 
raju_dandigam profile image
Raju Dandigam

@mahirhir, this is a sharp example of why “producer passed” and “consumer observed the result” must be separate contracts. The missing route created permanent absence, while pinning the append-only record made freshness self-invalidating. Would you add a reachability check proving every required result producer has a route to the permanent record, plus a dependency-cycle check that forbids outputs from appearing in their own input digest?