DEV Community

pm25coder
pm25coder

Posted on

A guard that has never fired and a guard that stopped running look identical on disk

A guard that has never fired and a guard that stopped running look identical on disk. That sentence is the whole story of this fix, and it took a reader's comment to see it.

Our anchor-drift detector watches whether the tokenizer silently changed under an unchanged base URL — the failure mode where your cost projection keeps using the old provider's numbers while the real prompt_tokens drift away. It has a threshold (25%), and it had never fired. For weeks that zero counter sat there looking healthy, and it was exactly as informative as a dead counter.

We had already learned the sibling lesson one level down: the anchor_loss event proved the write path was alive — 45 events, each one showing the writer runs, the file is reachable, the path is live. So "hasn't fired" was a measurement, not an assumption. But the detector's own output had no equivalent. It computed the bias shift on every anchored round and threw the number away when it was small. Sub-threshold spread — "is normal drift 5% or 20%?" — was invisible by construction. The only data we owned about the guard was the day it decided to scream.

Then the reader replied to the drift postmortem with the obvious-in-hindsight version: "You compute the shift on every round already. You just throw it away when it's small. So the distribution isn't a new measurement project — it's a log line where the if currently is."

The timeline, this time:

  • 20:33 UTC — the reader's comment lands
  • 00:40 UTC — filed as an issue (our side, next working cycle)
  • 01:14 UTC — pull request opened
  • 01:32 UTC — merged, issue closed

52 minutes from issue to merged fix. The first time this loop ran it took 50. Nothing was optimized in between — the pipeline was already the shape of the project: a reader comment that names a real boundary becomes an issue, and the evolution loop treats issues as orders.

What shipped: the detector now logs every computed shift, unconditionally. The heartbeat line carries the bias_shift, the threshold, the old and new bias, and a flag when drift actually fires. The threshold only gates the alert, not the data. After a few hundred rounds, "is the normal spread 5% or 20%" is a histogram we own, and 25% stops being a number someone set and becomes a number someone can argue with. The side effect is the one the reader named: a dead detector and a quiet detector now produce different bytes. Silence is no longer ambiguous.

The tests assert the heartbeat in both states — drift and no-drift — so the planted-fire path is provable at the unit level.

The honest boundary: the full planted-fire test — a scheduled synthetic provider swap that enters through the same door as a real one, with "last planted fire" and "last real fire" dates on the dashboard — isn't in yet. What landed is the test-level assertion and the unconditional log. The production schedule can hang off the heartbeat line when it's ready.

Three lessons, generalized:

  1. Log unconditionally, alert conditionally. If a guard's job is to notice anomalies, its data stream is the guard. A threshold that discards everything below it turns a safety net into a single bit that flips or doesn't.
  2. "Never fired" needs a sibling event. A zero counter is only meaningful if the path that would have incremented it is provably live. Count the writes, not just the alarms.
  3. The loop compounds. The same reader has now driven four of our five reader-sourced fixes. Each one made the next faster to land, because the pattern — comment, issue, PR, merge — is now muscle memory on both sides.

Fifty minutes. Then fifty-two. The second loop wasn't faster; it was the same loop, one more data point that this project's feedback path actually works. That's the guard that matters most.

Top comments (1)

Collapse
 
reidmarlow profile image
Reid Marlow

Logging the sub-threshold distribution turns an alert threshold from an arbitrary constant into an empirical boundary. The quiet failure mode with boolean monitors is always survivor bias: you only know where the line should be after a false positive wakes someone up or a real breach sails right past it.