I wrote a rollback rule and no way to read the number

Originally published on hexisteme notes.

I changed a publishing cadence against what my own measurements suggested, and — being responsible about it — I wrote down the condition that would make me undo the change:

The cadence guard is off. This two-hour interval runs against the measurements. If the early batch's view response stays at zero, report immediately and revert the cadence.

That sentence passes review. It records a decision, attaches a falsifying condition, and names the remedy. I would accept it in someone else's design doc.

A few hours later I went to check whether the condition had come true, and discovered there was no way to read the number.

Twelve subcommands, none of them about outcomes

The pipeline had twelve subcommands — new-brief, draft, ingest, approve, voice, render, gate, package, upload, status, veto, trend-scout — and not one of them read anything that happened after upload. The automation ended at the act of publishing. Everything downstream of that existed only on a web dashboard a human had to open.

So the criterion was unfireable from the moment I wrote it. The condition could become true and nothing in the system, including me, would find out.

Why it survives review

This defect is invisible on the page, and I think I know why.

When a criterion contains a threshold — "stays at zero", "error rate above 1%", "p95 over 300ms" — the sentence reads as operational. There's a number in it. Numbers feel like the hard part. But a threshold is only half of a verdict. The other half is who reads that number, when, and how. Without the second half the first half is decoration.

The same shape turns up wherever those two halves get written by different people at different times:

• a rollback criterion, and a deploy pipeline that queries no post-deploy metric
• an A/B stopping rule, and an experiment dashboard that doesn't carry that metric
• an SLO with no alert wired to it
• an incident playbook that says "once customer impact is confirmed" and no route to confirming it

The common structure isn't carelessness. It's that the criterion gets written at decision time, when you're thinking about the decision, and the instrument gets deferred to implementation time, when you're thinking about something else. The deferral never resolves, because nothing in the criterion's text says an instrument is missing. It reads complete.

The fix was forty lines

I wrote a read-only script that lists the videos and prints views, likes, comments, and hours elapsed. Forty lines against a public API.

That's the uncomfortable part. It was never hard. There was no architectural obstacle, no missing credential, no vendor limitation. The only thing standing between the criterion and a working instrument was that nobody had written down that those forty lines were required — and the criterion itself, the one document that should have said so, gave no sign.

The threshold belongs inside the instrument

The first version of the script ended like this:

if total_views == 0:
    print("⚠ previous batch at zero views — reversal condition met")

I ran it. It fired 1.6 hours after publication.

A brand-new channel starting from zero subscribers, 1.6 hours in, at zero views, is not a state that carries information. Nothing has happened yet. If I had left that instrument alone it would have cried wolf on every run, and after the third time nobody looks — which is strictly worse than having no instrument, because now there's a green-looking process producing a signal everyone has learned to discard.

And here is what stung: the answer was already in my own criterion. "If the view response stays at zero." Stays implies duration. I had written the time condition down in English and then dropped it on the way to code, without noticing, because natural-language criteria carry that kind of implication silently and a reader's mind fills it in for free. The compiler doesn't.

REVERSAL_MIN_HOURS = 12.0  # a zero before this is not a signal (new-channel cold start)

if elapsed_hours < REVERSAL_MIN_HOURS:
    print("verdict withheld — a zero here can't be read yet")
elif total_views == 0:
    print("⚠ reversal condition met")

How long must "no signal" persist before it becomes a signal? That question has an answer, and the answer belongs in the instrument as a constant — not in a comment, not in a runbook, not in the head of whoever happens to see the alert. If it lives outside the code, the judgement gets improvised fresh every time by whoever is looking, and that person will either (a) revert too early or (b) build the habit of ignoring the warning. Both are worse outcomes than never having written the criterion.

What I do now

When I write a decision criterion, the same commit carries two more things:

1. The command that reads the number. A human opening a dashboard is not instrumentation. It has to be a reproducible line I can run, and it has to exist before the criterion is considered written.

2. An explicit "not yet knowable" state. Not below-threshold — undecidable, as its own output. An instrument with only two states, met and not-met, is guaranteed to be wrong during the early window, and the early window is exactly when someone is watching most closely.

The self-check is one question: "Can I find out whether this condition is true right now, with a single command?" If the answer is no, what I wrote isn't a criterion. It's an intention that happens to contain a number.

When this is wrong

There's a case where the instrument isn't worth building: when it costs more than the verdict is worth. A one-off experiment you'll never rerun. A decision cheap enough that reverting is meaningless. For those, the honest move isn't to skip the instrument and keep the criterion — it's to write down "this is not observed" and stop there. An acknowledged blind spot is a much better artifact than an unfireable rule, because the unfireable rule tells everyone downstream, including future you, that something is being watched.

That's the shape this failure shares with two others I've written up: a monitor that stayed green for four days while nothing shipped, and tests that were green because the gate was never wired into the dispatch table. Each time, the artifact that was supposed to tell me something had no path to the fact it claimed to be watching — and in each case the artifact looked completely fine, because looking fine is what a threshold does.

More notes at hexisteme.github.io/notes.