We ship a small open-source bridge that copies health data into a training site once a day. It runs as a GitHub Actions workflow in the user's own repository, and it has exactly one way to reach a human: a daily workflow that turns the run red on purpose, because a red run is what makes GitHub send an email.
Last week we measured that channel. It had been green every morning for three months while three months of data were missing.
Here is the bug, how we measured it, the second bug hiding behind the fix, and the general rule we got out of it.
The failure
A long catch-up run had broken part-way: it asked for 200 days of history, wrote 90, and the second window was refused. On a laptop the next check names the gap and exits non-zero, like this:
$ ghti status
A backfill stopped part-way on 2026-07-04 and was never finished:
2026-07-05 to 2026-11-06 are still missing on intervals.icu.
Finish it with: ghti backfill --from 2026-07-05 --to 2026-11-06
$ echo $?
7
We ran the same check against the same broken run with GITHUB_ACTIONS=true:
| where | exit code | what the run page said |
|---|---|---|
| laptop | 7, gap named with dates | — |
| GitHub Actions | 0 | ## Google authorization OK |
Zero. Green. "OK". For three months.
Why
The line that found the gap looked like this:
if not in_actions():
gap = read_gap_from_diary()
if gap:
report(gap)
The reason in the code was true: the diary is a local file, and on a hosted runner the filesystem is new every time, so the diary is always empty. Reading it there would be pointless.
The conclusion drawn from that true reason was wrong. "The diary is empty in CI" was turned into "don't look for a gap in CI", which means the cloud could not see a gap at all — and the cloud is where everyone who installed this with Use this template actually runs it. The laptop, where we develop and where the check worked, is the one place that didn't need it.
The thing we could not measure
Our first instinct was to measure the email: what does the buyer's inbox actually receive when the run goes red?
We couldn't. GitHub's own notification docs don't specify the format of the email a failed run sends, and a stranger's inbox isn't measurable from here anyway. Betting on an undocumented format isn't a measurement.
So we measured the thing we could pin down with certainty: the run that sends the email. Its exit code and its run summary page. That is a smaller claim than "the email says X", and it is the claim we can actually defend. If you can't measure the thing you care about, measure the nearest thing you can, and say which one you measured.
The fix: state, not diary
The gap stopped being a diary entry and became state: two dates in state/gap.json, next to the encrypted token, in the directory the workflows already commit — and already commit with if: always(), so a run that dies still saves it. It outlives the runner.
Two details that mattered more than they look:
- The note holds only two dates and a timestamp. It lands in a user's repository, which may be public. A gap note is not a place for payloads or identifiers.
-
The check now reads it unconditionally. In CI it prints the range and writes a Stopped part-way summary that gives the browser route first (Actions → Sync → the day in
from) and the command second, because the person reading a red email on a phone is not about to open a terminal.
The second bug, found by asking "what if the fix doesn't save?"
A note only helps if the step that saves state/ actually pushes it. So we measured that step failing, in a sandbox repository:
- a commit landing in between → the save step rebased onto the branch and died in
CONFLICT, exit 1; - a branch that requires a pull request →
[remote rejected], exit 1.
Both times the note never left the runner. And a new test spells out the cost: the next cloud check exits 0 over the missing months, and the run page reads Google authorization OK. Exactly the bug we had just fixed, re-entering through the back door.
So the save step is now one script shared by the four workflows. It replays the runner's state/ on top of the branch instead of rebasing onto it, retries a push that lost a race, and when it still cannot save, it says on the run page which days are missing, the two ways to close them, and why a push gets refused.
Exit codes, while we are here
A check used as an alarm needs more than zero and one. 1 for "three days are missing" and 1 for "your API key is wrong" is an alarm that rings when nothing is wrong and stays quiet when everything is. Ours now keep the answers apart — 7 a run stopped part-way and those days are still missing, 5 only a reminder, 3 the authorization really expired, 1 the check itself could not run, 2 not configured — and every code is in --help and in a README table with the reason next to it.
The rule
A check that reads state written locally is worth nothing on a disposable runner. It will report the absence of a problem it structurally cannot see, and it will do it in the voice of a passing test.
If you have a scheduled check anywhere, four questions:
- Does it read anything a previous run wrote? Where does that live?
- Does that location survive the runner? A committed directory, an artifact, an external store — or a filesystem that is gone in 30 seconds?
- If a run dies part-way, does the state still get saved? (
if: always()is cheap.) - If the save fails, does anything say so — or does silence read as success?
The expensive part of this bug was never the missing data. It was three months of a green check telling us there was nothing to look at.
Code, measurements and tests are all in the open: https://github.com/antivex/google-health-to-intervals
This article was written with AI assistance and published by the antivex team.
Top comments (0)