Your Release Process Should Fail Closed
Last week our Obsidian plugin disappeared from its store listing. Nothing was wrong with the code. The plugin's manifest said version 1.3.0, but no matching release existed — so the listing dropped. A human checklist had every step on it. One step just quietly didn't happen, and every other step reported success anyway.
That is what failing open looks like: a check that errors, times out, or gets skipped counts as done. Most release processes fail open, because they're built from checklists, and checklists don't execute themselves.
So we built the opposite and shipped it: release-gate, a single 245-line Python script (standard library only, nothing to install) whose whole design is that the default answer is no.
The five checks
Give the gate a directory of release artifacts and it runs, in order:
- SCAN — the directory exists and contains at least one artifact.
- INTEGRITY — no artifact is zero bytes. A truncated or placeholder upload fails here, by name.
-
EXPECTED — every artifact you named with
--expectis actually present. -
SEAL — SHA-256 hashes of every artifact are written to
SHA256SUMSin the release directory. -
VERIFY (with
--verify) — every artifact is re-hashed and compared against the seal. A changed byte, a missing file, or a file that was never sealed is a failure.
Every run, pass or fail, is appended to a hash-chained ledger: each entry records the hash of every artifact and chains to the previous entry, so a PASS is bound to the exact bytes that were checked. Change one byte afterwards and the old PASS no longer describes your release.
The part that matters: there is no override flag
No --force. No --skip-checks. A check that throws an error is recorded as a FAIL, not skipped. Sealing is refused if any earlier check failed. The exit code is the entire contract: 0 means every check ran and passed, 1 means do not publish.
That sounds strict until you've watched the alternative ship a broken release because somebody was in a hurry and the checklist "basically passed."
Watch it catch one byte
Real output, from running the shipped v1.0 today. Seal a two-file release:
$ python3 release_gate.py ./my-release --expect app-linux.tar.gz --expect app-windows.zip
RELEASE GATE v1.0 — SEAL — /tmp/rg-demo/my-release
SCAN: PASS
INTEGRITY: PASS
EXPECTED: PASS
SEAL: PASS
sha256 9a3719788de597c4… app-linux.tar.gz
sha256 774640a50264d6f7… app-windows.zip
VERDICT: PASS — ledger entry #1 (hash-chained) in ./rg-demo/ledger.jsonl
Append a single byte to one file — the smallest corruption there is — and verify:
$ python3 release_gate.py ./my-release --verify
RELEASE GATE v1.0 — VERIFY — /tmp/rg-demo/my-release
SCAN: PASS
INTEGRITY: PASS
VERIFY: FAIL
sha256 9cd6e59bd2a37184… app-linux.tar.gz
sha256 774640a50264d6f7… app-windows.zip
detail: VERIFY: app-linux.tar.gz: checksum mismatch
VERDICT: FAIL — ledger entry #2 (hash-chained) in ./rg-demo/ledger.jsonl
Exit code 1. The release does not ship, and the output names the check and the file instead of leaving you to guess.
The gate's own test suite makes the same point sixteen ways — good release, tampered release, stale checksums, a zero-byte artifact, a missing expected file, an unsealed release — asserting the exit code and the named failed check each time. I re-ran it today before writing this: 16/16 checks passed.
The second rule the incident taught us
The gate proves the bytes. It doesn't prove the publish happened the way the success message claims — so our second rule is: verify on the authoritative surface, never from the success response. When release-gate itself shipped (public repo, November 16), the push wasn't trusted until all 11 files were read back and compared byte-for-byte against the local package, and the v1.0 release wasn't called live until its page said so.
A gate that fails closed, plus a read-back that doesn't take anyone's word for it. Neither is clever. Together they've ended the class of incident that cost us a listing: the step that silently didn't happen.
Facts in this post, checkable on the repo linked above: release-gate v1.0, commit 6e9d39c, 11 files, MIT licensed; the self-test output and the seal/verify demo above were run on 2026-11-20 against the shipped code, and the demo hashes are from that run — yours will differ, the verdicts shouldn't.
Top comments (0)