I Signed a Regression and Almost Shipped It

The regression report said blocked=True. A critical task had regressed. The promotion proceeded anyway. The attestation was written. The workload was admitted to production.

I signed a regression and almost shipped it — on HivePlane v0.2.0, the release where I added the promotion gate I'd been promising since v0.1.0. The gate that was supposed to make "certified" mean "safe" admitted a workload its own regression report had just flagged.

This is the article about when "certified" does not mean "safe" — and about a gate that admits on the wrong evidence.

Four ways the gate admitted wrong

Finding	What the gate did	Why it was wrong
#522 (Critical)	recertify_and_promote admitted even when the regression report was blocked=True	The report was a footnote, not a precondition
#523 (High)	Default min_production_runs_survived=50 made certified unreachable	The production gate deadlocked against its own threshold
#530	The drift detector compared failure counts across corpus versions	Growing the benchmark auto-quarantined a healthy agent
#527	The regression report diffed against any-status prior attestation	A non-comparable baseline masked a real regression

Pass-rate thresholds are necessary. They are not sufficient. Each of these is a way to be green and wrong.

The signature on a regression

522 is the one that kept me up. The flow was: re-certify, generate a regression report, promote. The report said blocked=True — a critical task had regressed. The promotion proceeded anyway. The attestation was written. The workload was admitted.

The regression report was generated, attached to the record, and ignored. It was a comment, not a gate.

tip: A check that annotates but does not block is a suggestion, not a control. If your promotion can succeed while the regression report is red, your regression report is decoration.

The fix: the regression report is a hard precondition. recertify_and_promote refuses if the report is blocked:

if regression_report.blocked:
    raise PromotionRefusedError(
        workload,
        f"regression report is blocked: {regression_report.critical_failures}",
    )

A critical regression feeds the refusal reason. The promotion cannot complete until the regression is resolved or explicitly waived with attribution.

The unreachable status

523 was a different kind of admission failure. The default min_production_runs_survived=50 meant a workload had to survive 50 production runs before it could be certified. In a field test that runs once, against a local stack, that's never. The production gate deadlocked against its own threshold — the workload could pass every benchmark forever and never reach certified.

A gate that's unreachable is a gate that doesn't exist. Operators will route around it, and a bypassed gate is an expensive dashboard. The threshold has to be reachable in the environment where it's enforced, or you've built a status cycle, not a ladder.

The baseline that wasn't

530 is the one that changed how I think about drift. The detector compared failure counts across corpus versions. So if you grew the benchmark from 10 tasks to 20 — because the learning loop added a reviewed case — a healthy agent that passed 10/10 now "failed" 10 of 20 by your accounting. Corpus growth looked like decay. Growing the benchmark auto-quarantined a healthy agent.

The fix: drift is measured against the agent's own baseline with comparable corpora. You compare the same tasks, not different counts. A corpus version change is a new baseline, not a drift signal.

527 was the same lesson on the promotion side: the regression report diffed against any-status prior attestation, including a quarantined one. So a regression against a known-bad baseline looked like no regression. The baseline has to be the last certified record, or you're comparing against a version that was never safe.

What I learned

A signature on a known-bad artifact is worse than no signature — it carries the authority of certification without the safety. The regression report must block, not annotate. Identity binds to the attestation, not the manifest: change the hash, lose the certification. And the survival gate must be reachable, or you've built a status cycle, not a ladder.

The v0.2.0 field test demonstrates the promotion gate blocking a regression with a replayable diff (scenario S3), drift auto-quarantine and reinstatement (S4/S5), and the regression diff API. Full evidence in the field-test report; the certification and promotion design is in the certification v2 design doc.

References

• HivePlane — pip install hiveplane==0.2.0
• Field test report (36/36)
• Certification v2 design · Learning loop design
• Tutorial — Certify, Promote & Survive Drift

When was the last time your promotion gate refused something the regression report had flagged? If you can't remember, your report is a comment — not a gate.