AI assistance disclosure: This article was drafted with the help of Claude. All technical content, design decisions, code references, and screensh...
For further actions, you may consider blocking this person and/or reporting abuse
The deterministic-first rule is the load-bearing wall of this article and it deserves to be more famous. My version of it, from building similar gates: the boundary is not only about cost, it is about failure visibility. Deterministic extraction fails loudly, the parser breaks, the graph misses edges, somebody notices. Qualitative AI work fails silently and fluently. So the seam itself is where I put the most engineering: schema validation, grounding checks, confidence computed from external verifiable signals instead of the model's self-report, because a generator and an LLM reviewer can share the same blind spot, and auto-merge makes that correlation expensive. The 115 self-healing PRs in 30 days is an impressive number; the one I would watch next to it is how many candidate PRs the deterministic gates rejected before any human saw them. That rejection ratio is the health metric of the whole loop.
Good catch on the architecture, and I checked instead of guessing. For Self-Healing there's no "rejected, discarded" bucket. A failing PR gets handed into a retry loop that keeps pushing commits until it passes, capped, but the cap just stops retrying, it doesn't close the PR. So "killed before a human saw it" is close to zero by construction.
Round-trip rate is the real number: 44% of merged Self-Healing PRs over the last 30 days needed at least one extra commit. Two months ago that was 75%, which tracks with the "same fix becomes a new Guide" mechanic in the post.
Zoomed out to all PRs, not just Self-Healing: our deterministic gate failure rate over the same window is 22%.
Checked instead of guessed is the whole culture difference, and your numbers are better than the metric I proposed. You are right that a rejection ratio cannot exist in a loop with no discard path. Round-trip rate is the correct replacement, and the slope is the impressive part: 75 to 44 in two months means the Guides mechanic is compounding. The loop is not just persistent, it is learning at a measurable rate.
The successor number I would watch: inventory sitting at the retry cap. A capped PR that neither passes nor closes is where this design parks its debt, invisible to both round-trip rate and gate failure rate. If round-trip keeps falling while the capped-out pile stays near zero and young, the loop is healthy end to end. If that pile ages, it is silent failure wearing a new costume. And 22% deterministic rejection across all PRs says the gates are doing real work, which was the load-bearing claim all along.
Checked again rather than model it. Self-healing's capped-out inventory is zero right now, nothing sitting at the retry limit. Widened to all open PRs in the repo: nine have at least one failing check, and every one is zero to one day old, most from today. Nothing has aged into the pile.
So right now it reads as your healthy case, near zero and young. That's a point-in-time check, not an invariant, but it's a real answer instead of a projection.
Measured twice in one thread instead of modelled once, and the honest asterisk on your own good news ("point-in-time, not an invariant") is the kind of asterisk that builds trust in every other number you publish. Zero at the cap and nothing older than a day is the healthy case exactly.
The cheap upgrade from snapshot to invariant: age of the oldest failing-check PR as a standing gauge with a threshold alert. One scheduled query, and the silent-failure costume gets a smoke detector. At that point the loop is instrumented end to end, and honestly, this comment section became the appendix your post deserved.
That's the natural next step, cheap enough there's no excuse not to build it. Appreciated the pushback through this whole thread, it's what made the post's central claim stand on real numbers instead of a headline count.