DEV Community

Cover image for A green build over a red tree
RLM Labs
RLM Labs

Posted on Originally published at rlmlabs.ai

A green build over a red tree

On 8 July 2026 Merlin committed a failing test with a green build.

My rules ask for a test with every fix. In session 1e823af6 the model did the textbook thing. The model wrote analyzer_layer_test.go first, a test meant to fail until the bug was fixed, then worked on the fix. The fix got stuck in a blocked change and never landed. The test landed.

My commit gate ran the build. A build compiles test files, so a test with a syntax error fails the build. A build does not run tests. A test which compiles cleanly and fails at runtime passes. The gate showed green, and the user's tree now held a failing test. Green is a colour, not a verdict.

Nobody did anything wrong. The model followed my rule. The gate checked what the gate was built to check. The defect was the gap between what the gate checked, compilation, and what everyone assumed the gate checked, the correctness of the change. I have watched the same gap swallow a release more than once, from the other side of a conference table, and here I had built the gap myself.

The fix added a second half to the commit gate. A staged change adding or editing a Go test file must now pass go test for those packages inside the staging area, after the build and before the commit. A failure rolls the change back and rejects the task with a hint: the test and the fix pass together, in the same task, or neither lands. Editing a test means owning the outcome of the package's tests.

Two limits, stated plainly. The test half covers Go only, while the build half covers more languages. And the rule fails open on infrastructure errors: when go test fails to start, the commit proceeds and the gate records the result as unavailable.

The test: read the command behind your green check, write down in one sentence what the command proves, and compare the sentence with what your team believes the check proves.

Source: Spec 003, AC-003-004-10, written 8 July 2026. The run's logs have aged out of my log store.

Top comments (0)