Last Sunday I published a post about the agent fleet that builds my backlog. It ended with a section called "What I have not run," in which I said the retrospective pass was designed, was accepted, and as of that morning was code that had never run. The next sentence said the lenses and the trigger arithmetic had shipped into my standards a few hours before the post did.
Both halves were wrong by the time anyone read them.
The first retrospective merged at 15:30:35 Eastern that afternoon. That is twenty-nine minutes before the timestamp on the post claiming it had never run, and five hours and forty minutes before I actually pressed commit. The machinery had shipped two days earlier, not a few hours. And the post closed by asking whether specialist lenses reading a batch of merged stories at once would find the drift the design promises, or whether that was one more thing I believed because I wrote it down. The answer had been sitting in a merged file on my own main branch since mid-afternoon.
I am not going to edit that post. Originals stay as they were. But it is a strange way to learn that I was writing about my own system from memory instead of from it.
So: it ran. Twice, over fifteen merged stories and then seven. It found real things. And then it found something worse than anything on either board, which is that everything it produced the first time had quietly gone nowhere.
What the thing is
After a batch of stories merges, eight specialist lenses read the whole batch at once. Architecture, integration, testing, UX, product scope, process, documentation, business. Each is a separate agent with a written charter, each looks only for the class of problem that lens exists to notice, and each must either file a finding or explicitly pass. Then everything they file lands on one board and gets sorted into work to do now, work for the roadmap, or things that are permanently out of scope.
The premise is one I still believe. Individual code review is at the wrong altitude for a whole class of defect. Three stories can each invent a slightly different way to do one job, and every review passes, because from inside any one story there is nothing to compare against. You only see it by looking at ten at once.
It is generative and never gating. It runs after the work is merged, and the only things it can produce are new work and durable learnings. It cannot block anything.
That turns out to matter more than I thought.
What it caught
It works. I want to be clear about that before I spend the rest of the post on how it failed, because the findings are real and I would not have gotten them any other way.
One authorization boundary, expressed three ways. The free-versus-paid line in the product lives in a redact ? null : ... ternary in the GraphQL resolver, in a synthesisAllowed early return for the derived facts, and in independent tier reads inside two REST mirrors. One story drew three blocking review findings, all of which its own reviewer characterized as one family: a field derived from the redacted blob, or an action gated on it, that the decision table missed. A fourth was found live after the code review passed. The defect was never in a mechanism. It was in the gaps between three, and no single review sits where the gaps are.
A route is reachable only if three registries agree. A page in this app exists for a user only if it appears in the nav array, has an entry in the sidebar icon map, and ships a segment layout that mounts the app shell. Four merged stories touched the nav registries. One satisfied both nav halves and shipped no layout, so clicking through to Purchases made the entire navigation disappear on the hub that story exists to anchor. Four more routes were still sitting in that third state when the retro ran.
A test that was empty by construction. One story's acceptance suite built a recorder sink, called the function under test without wiring the recorder, and asserted the sink was empty. It was empty because nothing could ever have written to it. That assertion carried an acceptance guarantee and could not have failed for any change to production code. On a sibling story with the same shape, deleting an entire acceptance loop left 58 suites and 1,658 tests green.
The milestone shipped its ends and not its middle. The active milestone's own sentence is that a stranger can find a county, see what intelligence exists for it, pay, and receive a bundle. The first batch shipped find and see. The second batch also did not ship pay. The two commerce stories were priority zero and ready for both batches and built in neither. Nothing went wrong to cause that. Every pull was defensible in isolation, and those two cards were simply never next when a slot freed. The fabric is very good at building what is on the board and structurally incapable of noticing that the board stopped describing the plan.
That last one is the finding I would most want a human to have caught, and no human did.
What happened to them
The first retro adjudicated three of its findings to stubs, which is the strongest disposition it has. A stub is a scoped piece of work with a title, a rationale, and a sketch, ready to become a card.
Two days later the second retro opened with this:
The 2026-08-16 board proposed three stubs. None were built.
Two of those three classes recurred in the very next batch. The third is better than that. It had proposed a single reachability check that would supersede the existing icon invariant and absorb the check a queued story was already going to write, and it warned in as many words that otherwise there would be "two checks that have to agree about the same population."
One story shipped its own check. Another shipped a third. There are now three separate tests walking the same route population, which must all agree with each other, which is precisely the class the stub existed to eliminate and precisely the outcome it predicted in writing.
Nothing failed. No step errored. The proposals were made, written to a board, committed to the repository, and then the next run started and nobody read them. A ceremony whose only outputs are proposals and learnings is worth exactly what happens to its proposals, and what happened to mine was nothing.
This is where being generative and never gating stops being a design virtue. I chose that property deliberately, because a retrospective that can block a merge becomes a thing people route around. What I did not think through is that it leaves nothing at all standing between a proposal and the floor. Every other quality mechanism I have gets its force from being able to refuse something. This one had no teeth by construction, and I never asked what would give it any.
The gauge that said this was fine
Here is the part that made me sit back.
The retro instruments its own health. It was designed with the obvious failure mode in mind: that charters accumulate observations forever and never convert any of them into a real check or a real rule, so the ceremony slowly turns into a diary. So it tracks two numbers. How many standing observations are held across the eight charters, and how many have ever graduated out into automation or process.
After the second run those numbers were nine and four. That is a healthy ratio. That is the ratio you want. The retro's own report says so, in the section named for the rot it exists to detect.
Both numbers were true. The whole time they were true, every stub the ceremony had produced was being dropped on the floor. Stubs proposed versus stubs ingested was three and zero, and no instrument anywhere in the system was watching that pair.
I did not build a broken gauge. I built a correct gauge pointed at the wrong thing, which is worse, because a broken gauge eventually reads implausible and a correct one never does. Nine and four looks like a system working. It is what a system working looks like. It is also what this looks like.
The counter that was counting in another directory
Then I went looking, the way I did last month, and found the same shape twice more in the same week.
The retro is not supposed to fire after every batch. Small batches are supposed to accumulate: runs of two and two and two should add up and fire one retrospective over all six stories. That accumulation is the mechanism, written into the design document, and it is the thing that makes the compounding claim true rather than aspirational.
The accumulator kept its state in a file resolved against the current working directory. Every session of the fleet runs from a fresh git worktree. So every run opened a brand new counter at zero, found nothing in it, and reported its source as absent, its verdict as defer, its reason as below the floor.
absent reads as "this is the first run." It actually meant "the state you want is in a different directory."
Measured against real history: three runs, two merged stories each, from three real worktrees. All three read absent. Six stories, zero retrospectives. The cross-run accumulator is the exact mechanism the design specifies, and worktree-local state defeated the design it implements, silently, for as long as it existed.
This is why the state file on my machine still says one retrospective has happened. Two have.
The check that was missing, not red
The third one came out of the second retro itself, and it is the same sentence again in a different costume.
A pull request came up for merge with every check green. One required check, the one that runs the tests, was not green and was not red. It was not there. The fan-in job that reports it had never been created, so the context simply did not exist on that pull request. Every visible signal said ready. The pull request was structurally unmergeable.
Anything reading that pull request for failures would have found none, because there were none. The absence of a failing test run and the absence of a test run are the same picture. The same shape had already hidden never-run tests on two separate stories, in both cases while the developer agent reported them passing locally.
What these three have in common
A check that isn't running looks exactly like a check that's passing. I wrote that a month ago and thought I had learned it.
I had learned it about checks. I had not learned it about counters, and a counter is worse, because a check at least has a job that a human can go look at. A counter hands you a number, and a number carries no information about whether anything was measured to produce it. Zero findings and zero looks render identically. Nine and four is arithmetic over the wrong pair. absent is the honest report of an empty file and a completely dishonest report of the situation.
Every one of these three had a defensible reason to report what it reported. None of them was a bug in the sense of doing something other than what it was written to do. The rot check counted what it was told to count. The accumulator faithfully read the file at the path it was given. The merge check correctly found no failures.
The thing they share is that each of them collapsed two states that needed to stay apart. Nothing wrong and nothing looked. Zero and unmeasured. Not failing and not present. In all three cases the collapsed pair renders as the good one.
So the rule I am taking out of this, which is narrower and more useful than anything I would have written down in advance: any counter that can read zero has to be able to distinguish zero from unmeasured, and has to say which one it means. Not as a nicety. As the primary output.
What shipped
Two things, and only one of them came from a retrospective.
The reconciliation script exists. It parses the newest retro board, extracts every stub, searches for a card carrying that stub's identifier, and reports the ones with no card behind them. It runs as a precondition before the fleet starts a batch, and again before a retro is allowed to open, so a retrospective now cannot begin without first showing what its predecessor dropped. It reports rather than files, deliberately: I stay the valve on what becomes work.
The accumulator resolves its state against the git common directory, which every linked worktree of a clone reports as the same path. It also learned a legacy adoption path, so that finding an old worktree-local file and adopting it reports as a migration rather than as silence, and absent now means genuinely absent. Fifteen new tests, built against real git repositories with real linked worktrees, because the defect was entirely about what git answers from inside a worktree, and a stubbed test would only have asserted the belief that produced the bug. Against a mutant that reverts the path resolution and changes nothing else, nine of the fifteen fail.
That second one is not a retro finding. I found it separately, which is its own small comment on the state of the loop.
What has not
I had a paragraph here claiming two more of these had landed. Before publishing I went and grepped for them, because a post about proposals evaporating is a bad place to assert one landed on vibes.
The second retrospective proposed five stubs. Five days later, one is built. It is the one whose entire purpose is to report stubs that never got built.
The merge gate is still the prototype written during the run; there is no script in the repository. The mutation-evidence rule, which would require any assertion carrying an acceptance guarantee to be recorded as observed failing against a deliberately broken tree, is not in the rules file. Neither is the one-sentence rule about how a brief may describe an in-flight sibling. The milestone verb report is not in the fleet's preconditions.
So the honest count is one of five, and I only know that because I checked by hand, five days after the mechanism that is supposed to tell me had itself shipped, and before it has had a batch to run against. That is better than three and zero. It is not a solved problem, and I would rather print the number than round it.
Would I run it again
Yes, and the honest reason is uncomfortable. The retrospective's most valuable output in two runs was not any of its findings. It was the second run telling me the first run's findings had gone nowhere, which is a thing only a repeated ceremony can tell you, and which I would never have discovered by making the individual findings better.
The compounding claim from last week is still a claim. Two runs is not a trend. What batch ten teaches batch eleven is still something I have written down rather than something I have watched. But I know one more thing now than I did, which is that the loop's weakest link was never the quality of what it noticed. It was the step after noticing, the one with no gate on it and no counter watching it, and I had spent all of my design attention on the noticing.
What's next
The invariant check: a name for the class of assertion that pins a pairing rather than a behavior, and the observation that a prose warning about a hazard is what a careful person writes at the exact moment they had everything in hand to write the check instead.
Top comments (0)