Four Things I Don't Let the Agent Decide

The project my harness was building is a report generator. It takes a land parcel and produces a document: income figures, dwelling counts, market series, an elevation profile, a walkable-distance section, a list of legal constraints.

Think about what a bug looks like in that. Not a stack trace. A number.

A report that says a municipality's median income is €18,400 when it's €18,004 renders perfectly. It lints clean. Every test passes, because the tests check that a number came out and this is a number. A chart that draws an elevation transect backwards is still a chart, and the figures printed beside it are still correct.

There's an entire module in that codebase whose only job is to stop a missing value from becoming a zero — a Known/Unknown type where Unknown deliberately has no value field at all, so the mistake can't be made. A report that prints 0 when it means we don't know is not a crash. It's a confident answer that's wrong.

That is the characteristic failure of this project: correct-looking output from wrong code.

Which means a green test suite is weak evidence. So the harness doesn't delegate four things.

1. It runs the gates itself

Every implementer returns a verification block reporting that its tests pass.

That block is recorded and never believed.

ruff check, pytest, and for the frontend npm run lint and npm run build are executed by the harness, inside the worktree, and its parse of the output decides whether the work is even eligible for review. The agent's account of its own work is stored as testimony, not as evidence.

This sounds like paranoia until you notice it costs nothing. The commands were going to run anyway. The only question is who reads the exit code.

2. It plants the tests

Files under tasks/supplied/<ID>/ are copied into the worktree and committed before the model starts, then declared off-limits.

The point is what git diff against that commit proves. "Here's the test, make it pass" is the oldest instruction in TDD and it's completely unenforceable against something that can edit the test. Committing first turns an instruction into a check.

There's a whole failure mode buried in that, and it's the subject of the next article.

3. It overrides the reviewer

A second model reviews each diff against the packet's declared invariants. It returns accept or revise.

An accept returned over a red gate, a violated critical invariant, or a blocker finding gets downgraded automatically — and the reason is written to a column.

That column is the part I'd defend hardest. A reviewer that's too agreeable is a real and boring failure mode, and the difference between catching it and not is whether over-acceptance accumulates somewhere you can query, or somewhere you can only have a feeling about.

4. It refuses a review that skipped a question

Every packet declares its invariants by id. The reviewer must return a verdict on each one — held, violated, or unverifiable — with a file:line behind it. A review that omits an invariant is rejected by the contract.

Silence is not a pass.

And unverifiable is a real answer that blocks a critical invariant, rather than a shrug that lets it through. The moment unverifiable becomes safe, it becomes the answer to everything hard.

The question underneath all four

Strip the mechanisms away and one question is doing the work:

Does this failure show up as red?

If yes, the gate handles it and you need nothing else. If no, the gate is decoration and you need something that isn't the gate.

That second case splits again. Some invariants fail green but are still testable — you can write an assertion that catches them, it just wasn't going to exist unless someone wrote it deliberately. Five packets in the queue got tests planted for exactly that reason: Unknown has no value; a key survives serialization; a digest is order-independent; a decay function hits both of its stated anchors; a section-walk reaches every table and constraint row. Small, dull, and each one pinning a mistake that would otherwise render beautifully.

And then there's the residue: invariants that fail green and that no assertion I could write would catch.

Five of thirty-seven

Those get gate = "human", which holds the merge until I approve it personally.

Five packets out of thirty-seven carried it. The rule was narrow: the failure mode is silent and the blast radius is the whole stage.

• A database migration. Silent because it applies cleanly; whole-stage because undoing it on live data is a different kind of afternoon.
• A cache layer that never actually hits. Every assertion still passes. Only latency changes — and latency isn't in the test suite.
• An importer that deletes before it flips, so a crash mid-run caches a confident no amenities found for a place full of amenities.
• The elevation transect drawn backwards, with correct numbers printed next to it.
• A data-boundary change that degrades a map overlay without producing a single type error.

Read those again and notice they have nothing in common technically. Migration, cache, importer, chart, type boundary. What they share is the shape of their failure: nothing turns red, and the damage isn't local.

That's the criterion. Not "this task is important." Not "this task is hard." Not tier, not line count. How does this one fail, and who notices?

The concession

gate = "human" is not a feature. It's the concession in the design — the point where an automated green gate cannot certify an invariant that fails green, and no amount of further automation closes the gap.

Five packets out of thirty-seven is a 13% manual rate. It would be easy enough to leave those out of a write-up and describe a fully autonomous pipeline; the pipeline that actually claims autonomy over those five is the one that merges a backwards chart with correct numbers beside it.

Declaring the gap doesn't close it either. It decides who finds it — an operator at merge time, or a user at read time.

What this actually is

Every one of the four is the same move: delegate the work, keep the verdict.

The model writes the code, runs its own checks, forms its own opinion, and reports all of it. None of that is the decision. The decision is made by something that re-derives it from evidence — the exit code, the diff against the planted commit, the invariant list, and for five packets out of thirty-seven, me.

Which reframes autonomy, usefully I think. It isn't a property of the agent, and it isn't a dial you set once for the whole system. It's a property of the task — granted per packet, in writing, with a reason you can point at afterwards.

I got to write gate = "auto" thirty-two times. That number means something precisely because five times I didn't.