The executor writes most of the code; the check has to come from outside it.
👋 Hi, I'm Anton — a software engineer working mostly in PHP/Symfony and Go, currently carving a live PHP monolith into Go services. This series is about the order the work happens in: a wish becomes requirements, requirements become a contract, the contract becomes tasks and specs, and the specs get run. This part is about the thing that does not travel down that road with the work — who is answerable for the result. Running notes live on my GitHub: github.com/brilliant-almazov.
This is how I do it right now, with the price attached — maybe you already do it better, maybe you see it differently.
Primer, in one paragraph
The cast is small. A spec is a self-contained brief for one unit of work: it names the files, states the change and states what makes it accepted. An executor performs one spec end to end; here that is usually an automated one, and nothing below changes if it is a person. A lead prepares the facts, hands the specs out and accepts the results — that is me. The rules an executor is expected to follow live in an instruction file it reads before it starts. That file is the setting for everything that follows.
The rule everybody had
There was a rule, and it was not a vague one. In a test, the context comes from the test runtime; the empty placeholder context is forbidden. It was written down, in the instruction file, in the place rules are kept, and every executor read it before writing a line.
Tests kept arriving with the empty one.
Not once, and not from one executor. Across weeks, in whatever direction the work went, in files written by different runs on different days. The rule existed. It had been read. Nothing enforced it, so it did not hold.
It started holding on the day writing such a file began to fail.
That is the whole article in three lines. The rest of it is the price:
reminder a sentence in the conversation lives for one session
rule a line in the instruction file works while it is read
check a run that fails works whether or not anyone remembers
What the rule was worth unenforced: 421 calls in 184 files
Eventually I counted instead of assuming. In one service tree, with the rule in force and visible: 421 calls to the empty context, across 184 test files.
The number is not the interesting part. What those tests were worth is.
A test built on the empty context has two properties, and both of them are quiet:
- It does not cancel along with itself. The test's own lifetime is not attached to anything the test runtime controls, so when the thing under test hangs, the test hangs with it until the package timeout — instead of failing at the moment the deadline passes.
- It checks nothing at all about context propagation. If the production code drops the context it was handed and quietly substitutes its own, the suite stays green. The exact defect the rule existed to prevent is the one defect that class of test cannot see.
So the suite was green, and green meant considerably less than everyone reading it assumed.
What the rule became. The form was made explicit rather than implied: the context in a test comes from the test runtime; a port stub does not discard the context it is handed but keeps it in a field and hands it back to the test; and every transport, repository, manager and consumer carries a cancellation test — cancel the context, assert the failure, assert nothing was written. Then the part that actually changed the outcome: writing a test file that uses the empty context fails. Not a warning in a linter report somebody reads on Friday. The write does not complete.
The repair took a set of small mechanical iterations, one file at a time, and none of it was interesting. That is the shape of this kind of debt: the discovery is one command, the fix is a queue of boring edits, and the only thing that stops the queue from re-forming next month is the thing that fires on the write.
Two more findings from the same period, same class, one line each:
- A cache-invalidation consumer was written and registered in no daemon at all. The package compiles. Its unit tests are green. In production, invalidation does not run — not slowly, not intermittently. It does not run.
- A decode failure is returned as an ordinary error. The driver reads a returned error as "hand it back", and the subscription declares no redelivery limit, no backoff and no dead-letter queue. One undecodable message retries forever.
Three findings, one shape. Every one of them is caught by a check that runs outside the executor's own account of what it did. Not one of them is caught by a green build — because in all three cases the build was green.
How this is normally handled
Nothing below is a criticism. It is the baseline, and each line of it is reasonable where it came from.
| Practice | The common shape | Where it stops |
|---|---|---|
| Review | Reading the change is the main way a machine-written error is caught | A reviewer's attention is finite and uneven, and it degrades exactly when volume rises |
| Instruction file | Conventions written into a file the executor reads — the AGENTS.md convention and its equivalents |
It works while it is read, and reading it is not verifiable from outside |
| Green tests | The suite passed, so the change is fine | A test can be green because it asserts nothing — see coverage as a signal |
All three are true, and none of them is enforcement. Each leaves a gap that is filled by somebody remembering — a reviewer remembering to look for the thing, an executor remembering the line it read, a reader remembering that green is a weaker signal than it looks. Remembering is exactly the resource that does not survive volume.
The share of the code is high; the accountability does not divide
Most of the code in this work is produced by an executor. I am not going to attach a percentage to that — I do not measure it, and putting an invented number next to a real one is how the real one stops being believed.
What I will state flatly is the part that does not scale with the share:
Setup, checking and acceptance stay with a person. Whatever fraction of the code the executor writes, none of the answerability moves with it.
This is not an ethical position and I would rather not dress it up as one. It is about who answers to whoever is paying. Nobody on the other side of that conversation has any interest in which part of the output was typed by whom. There is one name attached to the result, and it is a human one — so the checking has to sit in the same place the answering does.
The reason is technical, not ideological
The practical reason is narrower and more useful than the moral one: an executor fails in a predictable, small set of ways.
Not randomly, and not creatively. It reproduces a form it saw nearby. It satisfies the letter of an acceptance criterion that was written loosely. It writes the code and does not wire it up. It answers "done" because the last command it ran exited zero.
A predictable failure has exactly one reliable answer, and trust is not it. When a mistake recurs, the response is not a firmer reminder — it is a check that makes the mistake impossible to commit. Everything else in this article follows from that one sentence.
The ladder: reminder → rule → check
I named this ladder in passing in the previous part. Here is the version with each rung honestly priced.
Reminder. A sentence in a conversation: "don't do that". It works for the duration of that conversation and is gone at the end of it. The next executor never saw it. That is not a weakness of the executor — it is what a reminder is.
Rule. A line in the instruction file. Better: it survives the session, it is quotable, it is the same for everyone. And it is conditional on being read, and on being read the same way twice. The 421 calls are what that condition costs once the file is long and the rule is one line in the middle of it.
Check. A linter rule, a forbidding test, a structure test, a blocking hook on the tool that writes the file. It does not care whether anybody remembers it exists. It fires at the moment of the mistake, on the mistake, with the fix in the message.
The rungs are not alternatives. A rule that keeps being broken is a check that has not been written yet.
Five rules that became checks
Concrete, from the current setup. Each row was a rule first, was broken, and became a check afterwards.
| The rule it used to be | The check it is now |
|---|---|
| read rows through the one shared core, not by hand | a forbidding test: for rows.Next() appears in exactly one package |
| keep the environment catalog and the metrics snapshot true | a drift check in the build: regenerate, compare, fail on a difference |
| do not let test coverage slide | a ratchet on the threshold — it can rise, it cannot be lowered |
| do not write a spec containing a phrase that makes it unrunnable | the file write is blocked when that phrasing appears in the text |
| take the context in a test from the test runtime | the file write is blocked when the empty context appears in a test |
Read the right-hand column as a group. None of those five is clever. Every one of them is mechanical, cheap to run, and — this is the point — indifferent to who is at the keyboard and how the day is going. That indifference is the entire property being bought.
A gate leans on an artefact, not on a claim
From which one requirement on the process follows, short enough to fit on a line:
Every gate leans on an artefact — a number, a run, a file — and never on the executor's statement that the thing is done.
Drawn as the two things that arrive at a transition, it is not a subtle design:
"the iteration is finished" ──▶ ┃ closed a claim is not an input
┃
number · test run · file ──────────────▶ the artefact is the input
"It's done" is a claim. It carries no information about whether it is true, and it costs nothing to produce — including when I am the one producing it. A number that was measured, a command that was run with its output read, a file that exists at a named path: those can be wrong too, but they can be wrong checkably, which is a different category of thing.
This is not scepticism about the executor. It is the same rule turned on me: when I say a spec is ready, that is a claim; when the spec contains no open question, that is a fact somebody else can verify without asking me.
What this costs
The bill is real, and it arrives in four parts.
- Every check is code. It has to be written, and then maintained indefinitely, against a tree that keeps moving. The five above were not free and they are not finished.
-
A check catches form, never intent.
for rows.Next()in exactly one package is a fact about shape. It says nothing about whether the query behind it is the right query. No amount of this saves a wrong architecture; it only stops a known-wrong form from spreading. - A false positive is expensive twice. It costs the executor's run, and it costs the credibility of the check — one that cries wolf gets routed around, and then it is worse than nothing.
-
One of these is honestly unfinished. The coverage ratchet exists and works in the direction it was built for: the threshold can only ever rise. But the threshold is currently set to
0, while actual coverage sits at 86.7 %. The mechanism is in place; the bar has not been raised to meet it. That is a job not yet done, and I would rather list it here than let the word "ratchet" imply a discipline that is not being applied yet.
When not to do this
- A one-off script nobody will maintain. The check outlives the thing it guards, and then it is pure cost.
- A rule that has never actually been broken. Start with the reminder. A check written for a hypothetical failure buys nothing and adds a thing to maintain. Let the rule fail first, then automate it.
- An area with no checkable signal. If the property you care about has no mechanical fingerprint, an invented one produces false positives — and false positives are how a check loses its authority.
The honest caveat
All of this is kept on my own tasks. There is no multi-person sample behind any of it. The 421 and the 184 are counts from one tree on one day, not a study; the five checks are the ones I happened to need, in the order I happened to need them.
I would be misrepresenting the material if I presented it as an industry finding rather than as one practitioner's log, and I have no way of knowing which parts of it survive contact with a team of eight.
The multiplier
The uncomfortable version of the throughline: speed of production amplifies whoever set the definitions, and it amplifies a bad definition exactly as efficiently as a good one. An executor that writes a correct file in four minutes writes an incorrectly specified one in four minutes too — and then it writes a hundred of them.
Which is why the ladder ends where it does. The faster the production side gets, the less the middle rung is worth on its own, and the more of the outcome rests on checks that fire without anyone being present. That leaves the definitions — what counts as done, what counts as an artefact, which mistake is worth a check — sitting exactly where they were before any of this started: with a person, who cannot delegate them, and who is the one asked about the result.
From requirement to contract — Part 5. Next: the checkable condition comes before the thing that satisfies it — writing the test, the criterion or the guard first, and what changes when the order runs the other way.
If you do this better, tell me which of your rules you managed to turn into a check and which ones resisted. If you have been through this, what was your version of the 421 — the rule everyone had read and nobody followed? If you see it differently, say where a blocking check cost you more than the mistake it prevented. How is it solved on your side, and what broke there?
Top comments (0)