One executor, one pass, one commit - and the work can be dropped at any boundary.
👋 I'm Anton - a software engineer working mostly in PHP/Symfony and Go, currently carving a live PHP monolith into Go services. Earlier parts of this series were about what an executor must know, how small a unit of work has to get, and how to write a task with nothing left to interpret. This part is about the thing I only learned by having to stop in the middle: the unit at which work becomes droppable. Notes: github.com/brilliant-almazov.
Maybe you already do this better, maybe you organise it differently - either way I'd rather hear how than assume mine is the shape. As always: these are my habits on one codebase, not advice for yours.
The evening I had to stop mid-stage
A stage of 13 iterations. The first wave had closed, the second wave was running, and I had to
put the work down right then - not "wind it up", not "finish the current thing", stop.
There was exactly one question worth asking, and it wasn't "how far did I get". It was:
What is sitting uncommitted, and what would I have to reconstruct from memory?
The second half of that question is the trap, because of how the executor works. It carries nothing
between tasks. No dialogue history, no memory of the previous iteration, no accumulated sense of
"where we were". Every task arrives complete or it doesn't arrive at all. So "I'll pick it up
tomorrow and it'll remember roughly where we were" is not available to me at any price. Tomorrow's
executor starts from zero, from the text I wrote.
Which means the only thing that survives an interruption is what's written down and committed.
Everything else is in my head, and my head is the least reliable component in the pipeline.
I want to be precise about the framing, because this is the part I got wrong for a while. Being
interrupted is not an emergency to be handled. It is the normal case. A day ends, a production
incident lands, a call runs long, priorities move. If the way I cut work only survives
uninterrupted evenings, the cut is wrong. Droppability is a design requirement on the split, at
the same level as "the executor must not have to look anything up".
The unit: a batch
An iteration is one concern - one or two files and their test. That's the unit of work. It is not
the unit of stopping, because a single iteration is usually too small to be worth a commit on its
own and too entangled with the next two to be committed alone.
The unit of stopping is a batch:
- a batch is one executor, one pass;
- iterations inside a batch run strictly in order - 02, then 03, then 04, then 05;
- batches inside a wave are independent by files, so they run at the same time;
- the commit happens on the batch boundary.
That last line is the whole article. The commit is not per iteration ("too noisy, and half of them
don't stand alone"), not per stage ("a day of work in the air"), but per batch. And the consequence
I actually care about:
No uncommitted batches are left behind. Work is cut off on a batch boundary and nothing is
lost.
If I stop between waves, the state is: everything closed is committed, nothing is half-written, and
the next thing to do is a named batch whose text already exists. Reconstruction from memory:
none.
iteration one concern 1-2 files + their test unit of work
batch one executor, one pass ordered iterations unit of commit / stopping
wave parallel batches independent by files unit of scheduling
stage all waves the whole task unit of full acceptance
Stage one: 13 iterations, five batches, three waves
Here is the real breakdown of the stage I was in the middle of - the one where a service stops
hand-rolling runtime it can take from the shared platform library.
| Batch | Iterations | Wave |
|---|---|---|
| A | 02 → 03 → 04 → 05 | 1 |
| B | 06 → 07 → 08 | 1 |
| C | 09 → 11 → 12 | 2 |
| D | 10 → 13 | 2 |
| E | 01 | 3 |
Read it top to bottom and the schedule falls out:
- Wave 1 - A and B start together. Two executors, two batches, no shared files.
- Wave 2 - starts only after both batches of wave 1 have closed. C and D then run together.
-
Wave 3 - starts after wave 2, and it is a single batch containing a single iteration:
01.
Two things about that layout usually get a raised eyebrow, so let me take them in turn.
Why is iteration 01 executed last? Because numbering is the order the iterations were
written, not the order they run. 01 registers the finished piece in the daemon - it depends
on the results of 05 and 07, which live at the end of batches A and B respectively. So it
cannot start until both of those batches have closed, and it gets a wave of its own. Renumbering it
to 13 would have been cosmetic; the dependency is the real thing, and the dependency is written
down.
Why do the waves have hard edges? Because the wave boundary is where a batch boundary of every
running batch coincides. That is the cheapest possible place to stop: everything before it is
committed, nothing after it has started. Wave edges are the checkpoints I actually rely on.
And the answer to my evening's question, in this specific case: whichever batch of wave 2 was
mid-pass would be dropped and re-run from its first iteration; everything else was already on disk
and committed. The cost of stopping is bounded by one batch, and I know that number before I
start.
Stage two: 12 iterations, two executors, eight batches
The second example is a different stage - a generator that produces the skeleton of a new service -
and it's the more instructive one, because it has a hard constraint: at most two executors on the
main work at any time.
| Batch | Iterations | Executor | Starts after |
|---|---|---|---|
| B1 | 01 | 1 | — |
| B2 | 02 | 2 | — |
| B3 | 03 | 1 | B1, B2 |
| B4 | 04, 05 | 1 | B3 |
| B5 | 06, 07, 08 | 2 | B3 |
| B6 | 09, 10 | 1 | B4, B5 |
| B7 | 11 | 2 | B3 |
| B8 | 12 | 1 | B6, B7 |
Look at the executor column: it only ever holds 1 or 2. That single constraint changes the
character of the plan completely.
With unlimited executors you'd fan out - one per iteration, twelve at once, and let dependencies
sort themselves out at merge time. With two, the schedule is built by dependencies, not by the
wish to parallelise. Every row above answers a question I had to answer before any of it ran:
- what must exist before this can start (
starts after); - which of the two executors is free at that point (
executor); - which iterations are close enough in shape to ride in one pass (
iterations).
Notice B4 and B5 have very different sizes - two iterations against three - and that's fine. A
batch is sized to be one coherent pass for one executor, not to be equal to its neighbour. B7 is
a single iteration on executor 2 precisely because executor 2 would otherwise be idle after B5, and
B7's only dependency is B3.
The uncomfortable part: this table is handwritten, and it's written before anything runs. There is
no scheduler doing it for me. That is a real cost and I come back to it below.
Status lives in the folder, not in the filename
Cutting work into batches only helps if I can see, without reading anything, what state the work is
in. So the layout of the prompt files carries the status. Four places, and the boundaries are hard:
prompts/
├── done/ closed stages - not edited again, ever
├── in-progress/ exactly what is being worked on right now
├── todo/ written stages, not taken up yet
└── <rules> rules that apply to every stage
The rules for it are short:
-
done/- closed stages. Not edited. If something in a closed stage turns out to be wrong, that's a new stage, not a rewrite of history. -
in-progress/- exactly what is being worked on right now. Not "roughly current", not "the three things I intend to get to". -
todo/- fully written stages that haven't been taken up. Written, not sketched: a stage lands here only when its text is executable as-is. - the root - the rules that apply to every stage, so no stage has to restate them.
And one prohibition that does most of the work:
A progress marker inside the filename is forbidden. Moving the file between folders is the
only way to change status.
No wip, no todo glued into a name. If the status is in the name, the name has to be edited to be
true - and the moment work stops unexpectedly is exactly the moment nobody edits anything.
Status of a single iteration: in the name, because the folder is shared
There is one exception, and it exists for a structural reason. All the iterations of one stage live
in one folder - the folder already carries the stage's status, so it can't also carry each
iteration's. So an iteration's status goes into its own name:
| Filename | Meaning |
|---|---|
NN-<name>.md |
not closed |
NN-done-<name>.md |
closed - not edited again |
NN-hold-<name>.md |
frozen |
In place, mid-stage, it reads like this:
in-progress/<direction-of-work>/
├── 01-done-<name>.md
├── 02-done-<name>.md
├── 03-<name>.md ← next up
└── 04-hold-<name>.md ← frozen, deliberately
Two more conventions live here, both of which I got to by getting them wrong first:
The launching document is named after the direction of work, not after a ticket number. There
are several directions, and they outlive individual tasks. A folder named after a ticket becomes
meaningless the week the ticket closes; a folder named after the direction is still the right place
to put the next stage a month later.
Replaced splits are not kept. When a stage is re-cut, there is exactly one live version of the
text - the previous one lives in version-control history and nowhere else. Two versions of a split
side by side means someone eventually executes the stale one, and "which of these is current" is
not a question I want to answer under time pressure.
What that adds up to, in files
The state of all of this at the point I took these numbers:
| Count | What |
|---|---|
| 298 | prompt files in total |
| 20+ | closed stages |
| 3 | stages in progress |
| 5 | stages written but not taken up |
| 41 | files in the largest single stage |
That's the honest scale of the overhead: nearly three hundred files of specification, all of it
written by hand, all of it before the code it describes. The three in progress are the only ones I
have to hold any live context about. The 20+ closed ones I never touch again. The 5 written-and-
waiting are the reason a free afternoon doesn't start with planning.
Why position, and not a note in the text
I want to name the single reason, because I tried the other way first and it failed in a specific
and repeatable manner.
State expressed by where a file sits is visible without reading, and it cannot disagree with the
file's content - there is nothing to disagree with. State expressed by a note inside the text
diverges from reality at exactly the worst moment: when work stopped unexpectedly. Nobody stops
mid-pass to go and edit a status line, so the line keeps saying "in progress" for three weeks, and
then it isn't a status any more, it's decoration.
The move-the-file rule survives interruption because it is a single atomic act that happens at the
boundary, together with the commit, when I'm already doing bookkeeping rather than thinking.
What it costs
Two prices, both real, neither of them hidden.
A batch is bigger than an iteration, so the commit boundary is rarer than I'd like. If work is
cut off in the middle of a batch, the cost is one batch of work - not one iteration. On a batch of
four iterations that's a genuinely annoying amount of re-running. I've decided I'd rather have a
predictable "one batch" than an unpredictable "somewhere between one iteration and a whole stage",
but it is a trade, not a free win.
The wave schedule is built by hand, in advance. The dependency table for a 13-iteration stage
and the eight-row schedule for a 12-iteration stage are both handwritten. That's preparation, not a
side effect of doing the work: it happens before anything runs, and it is part of the cost of the
stage. If I skip it, the batches stop being independent by files and the whole droppability
property quietly evaporates.
There's a third, softer cost worth naming: the discipline of not doing "just this one small thing"
myself while a stage is open. Every time I do, part of the state moves into my head, and the stage
stops being resumable by anyone - including tomorrow's me.
The part that's about working with an executor at all
Stopping without losing anything is a property of how the work is cut, not a property of any
tool. A batch boundary that carries a commit would be worth having on a team of humans, on a solo
afternoon, on paper. Nothing about it is new.
What changes when the work is executed by something with no memory between tasks is that this stops
being good hygiene and becomes mandatory. There is no "it'll remember where we were" to fall
back on, so the only continuity available is the continuity I wrote down. The executor didn't
introduce the requirement. It removed my ability to ignore it.
The one conclusion
Commit on the batch boundary, and make the boundary somewhere you're willing to be interrupted.
Then the cost of stopping is a number you knew before you started.
This is what I do now, with the reasons and the price attached - not a claim that it's the shape to
use. If you do this better, if you've been through the same interruption and landed somewhere else,
or if you look at batching and waves and think it's overhead solving a problem you don't have - I'd
genuinely like to hear it. How is this handled where you work, and what broke?
Working with agents - Part 6.
Next: what to do with the mistake that keeps coming back - the ladder from a reminder, to a rule,
to a check that makes the mistake impossible to commit.
Top comments (0)