What eight stages of automated production taught me about the limits of agents
Our production line has eight stages and moves an idea to working software:
research, architecture, task breakdown, build, review, human approval. Multiple
engines run in lanes underneath it, chosen per task.
The rule that keeps it alive is one sentence. If a task has no command that
verifies its output, that task does not enter the pipeline.
Why the rule exists
An agent can produce something for almost any instruction. The output arrives and it looks plausible. Plausible is where the trouble starts. Without a verification
command there is no difference between finished and abandoned, so the pipeline
advances on the appearance of work.
The check does not have to be clever. A test suite, a build that must compile, a
parser that must not throw, a count that must match a known figure. It has to be
mechanical and it has to be able to fail.
Tasks that cannot be checked mechanically still exist in our work. Judgement about
tone, a design direction, whether a client will accept something. Those do not go
into the pipeline. They go to a person, and pretending otherwise just moves the
failure later.
Engines are not interchangeable
We run several engines and route work by lane. I assumed the routing could be
naive, and it could not.
One engine cannot run our verification commands at all in its sandbox, so every
task with a gate attached silently fell back to another lane and consumed a turn
doing nothing useful. Another engine cannot read from disk, which means it cannot
see the existing code, which means giving it a modification task produces a
confident rewrite of something it never saw.
Both of those look like model quality problems in the output. Neither is. They
are routing problems, and they are only visible if you measure which lane did the
work and whether that lane could physically do it.
The limit I keep running into
The pipeline is good at anything with a mechanical definition of done. It is
useless at deciding what is worth doing, and it degrades quietly when the
definition of done is soft.
That is the honest boundary of an agentic setup today. Not the model's
reasoning, and not tool access, which is now routine. The boundary is how much of
your work can be stated as a condition a machine can check. In our case that turned
out to be most of the production and almost none of the direction.
Top comments (0)