Every agent demo I have seen ends the same way: a task, a burst of activity, a confident summary. Nobody ever shows the part where it should have stopped three steps ago.
Stopping is the actual product.
Three ways to stop, only one of which I trust
Step budget. Stop after N actions. Cheap, predictable, and completely blind to whether progress was being made. My step budgets are now emergency brakes, not the main mechanism.
Time budget. Same problem plus a clock dependency. Useful only as a hard ceiling.
Goal verification. Ask: is the thing I was asked to produce actually present? A file that parses. A test that passes. A message that appears in the outbox. This is the only one that tracks reality, and it is also the only one that is hard to write.
Why verification is hard
Because it requires you to say, in advance, what finished looks like. That sentence is the whole design. If I cannot write it, I do not actually know what I am asking for, and neither does the agent.
I keep a rule from this: no task enters my queue without a finish condition I can check mechanically. Not "the report is good". "The file exists, has more than 200 characters, and its last line is a date". Yes it is crude. Crude and checkable beats elegant and unverifiable.
The failure mode that taught me this
I had an agent tidying a notes directory. It ran for eleven minutes, reported success, and had moved everything into a folder called misc - including the source files for a post I was halfway through. The task had no finish condition, so any state counted as done.
The fix was not a better model. It was one line: the task may not end while any file it touched has a modification time newer than the start of the run and no recorded destination.
The uncomfortable conclusion
Most of the work of a reliable agent is not in the agent. It is in the sentences you write about what counts as finished, and in the small piece of code that checks them.
That work is boring, it does not demo well, and it is the difference between a toy and a tool you leave running overnight.
Top comments (0)