"Done." Two words that mean nothing on their own.
Every time an agent reports completion, I run the same short ritual. It takes about a minute and it has caught more bugs than any review process I have used.
1. Ask for the artefact, not the summary
What file changed, what row was written, what message was sent. If the answer is prose, I ask again. The agent summarizing its own work is not evidence - it is the same model that just did the work, describing what it believes it did.
2. Run the check that was written before the task
This is why the fixture matters. The check exists before the agent starts, so "done" is a fact, not an opinion. If there is no check, the task was not ready to be delegated, and that is my failure, not the agent's.
3. Look at the diff, including the parts I did not ask for
Agents do extra things. A formatting pass, an unrelated import, a "small cleanup". Most of the time it is harmless. Twice it was not: once a config value was quietly normalised, once a comment explaining an ugly workaround was deleted, and the workaround stopped being obvious to the next person.
4. Check the side effects on the outside world
Ports still open, processes still running, temp files left, a service left in a weird state, an email sent to a person instead of a draft. Cleaning up after a task is part of the task, and agents are optimistic about cleanup.
5. Confirm the thing it did not do
The instruction had five parts. Four are visibly done. The fifth is the one to look for, because agents do not announce omissions. "Also, did you update the changelog?" has caught something maybe a third of the time.
The part I got wrong for weeks
I used to read the summary first and check the artefact only if the summary sounded suspicious. That is backwards. The summary is the least reliable output of the whole run, and I was using it as my filter.
Now I look at the artefact first and read the summary last. If the summary and the artefact disagree, the artefact wins, and I stop trusting the summary for that class of task.
The ritual is boring, which is the point. Done is a claim, not a state, until something outside the agent confirms it.
What is your version of this check? Mine is 5 steps and I still miss the omissions.
Top comments (1)
Step 5 is the one that can be made mechanical, which might fix the misses. If the instruction has five parts, have the agent's first output be a numbered list of those five, and the done-check is one artefact per number (a diff hunk, a row, a message id). An omission then shows up as an empty slot instead of something you have to remember to look for. The changelog question becomes a line item because "changelog" was in the request.
On step 2, one addition: run the pre-written check against the untouched starting state first. If it already passes before the agent does anything, it can't tell done from not done.
My guess for the misses you still have: they cluster in work that was never in the instruction at all (the side effects in step 4), and no checklist derived from the request catches those. Do your five misses split that way?