I do a short review every Friday. Twenty minutes, sometimes less. It is the only maintenance ritual I have kept for more than two months, and it is the reason the setup has not quietly rotted.
What I look at
Five things, in this order:
- The refusals. Every task the agent declined or escalated. Not to fix them, just to check whether the reason still holds. Half of them were right last week and wrong now.
- Tasks that took more than twice my estimate. Either the estimate was wrong or the task is not automatable yet. Both are worth knowing.
- The retries. Any task run more than once this week. The reason for the retry is the bug, not the retry.
- One log, read end to end. Not skimmed. One, chosen at random. Skimming ten tells you less than reading one.
- Things I approved without reading. If the list is long, my approval gate has become a rubber stamp, and the gate is now decoration.
What I do with the output
Usually two or three lines of change. A pre-flight check gets added, a task file gets a better finish condition, a note gets written to the notes file so the same thing does not happen next month.
Most weeks there is nothing dramatic. That is the point - the review is not for finding incidents, it is for catching drift before it becomes one.
Why twenty minutes and not an hour
An hour-long review becomes a project, and projects get postponed. Twenty minutes fits between other things on a Friday, so it actually happens. The value is in the repetition, not the depth.
The failure mode of an agent setup is not a dramatic break. It is a slow accumulation of exceptions that everyone remembers exists but nobody wrote down, until one day the automation is a fragile museum piece that only its author can operate.
I have run that setup before, in other forms. The review is cheap insurance against building it again.
If you do something similar, I would like to know what you look at that I do not. Mine has stayed the same for two months, which probably means it is missing something.
Top comments (0)