I did not want another chatbot. I wanted something that could take a sentence and finish the job on my machine: open the app, click the button, run the command, read the result, tell me when it is done.
So for 30 days I handed my own workday to an agent running locally. It drives the screen, the shell, the browser, and an Android phone over ADB. Roughly 400 tasks. These are the four things that actually decided whether it worked, and none of them were the model.
1. State is the bottleneck, not intelligence
Every serious failure I hit was environmental: a stale browser session, a lock file from a crashed run, a port already in use, a half-typed shell command left by the previous task.
Giving the agent more context did not help. Making it re-read the world before acting did. A cheap check ("is the app I think is open actually open?") beats a smarter plan built on a wrong assumption.
2. Hands beat opinions
An agent that only talks is a suggestion engine. An agent that can click, type, and run commands can be verified. The moment it could act, my review moved from "does this plan sound right" to "does this diff do the right thing" - a much cheaper question.
3. Compounding beats cleverness
The single biggest improvement was not a better model. It was making every step idempotent and logged, so a task could be resumed instead of restarted. Ten boring, resumable steps outperform two brilliant ones that cannot be retried.
4. Keep the veto
Every irreversible action - sending, publishing, deleting, pushing - keeps a human approval gate. Not because the agent is bad, but because approval is what lets you give it more autonomy everywhere else.
The failure taxonomy, in the order it hurt
- Stale UI state - it clicked the right coordinates on the wrong screen.
- Environment leaks - processes and temp files surviving between tasks.
- Permission prompts - a dialog nobody dismissed, blocking a queue of work.
- Partial writes - a file half-saved when a timeout hit.
Note what is not on this list: reasoning errors. The model was rarely the problem.
What I run now
A local-first agent runtime on my own machine, with a persistent workspace and a diff of every side effect before it lands. If that approach to local control is interesting to you: https://agent-home.ai (release notes: https://github.com/gal-agent/gal-release).
If you are building something similar, I would like to hear which of the four lessons bit you first.
Top comments (0)