AI tools tell you what to do. I wanted one that actually does it — opens the browser, edits the files, runs the terminal command — while I watch.
That became Kyboro: you describe a task in plain English (typos and Hinglish welcome), and it runs it on your own machine.
The architecture, in one breath
- Electron desktop app — the window you see
- Python local executor — the hands: browser, files, shell, screen
- Cloud orchestrator (FastAPI + Postgres) — the brain: plans the task and picks the model
- A WebSocket links the hands to the brain
Simple on paper. Here's what broke.
1. Timeouts have to nest
The shell had a timeout, the WebSocket had one, the cloud task had one. When an outer timeout was shorter than an inner one, the cloud gave up while the command was still running — the task just vanished with no error.
Fix: a strict ladder — shell < socket < cloud. Every inner layer fails first and reports why.
2. Loop detection must ignore the payload
To catch a stuck agent I hashed each tool call with all its arguments. But a stuck agent retries with tiny variations — a different timestamp, one changed word — so every attempt looked brand new.
Fix: hash the shape of the action, not its content.
3. "Ask before risky stuff" is a distributed systems problem
Deleting files, sending messages, paying: the agent must stop and ask. But the decision is made in the cloud and the user is on the desktop. If the confirmation can't make the round trip, the agent either hangs forever or acts without asking. Both are unacceptable.
Fix: a dedicated confirm bridge, with a clear "no answer = no action" rule.
4. Synthetic clicks don't work on the real web
element.click() from injected JS gets ignored by a lot of modern sites, and half the buttons live inside shadow DOM where normal selectors can't reach.
Fix: trusted input through the Chrome DevTools Protocol, plus selectors that pierce shadow roots.
5. Shipping fixes without a reinstall
Asking users to reinstall for every UI fix kills retention. So UI files update by content hash — the app compares hashes on launch and pulls only what changed. The catch: anything packaged inside the Electron bundle still needs a full release, so you have to know which side of that line every file sits on.
Where it is now
Windows + Mac, plus a Chrome extension. It's early, and I'd love brutal feedback:
👉 Try one real task you'd normally do by hand, and tell me where it breaks.
Top comments (0)