Most "personal AI agents" today still get stuck in the same two dead ends.
Dead end #1: API-only connectors. They work great when a service offers a clean integration. The problem is that 90% of annoying everyday errands—canceling a buried subscription, starting a return, booking a table, ordering dinner—live inside consumer apps that will never expose an AI API for that flow.
Dead end #2: Cloud browser bots. Spinning up a remote server browser sounds clean until you try to touch real accounts. You immediately run into login friction, anti-bot walls, and the bigger reality: half the services people actually use live inside mobile-only Android apps, not desktop web forms.
And early on-device phone agents had a different usability flaw: they hijack your foreground screen. Watching an agent slowly tap through your phone for two minutes while you can't even reply to a text doesn't feel like delegation—it feels like babysitting.
What we built instead
We're building Mateo, a personal AI agent on Android that runs real apps on your own phone inside an isolated background VirtualDisplay—so your phone stays free while the task finishes quietly in the corner.
Here is a 4-frame capture from an uncut Android device run (foreground Chrome stays completely usable while Mateo handles a restaurant lookup in the top-right corner):
Three engineering trade-offs we had to be honest about
If you've ever built Android automation, you know the edge cases matter more than the demo. Here is where our architecture draws clear lines:
- On-device sessions, not cloud cookie shipping. Because Mateo executes directly inside the apps you're already signed into on your own phone, we don't export your cookies into a remote server.
- Single-instance per app package. Mateo can run tasks across distinct apps, but we don't try to force simultaneous multi-instance execution of the exact same Android app package.
-
Respecting
FLAG_SECUREand system boundaries. Android intentionally blocks virtual screen capture on DRM and sensitive protected surfaces (FLAG_SECURE). We don't try to bypass Android's security model—Mateo focuses on everyday multi-step tasks that can complete cleanly within system rules.
Try it / break it
You can grab the Android app via Google Play from mateohq.com/phone-use (once installed, the app walks you into enabling the background phone-use Beta build; Web/iMessage entry is also available there for non-Android flows).
Curious where other builders here land on two things:
- If an agent runs quietly on your phone without taking over your screen, what is the first repetitive multi-step errand you'd actually hand off to it?
- For Android devs who have worked with
VirtualDisplayin production—which OEM skins (One UI, HyperOS, ColorOS) gave you the weirdest lifecycle or input quirks? Would love to compare notes in the comments.

Top comments (0)