DEV Community

Ashwar Sadh
Ashwar Sadh

Posted on Fully Autonomous

Building a phone remote for a desktop app that has no API

I keep many Claude Code sessions open in Claude Desktop's Code tab. When I walk away, one of them is usually sitting on a permission prompt by the time I'm back. I wanted to see and answer those sessions from my phone. They had to be the same sessions, not copies. The catch is that the desktop app has no public API for any of this.

This post is about the design I ended up with in Relaymote (MIT, Node.js, Windows first). The lessons apply to any "remote control for an app that wasn't built to be remote-controlled".

Rule 1: read from files, act through the app

The split that made everything else workable:

  • Reading never touches the app. Claude Desktop writes a small JSON file per Code session (title, folder, model, archived), and Claude Code writes each conversation as a .jsonl transcript. Relaymote reads those directly, and reads transcripts from the end, so a long session opens fast. So the phone can list and read sessions even when the desktop app is closed.
  • Acting goes through the app. Some actions only exist inside the app: sending a message, answering a permission, renaming, archiving. For those, Relaymote attaches to the app's main-process debugger (Chrome DevTools Protocol on 127.0.0.1). Where the app has an internal method for the action, it calls that method. It falls back to driving the UI only when no such method exists.

Reads are the bulk of the traffic, so a desktop update that changes the UI breaks only actions, never the view.

Rule 2: never type into what the human is doing

UI driving is dangerous when someone is sitting at the keyboard. Every UI action goes through one serialised lane and waits for an idle gate: no keyboard or mouse input for a short while. The remote is therefore polite by construction. It can't race you for focus.

Rule 3: one way in, and it is locked

The phone is a web app, which raised a security question: anything that can send a message to a coding agent can, indirectly, run commands on the PC. So:

  • The control port binds to loopback only.
  • The app port requires an access key on every request. Pairing is a QR link whose key is exchanged once for an HttpOnly cookie.
  • Behind a Cloudflare tunnel, the key is still required. If Cloudflare Access is configured, its JWT is verified (signature, audience, issuer, expiry), never trusted from a header alone.
  • Push notifications are end-to-end encrypted (RFC 8291) with keys generated on the machine.

Rule 4: make waiting someone else's problem

Two small features turned out to matter most day to day:

  • Auto-resume. When a session stops on a usage limit, Relaymote records the reset time and sends a short "continue" when it passes. A session that was mid-turn when the app quit is detected on the next start and continued.
  • Status from a passive read. Status dots and unread state come from reading the sidebar every 30 seconds. Nothing is opened just to look at it.

What I'd tell anyone building something similar

  1. Separate reading from acting early. Files are a stable interface; UIs are not.
  2. Serialise anything that touches the UI, and gate it on human idleness.
  3. Treat "can send a message" as "can run code", and lock the door accordingly.
  4. Write down the fragility. Mine is in the README: a desktop update can break actions until a fix ships, and reads keep working.

The code is on GitHub if you want to read the debugger bridge (lib/bridge.js) or the idle-gated UI lane (lib/desktop.js). I'd value feedback from anyone who has done CDP automation of Electron apps.

Unofficial; not affiliated with or endorsed by Anthropic.

Top comments (1)

Collapse
 
launchgatecheck profile image
Launch Gate •

Does the idle gate recheck the target session and permission prompt immediately before acting? A useful race fixture would queue a phone approval, then have the desktop user switch sessions or dismiss that prompt while the action waits. Once idle, the old approval should expire rather than attach to whichever prompt is now visible.

I'd pair that with a desktop restart while an action is queued, so any target identifiers are resolved again instead of blindly replayed. I haven't run Relaymote; these are stale-target checks suggested by the serialized UI lane, not findings in the implementation.