How I Run My AI Coding Agent From My Phone: Building a Remote Control Layer

TL;DR

I run an autonomous coding agent (built on Claude Code) on a Mac mini at home, and for months the only way to check on it away from my desk was SSH from my phone — which is exactly as miserable as it sounds. So I built a small remote control layer: a file-based command queue with an allowlist of safe intents, polled by a watcher daemon, with results pushed back to my phone. This post covers the architecture, the code, and 5 lessons about designing async interfaces for AI agents. 📱🤖

The Problem

For about a year I've been running a fully autonomous implementation system — a set of Claude Code sessions that pick up tasks, write code, verify their own work, and commit, mostly without me. It lives on a Mac mini in my apartment and runs on a schedule, day and night.

Here's the thing nobody tells you about autonomous agents: the more autonomous they get, the more your anxiety becomes the bottleneck. The agent is fine. It's been fine for months. But when I'm out for dinner and a run kicks off at 9pm, some part of my brain goes: did it start? did it get stuck? is it currently rewriting my git history?

My first answer was the obvious one: SSH from my phone.

I set up Tailscale (which, to be fair, is fantastic — zero-config WireGuard mesh, the Mac mini is always reachable), installed a terminal app on my phone, and connected to the tmux session where the agent runs.

It technically worked. It was also awful:

• Phone keyboards are hostile to shells. Autocorrect turned git log --oneline into git log —oneline (em dash!) more times than I can count.
• Sessions drop constantly. Walk between two subway stations and your SSH connection is gone. tmux saves the session, but reattaching and scrolling on a 6-inch screen is pain.
• One bad command has no undo. At 2am, half asleep, I once typed a command into the agent's tmux pane instead of my own shell pane. It got interpreted as user input to the agent mid-task. Nothing broke, but the agent politely tried to act on my gibberish, and I spent 20 minutes untangling what it had done. ⚠️

That was the moment I realized the actual requirement. I didn't want a terminal on my phone. I wanted three specific things:

1. Glance: "What's the agent doing right now?" answered in 5 seconds.
2. Nudge: send a small set of known-safe commands ("pause", "retry the last task", "run the test suite").
3. Walk away: fire a command, put the phone in my pocket, get the result when it's done.

None of those need an interactive shell. All of them are asynchronous by nature. So I stopped trying to make SSH pleasant and built a message queue instead.

How I Solved It

The architecture is deliberately boring:

sequenceDiagram
    participant Phone
    participant Inbox as Message channel
    participant Watcher as Watcher daemon (Mac mini)
    participant Queue as Command queue (files)
    participant Agent as Coding agent

    Phone->>Inbox: "status" / "run-tests" / "pause"
    Watcher->>Inbox: poll every 30s
    Watcher->>Queue: write command.json
    Agent->>Queue: pick up at next loop iteration
    Agent->>Queue: write result.json
    Watcher->>Inbox: post result back
    Inbox->>Phone: notification with result

Three pieces:

1. The transport: any messaging surface you already have

I use a private messaging channel (a Telegram bot works great for this; so does a private Discord channel or even email). The only requirements: it has a decent phone app with push notifications, and it has an API you can poll.

The key insight is that the phone never talks to the agent directly. It talks to a dumb inbox. This means no open ports on the Mac mini, no VPN required for the phone side, and the whole thing keeps working when I'm on hotel Wi-Fi that blocks everything except HTTPS.

2. The watcher: a 100-line Python daemon

A small Python 3.13 script runs on the Mac mini under launchd. Every 30 seconds it polls the inbox for new messages, validates them against an allowlist, and writes them into a queue directory as JSON files:

ALLOWED_COMMANDS = {
    "status":     {"risk": "read",  "needs_confirm": False},
    "log-tail":   {"risk": "read",  "needs_confirm": False},
    "run-tests":  {"risk": "exec",  "needs_confirm": False},
    "pause":      {"risk": "state", "needs_confirm": False},
    "resume":     {"risk": "state", "needs_confirm": False},
    "retry-last": {"risk": "exec",  "needs_confirm": True},
    "rollback":   {"risk": "write", "needs_confirm": True},
}

def handle_message(text: str) -> str:
    cmd = text.strip().lower()
    if cmd not in ALLOWED_COMMANDS:
        return f"Unknown command. Try: {', '.join(ALLOWED_COMMANDS)}"
    spec = ALLOWED_COMMANDS[cmd]
    if spec["needs_confirm"] and not confirm_pending(cmd):
        mark_pending(cmd)
        return f"⚠️ '{cmd}' is destructive. Send it again within 60s to confirm."
    enqueue({"command": cmd, "ts": now_iso(), "source": "phone"})
    return f"Queued: {cmd}"

Notice what's not here: there is no shell command. There is no way to send arbitrary text to the agent. Every command is a named intent with a known risk level. This is the single most important design decision in the whole system, and I'll come back to it in the lessons.

3. The queue: JSON files in a directory

The queue is not Redis. It's not RabbitMQ. It is a directory of JSON files:

command-queue/
├── inbox/
│   └── 2026-07-10T13-42-01_run-tests.json
├── processing/
└── done/
    └── 2026-07-10T13-42-01_run-tests.result.json

The agent's main loop already had a natural checkpoint between tasks, so I added one step: check inbox/, and if there's a command file, move it to processing/ (an atomic rename, which on the same filesystem gives you a free locking mechanism), execute the intent, and write a result file to done/.

A result file looks like this:

{
  "command": "run-tests",
  "started": "2026-07-10T13:42:31+09:00",
  "finished": "2026-07-10T13:44:02+09:00",
  "ok": true,
  "summary": "142 passed, 0 failed, 3 skipped (91s)"
}

The watcher sees the new file in done/, formats the summary, and posts it back to the channel. My phone buzzes. Total round trip for run-tests: about two minutes, of which I spent zero staring at a terminal. ✅

The status command deserves special mention

status is the command I send 10x more than all others combined, so it got special treatment: the agent continuously maintains a small status.json (current task, last heartbeat, tasks completed today, last error), and the watcher answers status directly from that file without touching the queue at all. Response time is under a second, and it works even when the agent is mid-task or wedged.

🤖 Agent status (13:58 JST)
▶ Working on: refactor retry logic in scheduler tests
⏱ Heartbeat: 12s ago
✅ Today: 4 tasks done, 0 failed

If the heartbeat is older than 10 minutes, the watcher flags it — which has caught two genuine hangs that I'd otherwise have discovered hours later.

Lessons Learned

1. You don't need a terminal, you need an inbox 📬

I wasted weeks trying to make phone-SSH ergonomic before admitting the interaction model was wrong. Remote-controlling an autonomous agent is not remote shell access — the agent is already asynchronous, so the interface should be too. Fire-and-forget with a notification on completion beats a live terminal in every scenario I actually encounter on my phone.

2. Commands are intents, not strings

The allowlist felt restrictive when I built it. It has never once felt restrictive in use. A closed set of named intents means: nothing to typo, nothing for autocorrect to mangle, a known risk level per command, and a system I can reason about when half asleep. Every time I've thought "I wish I could just send a raw shell command," the honest answer was that the task could wait until I was at a real keyboard — and that's a feature.

3. Make read-only operations free and instant

Roughly 90% of my interactions are "how's it going?" Serving status from a continuously-updated file — rather than through the queue — made the common case instant and zero-risk. If reading state is cheap, you check more often, worry less, and reach for risky interventions less. Cheap observability actively prevents dangerous meddling.

4. Polling every 30 seconds is fine. Really.

My first instinct was websockets and instant delivery. Then I asked: what's the actual cost of a command sitting in an inbox for 30 seconds, for a system whose tasks take minutes? Zero. The polling loop is ~40 lines, survives network blips by construction, and has needed no maintenance in months. For a single-user tool, latency tolerance is a gift — accept it.

5. The guardrails are for you, not the agent 🛡️

I designed the confirmation flow ("send rollback twice within 60 seconds") thinking about agent safety. In practice, every incident it has prevented was me: fat-fingered taps, a command sent to the wrong chat, a 2am "fix" that morning-me would never have sent. The agent is deterministic and predictable. The human with a phone at 2am is not. Design accordingly.

What's Next

Three things on my list:

• Failure-driven push, not just pull. The watcher already flags stale heartbeats; I want proactive notifications for failed tasks with a one-tap retry-last.
• Richer daily digests. One evening summary ("what the agent did today") instead of me sending status six times.
• Photos of diffs. Rendering a compact diff summary as an image is more readable on a phone than markdown — an experiment for a weekend.

What I will not add: raw shell passthrough, or a full web dashboard. Every time I get feature-hungry I reread lesson 2.

Wrap-up

If you're running any long-lived AI agent — coding or otherwise — I'd genuinely recommend building the boring version of this: one status file, one allowlist, one directory of JSON files. It's an afternoon of work, and it converts "vague background anxiety about your robot" into "a glance at your phone."

If you've built a different remote-control setup for your agents — a bot, a dashboard, something weirder — I'd love to hear how you handled the safety side. Drop it in the comments. 💬

And if this kind of build-in-public write-up about running autonomous coding agents is your thing, follow me here on Dev.to — I post regularly about what works (and what breaks) running Claude Code unattended. 🚀