Let an AI agent drive your real, logged-in Chrome — safely, on localhost

#ai #opensource #node #webdev

An AI coding agent in my terminal kept hitting the same wall: it needed to do real work in the browser — on sites I was already logged into. Pull an authenticated file, fill a review form, operate a web editor. Reading a public page is easy. Acting as me, inside my own session, turned out to be surprisingly hard.

Here's why every obvious approach failed, and the small tool I built instead: ai-browser-bridge.

Why the obvious approaches don't work

OS-level input scripting (AppleScript, xdotool, and friends) is platform-specific, steals window focus, and races against whatever you're doing with your own mouse and keyboard. It also can't be trusted by every app.

Synthetic DOM events — element.dispatchEvent(new MouseEvent(...)) — carry isTrusted: false. Rich editors built on contenteditable, ProseMirror, Slate, and most CSP-strict web apps deliberately ignore untrusted events. So your automated "click" or "paste" just... does nothing.

Headless / profile-isolated automation (Puppeteer, Playwright) spins up a browser that doesn't have your session. You'd have to export authenticated cookies to it — and shipping those to any cloud service is a hard no.

Classic CDP attach (chrome --remote-debugging-port=9222) stopped being an option for the default profile in Chrome 136+.

The architecture

Three small pieces, no cloud, no SDK:

AI agent / your scripts          bridge server               Chrome extension
   bridge <cmd> CLI    ⇄   ws://127.0.0.1:8765 (token)  ⇄   service worker (MV3)
                                                              │ chrome.tabs / scripting
                                                              │ chrome.debugger  → trusted input, CSP-proof eval
                                                              │ chrome.downloads → authenticated downloads

A CLI the agent calls once per operation. It prints JSON to stdout and exits non-zero on failure — a contract any agent can drive without a library.
A WebSocket relay bound to 127.0.0.1, gated by a token.
A Manifest V3 extension that lives in your normal Chrome profile, so every session you're logged into is already there.

The key mechanism: chrome.debugger

The extension attaches chrome.debugger and speaks the DevTools Protocol per tab:

Input.dispatchMouseEvent / Input.insertText produce genuinely trusted input events (isTrusted: true) — the ones strict editors actually accept.
Runtime.evaluate runs in the page regardless of its Content-Security-Policy.
Everything happens in background tabs (active: false), so your focus is never stolen.

Because it runs inside your existing profile, there's no re-authentication and no cookie export — your sessions just work.

What it looks like

bridge() { node /path/to/ai-bridge/server/cli.js "$@"; }

bridge listTabs
bridge newTab   '{"url":"https://example.com"}'          # opens in background
bridge eval     '{"tabId":123,"code":"document.title"}'  # CSP-proof
bridge click    '{"tabId":123,"x":420,"y":310}'          # trusted click
bridge insertText '{"tabId":123,"text":"Hello"}'         # trusted paste at caret
bridge download '{"url":"https://.../file.pdf","filename":"file.pdf"}'  # your cookies
bridge pdf      '{"tabId":123}' --out page.pdf
bridge screenshot '{"tabId":123}' --out page.png

For an agent like Claude Code, you drop a short block into CLAUDE.md describing these commands, and it composes the calls itself: "open my dashboard and screenshot the error" becomes a sequence of newTab / eval / screenshot.

Security was the design constraint

This tool can act as you on any site you're logged into, so the threat model came first:

The server binds 127.0.0.1 only — nothing is reachable from the network.
Every client must present a token (generated on first run, chmod 600).
An optional per-domain allowlist is enforced before every tab-targeting command.
Every command is logged.
Chrome's native "… is debugging this browser" banner is visible whenever the debugger is attached.
No analytics, no telemetry, no external requests of any kind.

I also deliberately kept it off the Chrome Web Store. The install friction — clone the repo, load unpacked — is a feature for something this powerful: it keeps the code auditable and keeps one-click installs out of the hands of people who won't read the security model.