DEV Community

Baptiste Le Bouquin
Baptiste Le Bouquin

Posted on

Your headless browser has as many cookie jars as it has CDP connections

Your headless browser has as many cookie jars as it has CDP connections

I learned this the expensive way while running an authenticated headless agent in production.

The setup was simple on paper: a Chrome extension exports the cookies of a site you are logged into on your desktop, a local relay injects them into Lightpanda, and an agent drives that Lightpanda instance through the Chrome DevTools Protocol. Real logins, no password storage, no OTP babysitting.

It worked. Then it randomly stopped working on dev.to specifically.

The extension synced. The relay reported the cookies as injected. Network.getCookies on the socket I was using returned the expected jar. And yet every navigation came back as logged out. No error, no redirect to a login page worth debugging — just an anonymous session staring back at me.

The jar is scoped to the connection, not to the browser

Here is the part the docs did not tell me: Lightpanda scopes its cookie jar per CDP connection, not per browser instance.

Which means:

  • Connection A (the relay) imports the session.
  • Connection A's getCookies confirms the cookies are there. True.
  • Connection B (my agent, a fresh WebSocket to the same :9222) navigates.
  • Connection B sees an empty jar. Also true.

Both statements are correct simultaneously. There is no corruption, no expiry problem, no SameSite mystery. The two sockets were looking at two different jars that happened to live in the same process. I spent a while suspecting cookie attributes before noticing that the verification call and the navigation call were made on different connections — I was confirming the state of a jar I was never reading from.

In a normal Chrome this is invisible, because there is effectively one browser-wide jar and CDP is a debugging surface on top of it. In an agent-oriented headless browser, the CDP connection is the primary control plane, and connection-scoped state becomes a first-class architectural property, not a detail.

The fix: one connection to rule them all

The version that ships today has the relay own the only CDP connection, permanently. Agents never open a socket to the browser. They POST their CDP commands to the relay, which replays them on the connection that holds the sessions:

from bridge_agent import AuthenticatedSession

s = AuthenticatedSession()          # talks to the relay, not to :9222
s.open("https://dev.to", wait=8)
s.js("document.body.getAttribute('data-user-status')")  # 'logged-in'
Enter fullscreen mode Exit fullscreen mode

Three consequences worth calling out:

Verification has to happen on the acting connection. getCookies from a side socket tells you nothing about what your navigation will see. If you are debugging an auth problem in a headless setup, the very first question is: same connection? Not same browser, not same process. Same connection.

Restart semantics become explicit. When the browser restarts, the jar is gone because the connection is gone. There is no browser state to fall back to. We now replay the last imported session automatically on reconnect and keep on-disk snapshots of each origin's cookie list so a relay restart is a re-import, not a manual re-sync from the desktop.

Security posture changes for the better. If agents can open their own CDP socket, they can create a new jar at will, and "who is allowed to read the session" becomes unenforceable. Serialising every command through one relay with a token and strict origin checks (chrome-extension:// only on the endpoints that deliver the secret) turns an implicit trust boundary into an explicit one. The anti-replay win was a side effect of fixing the jar bug.

What I'd check first, if this is your bug

  1. Print location.href after every navigation from the same connection you navigated on. A target that silently fell back to about:blank looks exactly like a cookie problem — the page you render isn't the page you think you're authenticating.
  2. Never verify state on a different socket than the one you act on.
  3. If your browser exposes /json/list, check how many live targets and connections you actually have. Two WebSockets attached is the smoking gun.
  4. Before blaming SameSite, Secure, Domain, or expiry, prove the reading and writing side share a jar. Attribute problems are real, but they are down the list from "you are literally in another browser context".

Take it with a grain of salt

This is one browser's implementation, not a CDP specification guarantee. What generalises is the lesson: when a headless browser is designed for automation, the CDP connection is a security and state boundary, and you should assume it holds state until proven otherwise. If your stack was built assuming one browser equals one session store, that assumption will fail quietly, and it will fail in the direction that looks like an authentication bug.

If you've hit an equivalent in Playwright storage states, Puppeteer's incognito contexts, or another headless engine, I'd genuinely like to know how you handled the "which connection am I actually reading?" problem. That question cost me a day and it should not have.

The bridge this came out of is open source: https://github.com/Raknaos/lightpanda-session-bridge

Top comments (0)