Your headless browser has as many cookie jars as it has CDP connections

I learned this the expensive way while running an authenticated headless agent in production.

The setup was simple on paper: a Chrome extension exports the cookies of a site you are logged into on your desktop, a local relay injects them into Lightpanda, and an agent drives that Lightpanda instance through the Chrome DevTools Protocol. Real logins, no password storage, no OTP babysitting.

It worked. Then it randomly stopped working on dev.to specifically.

The extension synced. The relay reported the cookies as injected. Network.getCookies on the socket I was using returned the expected jar. And yet every navigation came back as logged out. No error, no redirect to a login page worth debugging — just an anonymous session staring back at me.

The jar is scoped to the connection, not to the browser

Here is the part the docs did not tell me: Lightpanda scopes its cookie jar per CDP connection, not per browser instance.

Which means:

• Connection A (the relay) imports the session.
• Connection A's getCookies confirms the cookies are there. True.
• Connection B (my agent, a fresh WebSocket to the same :9222) navigates.
• Connection B sees an empty jar. Also true.

Both statements are correct simultaneously. There is no corruption, no expiry problem, no SameSite mystery. The two sockets were looking at two different jars that happened to live in the same process. I spent a while suspecting cookie attributes before noticing that the verification call and the navigation call were made on different connections — I was confirming the state of a jar I was never reading from.

In a normal Chrome this is invisible, because there is effectively one browser-wide jar and CDP is a debugging surface on top of it. In an agent-oriented headless browser, the CDP connection is the primary control plane, and connection-scoped state becomes a first-class architectural property, not a detail.

The fix: one connection to rule them all

The version that ships today has the relay own the only CDP connection, permanently. Agents never open a socket to the browser. They POST their CDP commands to the relay, which replays them on the connection that holds the sessions:

from bridge_agent import AuthenticatedSession

s = AuthenticatedSession()          # talks to the relay, not to :9222
s.open("https://dev.to", wait=8)
s.js("document.body.getAttribute('data-user-status')")  # 'logged-in'

Three consequences worth calling out:

Verification has to happen on the acting connection. getCookies from a side socket tells you nothing about what your navigation will see. If you are debugging an auth problem in a headless setup, the very first question is: same connection? Not same browser, not same process. Same connection.

Restart semantics become explicit. When the browser restarts, the jar is gone because the connection is gone. There is no browser state to fall back to. We now replay the last imported session automatically on reconnect and keep on-disk snapshots of each origin's cookie list so a relay restart is a re-import, not a manual re-sync from the desktop.

Security posture changes for the better. If agents can open their own CDP socket, they can create a new jar at will, and "who is allowed to read the session" becomes unenforceable. Serialising every command through one relay with a token and strict origin checks (chrome-extension:// only on the endpoints that deliver the secret) turns an implicit trust boundary into an explicit one. The anti-replay win was a side effect of fixing the jar bug.

What I'd check first, if this is your bug

1. Print location.href after every navigation from the same connection you navigated on. A target that silently fell back to about:blank looks exactly like a cookie problem — the page you render isn't the page you think you're authenticating.
2. Never verify state on a different socket than the one you act on.
3. If your browser exposes /json/list, check how many live targets and connections you actually have. Two WebSockets attached is the smoking gun.
4. Before blaming SameSite, Secure, Domain, or expiry, prove the reading and writing side share a jar. Attribute problems are real, but they are down the list from "you are literally in another browser context".

Take it with a grain of salt

This is one browser's implementation, not a CDP specification guarantee. What generalises is the lesson: when a headless browser is designed for automation, the CDP connection is a security and state boundary, and you should assume it holds state until proven otherwise. If your stack was built assuming one browser equals one session store, that assumption will fail quietly, and it will fail in the direction that looks like an authentication bug.

If you've hit an equivalent in Playwright storage states, Puppeteer's incognito contexts, or another headless engine, I'd genuinely like to know how you handled the "which connection am I actually reading?" problem. That question cost me a day and it should not have.

The bridge this came out of is open source: https://github.com/Raknaos/lightpanda-session-bridge

Comments

[Arturo Enrique Mata Garcia]

Hi. Excellent find and very well documented. Those kinds of subtleties in control architecture and how engines handle state per connection—instead of a process-wide global store—are the exact kind of things that make you waste an entire day chasing ghosts with SameSite or Secure attributes.

What's fascinating is how this behavior exposes a fundamental difference between debugging an interactive browser and operating an automated headless environment: the control plane (CDP) effectively becomes the state-isolation boundary.

I ran into something very similar when designing evaluation pipelines and automated crawling in isolated environments, where socket reuse or opening multiple concurrent connections broke session persistence without throwing any explicit errors. Centralizing the entire flow through a single relay or gateway that manages the session and serializes commands doesn't just solve the cookie issue; it also drastically tightens isolation and security posture (preventing any individual agent from exposing or tampering with storage at will).

I'm particularly interested in this point from your write-up: could you elaborate on how the CDP protocol behaves at the transport level (WebSocket frames) when multiple contexts or iframes are involved, or do you consider this socket-isolation behavior exclusive to lightweight implementations like Lightpanda compared to a standard Chromium setup?

Thanks for sharing the implementation details of lightpanda-session-bridge. Taking serious note of always verifying state on the exact same socket where the action is executed.

Great contribution!

[Raknaos]

Thanks Arturo! The per-connection scoping is exactly the part that bit me first. I kept assuming the jar was process-wide like a normal browser profile, so my raw CDP client would connect fine, see the target, and then hit login walls on every navigation. The 'aha' was realizing Lightpanda attaches the synced cookies to the connection that installed them, not to the target — which is why everything in the bridge now flows through one relay connection instead of handing agents a 9222 socket to share.

[Jo Do]

The per-connection jar scoping is a nasty default because every sanity check you naturally run lies to you: Network.getCookies on your socket returns your jar, so the probe and the problem live in different scopes.

The general lesson: after any cookie injection, verify from the consumer's perspective, not the injector's. Fetch something session-gated over the same connection the agent will actually use, and treat that as the only green signal. And a line worth adding to runbooks: "logged out anonymously with no error" is an auth-transport symptom, not an auth-credential symptom. Nobody should be rotating cookies for a scoping bug.

[Raknaos]

Agreed — the failure mode is extra misleading because the probe passes. That's what pushed us to make the consumer-side check the only green signal in our runbook: one session-gated fetch over the same connection the agent will actually use, nothing else counts. It caught a real jar-flip for us last week that an injector-side check would have happily signed off on.

"Logged out anonymously with no error" as an auth-transport symptom is also going straight into ours. We'd been reading that pattern as a credential expiry and wasting restarts on it.