By Chen Yu — building Operun, a local-first browser kernel. Written 2026-11-02.
TL;DR
The previous guides in this series were about what a snapshot reads: the three walls that hid
text from an a11y snapshot, the 40,000-token budget, and the size of a captured HAR body. This one
is about the other half of the contract — the @eN handle itself.
A snapshot line like @e7: [button 'Pay now'] is a promise: act on @e7 and you touch that button,
not "the seventh thing on the page". That promise is only credible if three things hold. The number
must be an identity rather than a position, so a cookie banner appearing above it does not renumber
the page under a workflow's feet. The handle must be able to fail honestly, so a stale ref returns
nothing instead of a detached node that reports SUCCESS for an event no page ever saw. And the handle
must know where it stops, because a ref names a node inside one frame and carries no frame identity
of its own.
None of that is free. It costs two maps, one pruning pass per snapshot, a structural type check
across realms, an exclusion list of things the kernel cannot actually click, and a refusal to accept
@e1 >>> #inner at all.
The failure a position would cause
Number the controls by their order in the walk and everything works until the page changes. Then it
does not merely get slower — it gets quietly wrong. A consent banner mounts above the form, one
table row is inserted, a "you have unsaved changes" toast pushes the layout down by forty pixels, and
every ref the agent is still holding now names a different control. Nothing throws. The next click
lands on the wrong button and the trace looks like the agent made a bad decision, when in fact the
address book was rewritten.
So the scanner assigns numbers by node identity, not by index. The in-page function keeps a
WeakMap<Element, string> called byNode — "which ref is this node" — and a Map<string, Element>
called byRef — "which node is @e1". When a control is handed a ref the code asks byNode first:
function refFor(node: Element, cache: Slot): string {
const known = cache.byNode.get(node)
if (known !== undefined) {
// Re-register: the index may have pruned this node while it was detached,
// and `byNode` is the authority on which ref the node owns.
cache.byRef.set(known, node)
return known
}
const ref = '@e' + cache.next
cache.next += 1
cache.byNode.set(node, ref)
cache.byRef.set(ref, node)
return ref
}
Two details in those ten lines matter more than the rest of the file. First, a node that already has a
number keeps it forever — across re-renders, across being pulled out of the DOM and put back. Second,
next only ever increases and is never reused. A number that has been spent cannot come back as a
different control later, which is what makes "a stale ref never resolves" a property rather than a
hope.
Two maps, because there are two questions
byRef and byNode look redundant until you ask what each one is for. byRef answers which node is
@e1 — that is what an action needs. byNode answers which ref is this node — that is what
stability needs, and it is a WeakMap on purpose: a node the page throws away stops being remembered
without anything having to sweep it.
byRef is the opposite: a plain Map holds its keys strongly and would pin every removed subtree
for the life of the document. On a dashboard that re-renders a table every few seconds, that is a
leak measured in megabytes per minute. So every snapshot starts with one pass:
cache.byRef.forEach((node, ref) => {
if (!node.isConnected) cache.byRef.delete(ref)
})
Pruning costs nothing in correctness because a node that comes back — a reparented row, a virtualised
list item re-mounted — is re-registered from byNode on the next walk, getting its original number
back. The index is a cache of the truth; byNode is the truth.
Where the cache lives, and who may rewrite it
The cache cannot live in the kernel. In an MV3 world the kernel's home is a service worker that
shares no memory with any page and that Chrome is free to tear down between two steps of the same
task — state kept there would be lost exactly when it is needed.
The cache also cannot live in the page's own JavaScript. The primitives run in the extension's
isolated world, which page script cannot see or write to. That is not paranoia for its own sake: a
cache a page could edit is a cache a hostile page could repoint, and an @e1 that silently becomes a
different control is worse than a ref that fails.
So the cache is an object on the isolated world's globalThis, under __operunRefCache__, written by
the snapshot function and read by the primitives that act on a ref. That lifetime is exactly right: it
outlives a single injected call, because Chrome re-evaluates each function but not the world it runs
in, and it dies with the document — and a navigation is a different document, so refs from the
previous page do not survive into the next one.
Two consequences are worth spelling out. chrome.scripting.executeScript serialises a function with
toString() and re-evaluates the source in the frame, so an injected function cannot import the cache
module — the key and the lookup exist twice, once for kernel-side use and once as a literal inside the
function, pinned together by a unit test. And the "is this our cache" check is structural rather than
instanceof Map, because a Map constructed in another realm is a perfectly healthy Map that this
realm's instanceof would reject as foreign. A cross-realm identity check would have turned a working
cache into a mysterious "ref not found".
A ref that resolves to nothing beats a ref that lies
Resolution is one line of judgement: a node either isConnected or the ref resolves to null. The
comment in the source is blunt about why null is the only acceptable failure mode — a detached node
still has click() and dispatchEvent(), its events fire into nothing, and the step would report
SUCCESS for a page that never saw it. A workflow that fails loudly can be debugged. A workflow that
succeeds silently writes the wrong price into the wrong product.
null deliberately does not distinguish "this ref was never handed out" from "the node left the
document". Both mean the caller can do nothing different, so inventing two error codes would only give
the caller more ways to be wrong.
Handles the kernel cannot pull
A ref is a promise that something can be done with it, so the walk skips nodes where nothing can be
done:
-
<option>inside a native<select>. No primitive in the kernel can pick one by ref — the platform opens its own popup, which no synthetic click reaches. The parent<select>reports the currently chosen value instead. A custom listbox built from[role="option"]elements is reported, because a click does reach those. -
input[type="hidden"]. Excluded by what it is, not by how it is styled. It usually computes todisplay: noneand would fail the visibility test anyway, but a page is free to lay one out, and it would still not be a control. -
Anything under
[hidden],[aria-hidden="true"], or[inert]— a singleclosest()test.inertexists precisely to make a subtree unreachable; handing out a ref into it would be handing out a handle to a node the platform refuses to route events to. -
Invisible elements, tested by walking ancestors for
display: none,visibility: hiddenorvisibility: collapse. Same rule as the Tab order, so what the agent reads and what the keyboard can reach cannot disagree. -
Empty prose blocks, in content mode only.
<p></p>is scaffolding; an@eNnaming it is a handle to nothing.
The prose path is worth a note because it is where role === null is handled rather than skipped.
A declared role that is not a control (role="heading", role="presentation") is a statement about
interaction, and content mode is not asking about interaction — so <h1 role="heading"> still gets
its text reported. That single branch is the difference between a content snapshot of a documentation
page and a content snapshot with a hole in it.
Where a ref stops: the frame boundary
Cross-origin iframes are the classic blind spot. A mailbox's login form lives in a frame the
same-origin policy keeps a top-level querySelector from entering, and no amount of >>> changes
that: the piercing chain is a CSS selector per frame walking inwards, and the browser still enforces
the origin boundary at every hop.
Operun's answer is not to pierce harder but to ask every frame at once.
executeScript with allFrames: true returns one result per frame, each carrying the frameId the
next step must name to act inside it, and those results are merged into one text:
=== Frame: 0 (Top: https://mail.example.com) ===
@e1: [button 'Compose']
=== Frame: 849201 (https://login.example.net/signin) ===
@e1: [textbox 'Email'] [frame_id=849201]
@e2: [textbox 'Password'] [frame_id=849201]
Note that frame 849201 has its own @e1. That is not a collision, because a ref's scope is the frame
its snapshot ran in — the top frame is the default context, so only a frame an action must name
carries the [frame_id=N] marker. This is also why @e1 >>> #inner is rejected at the payload
boundary rather than attempted: a reference already belongs to a frame and carries no frame identity of
its own, so the chain has no meaning that resolves to one element. Refusing it keeps the ref cache's
scope a single statable rule instead of one with a special case per position in the chain. Blank
segments like #frame >>> fail the same check — not a typo, an unanswerable question.
Two smaller decisions in the merge are the kind that only show up in production. A frame that answered
nothing — a blank ad iframe that navigated mid-injection, an origin the build is not granted — is
skipped rather than thrown, because one mute frame must not cost you the other eleven. And the frames
are sorted by id before merging, because executeScript promises no ordering and the same page must
read as the same text every time; an unstable snapshot is a cache-buster for every layer above it,
including the token budget, where each line is charged ceil(length / 3) against a 40,000-token
allowance and the trailer … N more elements omitted is the reader's only warning that what it holds
is not the whole page.
Conclusion
The interesting part of a browser kernel is not the click. It is everything that has to be true before
the click is allowed to happen: a number that means a node and not a slot, an index that can be pruned
without losing identity, a cache the page cannot rewrite, a failure mode that is a refusal instead of
a lie, an exclusion list that admits what the platform will not let you touch, and a frame boundary
that is respected rather than papered over.
Each of those is a small piece of code. Together they are the difference between a workflow you can
leave running and a workflow you have to watch.
About Operun
Operun is a local-first browser kernel: a Chrome extension plus a runner that
drives your own logged-in browser, with an a11y snapshot built for agents rather than for humans to
read. Snapshots, refs and selectors are resolved in the page and in the isolated world, which is why
the handle semantics above can be stated as rules instead of best-effort behaviour. The kernel keeps
your session, your cookies and your pages on your machine; the cloud, when you use one, routes steps
and never receives your page content by default. If you are building agents that touch real accounts,
start with the a11y snapshot and the token budget, then read the selector contract — the handles are
where reliability actually lives.
Top comments (0)