DEV Community

老毛
老毛

Posted on AI-assisted

One typed call per DOM step: a System One inner loop inside ego-browser

#ai #productivity #webdev #opensource

Browser agents have a dumb habit: they spend a full LLM turn — screenshot,
chain of thought, function call — to do something a radio button could have
told you. Click "next page". That costs seconds and a pile of tokens, every
single step.

jev-ultrafast (the standalone browser agent) showed a different shape: keep
the loop dumb and fast. Don't ask a model to reason about the page; ask a
typed decision layer to pick the next action from a fixed menu. I ported
that loop into ego-jev, an agent skill for ego lite — so the fast path runs
inside a real browser with your sessions and logins, and the slow, smart
agent is still there when a step actually needs it.

The step shape

Each iteration does four things:

  1. Take an ego-browser snapshot of the page.
  2. Number every interactive element. The a11y tree gives refs like @5. On top of that, a DOM walk finds "dark matter" — divs with click handlers, cursor:pointer cards, shadow-root and iframe content that never gets a ref — and tags them d1, d2, ...
  3. Send one System One request that answers several questions at once: which operation (click/fill/select/scroll/wait/done/escalate), and — speculatively — which target each operation would use. Only the selected operation's target is ever executed, so two decisions cost one network round trip.
  4. ego executes the pick: page.click("@5"), fill, selectOption, or a coordinate click for DOM-tagged elements.

On the TypeSafe endpoint this lands at roughly 300–550 ms per step with
20–120 candidate elements — versus multi-second reasoning turns plus a
screenshot upload each step.

What it refuses to do

The loop escalates instead of improvising: login/2FA pages, payment,
deletion, upload, free-text writing, low confidence, repeated no-ops, and a
guard word list ("pay", "delete", "下单", "确认订单"…) all bail back to the
outer agent with a reason string. done is also just a claim — the caller
passes a verify function that checks the real end state.

Try it 

npx skills add ZephyrDeng/ego-jev
Enter fullscreen mode Exit fullscreen mode

You need ego lite and either TYPESAFE_API_KEY or AI_GATEWAY_API_KEY. If you
don't have a key, scripts/selftest.mjs runs the whole loop inside
ego-browser with a scripted decider — same mechanics, nothing to configure.

The interesting question to me is where the boundary should sit: which steps
deserve a smart model and which deserve a 0.4s lookup. If you run it on
something real, I'd like to hear where it escalates.
https://github.com/ZephyrDeng/ego-jev

Top comments (0)