DEV Community

Cover image for I open-sourced DomA: an AI automation agent for the browser
yin shen
yin shen

Posted on

I open-sourced DomA: an AI automation agent for the browser

Most “AI browser” products ask you to abandon the browser you already live in.

New Chromium fork. New profile. Friction with passwords, extensions, and muscle memory. Your logged-in sessions don’t come along for the ride.
DomA takes the opposite bet: stay in Chrome (or Edge), keep the tab you’re already on, and run an agent from a side panel.

That’s why we open-sourced it.

The real pain isn’t “can it click?” — it’s accuracy

Browser agents often look impressive in demos and fall apart on real sites:

  • Every micro-step round-trips to the main LLM (slow + context drift)
  • Overlays, suggest boxes, shadow DOM, and CSP break brittle selectors
  • Successful workflows don’t compound — you re-solve the same site next week

DomA’s open core is built around two answers to that pain: Jev inner loops and solidified Skills.

Meet you where you are (extension, not a fork)

DomA is a Chrome/Edge extension, not a standalone AI browser and not a hosted “cloud browser.”
You keep:

  • bookmarks and extensions
  • existing logins / cookies
  • the browser you already trust day-to-day You chat in the side panel. The agent grounds on the page, then acts: click, type, navigate, collect, multi-step flows.

BYOK (secondary, but intentional)

The open edition is Bring Your Own Key: OpenAI-compatible providers (and similar endpoints).

You’re not locked into our model marketplace. Keys stay in your setup; model calls go to your chosen provider.
(That’s important for developers — but the product thesis is still “agent in your browser,” not “BYOK for its own sake.”)

How DomA perceives and acts

Set-of-Mark (SoM) grounding

Interactive elements get numbered overlays. The model sees a screenshot with markers and chooses index-based actions instead of guessing fragile CSS selectors.

Multi-strategy execution

Synthetic events for speed, CDP fallbacks for stubborn widgets, and coordinate-level interaction when needed.

Context hygiene

Large tool payloads can be spilled out of the chat transcript and re-read via refs — so long tasks don’t drown the model in noise.

Accuracy lever #1 — Jev inner loops

Classic agent loops look like:

screenshot → main LLM → click → screenshot → main LLM → type → …
That burns latency and invites inconsistency between steps.

Jev is DomA’s optional inner loop on browser_screenshot (act/verify):

  • Given a goal, it can decide and execute multiple grounded steps inside the tool
  • Refresh SoM → choose click/type → act → continue until the goal progresses or it must hand back
  • Verify passes can check whether the page actually moved toward the goal Net effect: fewer main-LLM round trips for the boring middle of a form/flow, and tighter coupling between perception and action — which is where a lot of “almost right” failures come from. Jev doesn’t magically solve every site. It attacks the failure mode of over-delegating every micro-action to a chatty outer loop.

Accuracy lever #2 — solidified Skills

Even with good grounding, accuracy compounds only if successful procedures become reusable.
DomA supports Agent Skills:

  1. Capture a workflow that worked on a real site (instructions, constraints, host rules)
  2. Load it later with browser_invoke_agent_skill instead of re-deriving the plan from scratch
  3. Treat skills as living playbooks — site-specific heuristics, not one-off chat luck This is how you move from “the model might figure it out” to “this site has a known reliable path.” If browser automation feels unreliable, it’s often because the system has perception + actions, but no durable procedural memory. Skills are that layer.

What’s in the open-source edition

  • Side-panel chat → tool-calling browser agent
  • SoM screenshot loop + click/type/navigate tools
  • Jev-style inner act/verify looping (when enabled)
  • Skills for reusable site/task procedures
  • Build from source or load a prebuilt package from Releases

There is also a commercial store build with extras (video download, userscripts, ad blocking, …). This post is about the open core.

Top comments (0)