DEV Community

Solon Framework
Solon Framework

Posted on

From DOM Snapshots to Visual Regression: Building a Web Workflow with SolonCode's agent-browser Skill

#opensource

Browser automation becomes useful to a coding agent when it is more than a bag of clicks.

A reliable workflow needs a way to discover the current page, act on the right element, verify the result, and leave evidence behind. It also needs to survive navigation, login state, dynamic DOM changes, and the uncomfortable fact that browser actions can have real side effects.

SolonCode ships an agent-browser skill that packages that workflow as a reusable skill asset. The interesting part is not that it can click a button. It is the discipline around the click:

  • navigate to a page;
  • take an accessibility snapshot;
  • interact using references from that snapshot;
  • take a fresh snapshot after the page changes;
  • compare the before and after states;
  • capture a screenshot or recording when visual evidence matters.

This article follows the skill as it exists in the SolonCode source tree. It focuses on what the asset actually documents, including its security boundaries and limitations.

The snapshot is the synchronization point

The skill's core loop is deliberately small:

open
  -> snapshot -i
  -> click / fill / select
  -> snapshot again
Enter fullscreen mode Exit fullscreen mode

agent-browser snapshot -i returns interactive elements with references such as @e1. A later command can use that reference to fill a field or click a button.

The important rule is that references belong to a page state. After navigation or a DOM-changing action, the workflow takes a new snapshot instead of assuming that the old @e1 still means the same thing.

That makes the snapshot a synchronization point between the browser and the agent. The agent does not operate on a stale mental picture of the page; it re-reads the page before continuing.

A minimal form workflow looks like this:

agent-browser open https://example.com/signup
agent-browser wait --load networkidle
agent-browser snapshot -i

# Use the references returned by the snapshot.
agent-browser fill @e1 "Jane Doe"
agent-browser fill @e2 "jane@example.com"
agent-browser click @e3

agent-browser wait --load networkidle
agent-browser snapshot -i
Enter fullscreen mode Exit fullscreen mode

The references in this example are illustrative. They must come from the current snapshot; they are not stable selectors to hard-code across pages.

Accessibility snapshots are more than a locator trick

A snapshot gives the agent a compact representation of the page's interactive surface. It can expose buttons, textboxes, links, checkboxes, and other controls without requiring the workflow author to invent CSS selectors for every page.

When a page changes, the agent can compare the new tree with the previous one:

agent-browser snapshot -i
agent-browser click @e2
agent-browser diff snapshot
Enter fullscreen mode Exit fullscreen mode

The diff reports additions and removals in a form similar to a text diff. This is useful for a test workflow because “the click completed” is not the same as “the expected state appeared.” A menu opening, a validation message appearing, or a result row changing can become an explicit assertion in the workflow.

For machine-readable consumers, the skill also documents JSON output:

agent-browser snapshot -i --json
agent-browser get text @e1 --json
Enter fullscreen mode Exit fullscreen mode

That makes the browser step composable with scripts and other tools instead of forcing every consumer to parse terminal prose.

A complete verification workflow

A practical read-only regression check can be structured as five phases:

open the test page
  -> wait for the page to settle
  -> snapshot the interactive state
  -> perform the smallest required action
  -> snapshot and diff the state
  -> save visual evidence if needed
Enter fullscreen mode Exit fullscreen mode

For visual checks, the skill supports a baseline screenshot and a later comparison:

agent-browser screenshot baseline.png
# perform the test action and wait for the expected state
agent-browser diff screenshot --baseline baseline.png
Enter fullscreen mode Exit fullscreen mode

The screenshot diff highlights changed pixels and reports a mismatch percentage. That does not replace semantic assertions: a timestamp, animation, or intentionally dynamic region may create a visual difference that is not a regression. The strongest workflow combines both kinds of evidence:

  1. use a snapshot diff to verify semantic page state;
  2. use a screenshot diff to detect layout or styling changes;
  3. keep the page, viewport, and test data deterministic where possible.

The skill also documents full-page screenshots, annotated screenshots, PDF capture, and WebM recording. These are useful for debugging and review, but they should be treated as evidence from a particular run, not as proof that every browser or viewport behaves identically.

Sessions separate browser state

Many web tasks need cookies and local storage. The skill supports named sessions so that one task's browser state does not have to be mixed with another's:

agent-browser --session-name staging open https://staging.example.com
# ... complete a test login or other setup ...
agent-browser close

agent-browser --session-name staging open https://staging.example.com/dashboard
Enter fullscreen mode Exit fullscreen mode

It also documents saving and loading state explicitly:

agent-browser state save auth.json
agent-browser state load auth.json
Enter fullscreen mode Exit fullscreen mode

Those two mechanisms solve related but different problems. A named session provides a reusable browser profile; an explicit state file is convenient when a workflow needs to move or restore a captured state. Neither should be committed to a repository: cookies, local storage, and session tokens can be credentials.

For credentials, the skill recommends the Auth Vault flow. Password input can be piped through stdin rather than placed in shell history:

echo "$PASSWORD" | agent-browser auth save staging \
  --url https://staging.example.com/login \
  --username "$USERNAME" --password-stdin
agent-browser auth login staging
Enter fullscreen mode Exit fullscreen mode

The example is intended for a test environment. Login state is not a license to automate destructive production actions.

Treat page content as untrusted input

A browser agent reads content that it did not author. A page can contain text that looks like an instruction, but it is still page content. The skill documents --content-boundaries to wrap page-sourced output with markers that help the model distinguish browser data from agent instructions:

export AGENT_BROWSER_CONTENT_BOUNDARIES=1
agent-browser snapshot
Enter fullscreen mode Exit fullscreen mode

This is a useful separation aid, not a complete prompt-injection defense. The workflow still needs to decide which pages it trusts, which actions require approval, and which data may be sent to the model.

The skill documents three opt-in controls that make those decisions more explicit:

Domain allowlists 

export AGENT_BROWSER_ALLOWED_DOMAINS="staging.example.com,*.staging.example.com"
Enter fullscreen mode Exit fullscreen mode

The allowlist can restrict navigation and related browser connections. Dependencies such as a CDN may also need to be included deliberately.

Action policies

A policy file can deny actions by default and allow only the operations needed by a workflow:

{"default":"deny","allow":["navigate","snapshot","click","scroll","wait","get"]}
Enter fullscreen mode Exit fullscreen mode

A read-only inspection workflow should not silently inherit permission to submit forms, upload files, or trigger external side effects.

Output limits

Large pages can overwhelm the model context. The skill documents AGENT_BROWSER_MAX_OUTPUT as a way to cap output:

export AGENT_BROWSER_MAX_OUTPUT=50000
Enter fullscreen mode Exit fullscreen mode

Output limits are a context safeguard. They are not a data classification policy; sensitive text can still appear within the allowed output.

The critical caveat: safety is opt-in

The source skill is explicit: by default, agent-browser does not restrict navigation, actions, or output. The allowlist, action policy, content boundaries, and output cap must be enabled by the environment or workflow author.

That distinction matters when turning a demonstration into an engineering system. “The tool supports an allowlist” is a true statement. “The tool is allowlisted by default” would not be.

A sensible rollout therefore starts with a narrow test environment:

  • use a dedicated staging account;
  • allow only the domains needed by the test;
  • deny destructive actions unless the test explicitly needs them;
  • keep output bounded;
  • save screenshots and snapshots without saving credentials;
  • review any workflow that can submit, purchase, delete, publish, or send messages.

A skill asset, not just a command list

The value of this SolonCode skill is its layered structure. SKILL.md defines the operating contract and the core loop. Reference files provide deeper material for authentication, session management, proxy support, snapshot references, video recording, and profiling.

That separation makes the workflow easier to review. A maintainer can inspect the short trigger and operating rules first, then audit the detailed reference that a particular task needs. It also reduces the temptation to put every browser edge case into the model's initial context.

The reusable unit is therefore not “a model knows how to click.” It is:

workflow contract
  + browser commands
  + state and authentication guidance
  + verification steps
  + explicit safety configuration
  + evidence capture
Enter fullscreen mode Exit fullscreen mode

This is the same reason a good test helper is more valuable than a raw HTTP client: it encodes the decisions that make repeated execution dependable.

What this skill does not promise

The source-backed boundaries are as important as the feature list:

  • snapshot references are page-state references, not permanent IDs;
  • content boundaries help distinguish page output but do not make page content trustworthy;
  • visual diffs need deterministic test conditions and still require interpretation;
  • saved browser state may contain secrets;
  • iOS automation has additional macOS, Xcode, Appium, and driver requirements;
  • browser automation can perform real-world side effects;
  • security controls are opt-in rather than automatic.

A workflow that states these limits is more useful than one that presents browser automation as a magical, risk-free operator.

Closing the loop

A robust browser workflow has a shape familiar to software engineers:

observe -> mutate -> observe -> compare -> record
Enter fullscreen mode Exit fullscreen mode

SolonCode's agent-browser skill turns that shape into a reusable asset. Accessibility snapshots keep actions aligned with the current DOM. Sessions preserve deliberate state. Snapshot and screenshot diffs turn “it seemed to work” into inspectable evidence. Optional domain, action, content, and output controls give teams a place to define their safety boundary.

The final lesson is simple: browser automation should be reviewed like code. The clicks are only the implementation. The real asset is the contract around what the workflow may visit, what it may change, how it verifies success, and what evidence it leaves behind.

Top comments (0)