DEV Community

Pointchecknote
Pointchecknote

Posted on

Browser Automation with Claude + Playwright MCP: Why Accessibility Snapshots Beat Screenshots

If you've tried "AI clicks the screen" automation, you know the flaky part is usually the vision step. Playwright's MCP server takes a different route: it feeds the model the page's accessibility tree, not a screenshot.

Why this matters

  • Deterministic: every interactive element gets a stable ref, so the agent targets "this button" instead of guessing from pixels.
  • Token-efficient: a snapshot is ~200-400 tokens vs thousands for a DOM dump or screenshot.
  • No vision model required.

Install (Claude Code)

claude mcp add playwright npx @playwright/mcp@latest
Enter fullscreen mode Exit fullscreen mode

It registers at user scope, runs as a local stdio subprocess, and uses headed Chromium by default. Claude Desktop, Cursor, VS Code and Windsurf work too.

Where it shines

Repetitive form flows, data extraction, and E2E/regression tests you can describe in plain language. Keep browser_snapshot as the default and reach for browser_screenshot only when you need a visual check.

Limits

It drives a real local browser, and elements outside the accessibility tree (canvas, custom widgets) are harder. Keep a human in the loop for anything sensitive.

Tags: ai, webdev, programming, testing

Disclosure: I publish Pointchecknote; a fuller walkthrough is here: https://pointchecknote.com/en/posts/2026-08-05-playwright-mcp-claude/

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen

Good default. The operational detail I’d add is that refs are only deterministic relative to a particular snapshot and page state—not durable identifiers across navigation or reactive rerenders. A robust loop is snapshot → act → verify a semantic postcondition → snapshot again. For example, after “Submit,” assert the expected record, status, or confirmation text, not merely that the click succeeded. I also keep screenshots as a second evidence channel for visual regressions, canvas, and layout while accessibility snapshots drive interaction. That gives you cheap control plus independent outcome evidence, which is what turns an agent demo into a regression test.