DEV Community

Neeraj Sharma
Neeraj Sharma

Posted on AI-assisted

Pixels vs the Accessibility Tree: I Measured What a Web Page Costs an AI Agent (67 Lines of Python)

A laptop showing a web page beside a glowing tree diagram

Vercel Labs open-sourced agent-desktop this month: a native CLI that lets AI agents control macOS, Windows and Linux by reading the accessibility tree instead of looking at pixels. Google's Computer Use also reached general availability. So there are now two serious ways for an agent to "see" a screen, and they bill you very differently.

I wanted numbers, not opinions. So I wrote 67 lines of Python that load real pages and measure what each view costs in tokens.

The three views

  1. Screenshot. One image of the viewport. Anthropic's published estimate for an image is width * height / 750 tokens, so a 1280x800 screenshot costs about 1,365 tokens regardless of what is on it.
  2. Full accessibility tree. Every role, name and link on the page, as text. I used Playwright's aria_snapshot().
  3. Viewport tree. Only the interactive elements (links, buttons, inputs) that are visible right now, one numbered line each.

The script

"""What does a web page cost an agent: pixels vs the accessibility tree?"""
import os, sys, json, math
import tiktoken
from playwright.sync_api import sync_playwright

ENC = tiktoken.get_encoding("o200k_base")
VIEW = {"width": 1280, "height": 800}
URLS = sys.argv[1:] or [
    "https://en.wikipedia.org/wiki/Model_Context_Protocol",
    "https://news.ycombinator.com",
    "https://github.com/microsoft/playwright",
    "https://docs.python.org/3/library/asyncio.html",
    "https://stripe.com/pricing",
    "https://www.bbc.com/news",
]

# Interactive elements that are actually visible in the current viewport.
VISIBLE_JS = """() => {
  const sel = 'a[href],button,input,select,textarea,[role=button],[role=link],[role=tab],[role=menuitem],[role=checkbox],[role=textbox],[role=combobox]';
  const out = []; let total = 0;
  for (const el of document.querySelectorAll(sel)) {
    const r = el.getBoundingClientRect(), s = getComputedStyle(el);
    if (r.width < 2 || r.height < 2 || s.visibility === 'hidden' || s.display === 'none') continue;
    total++;
    if (r.bottom < 0 || r.top > innerHeight || r.right < 0 || r.left > innerWidth) continue;
    const role = el.getAttribute('role') || el.tagName.toLowerCase();
    const name = (el.getAttribute('aria-label') || el.innerText || el.value || el.placeholder || '').trim().replace(/\\s+/g, ' ').slice(0, 60);
    out.push(`[${out.length}] ${role} "${name}"`);
  }
  return {lines: out, total, pageHeight: document.documentElement.scrollHeight};
}"""

def image_tokens(w, h):
    """Anthropic's published estimate: tokens ~= width * height / 750."""
    return round(w * h / 750)

def measure(page, url):
    page.goto(url, wait_until="domcontentloaded", timeout=45000)
    page.wait_for_timeout(2500)
    full_tree = page.locator("body").aria_snapshot()
    vis = page.evaluate(VISIBLE_JS)
    screens = math.ceil(vis["pageHeight"] / VIEW["height"])
    shot = image_tokens(VIEW["width"], VIEW["height"])
    return {
        "url": url,
        "screenshot_tokens": shot,
        "screens_to_scroll": screens,
        "whole_page_as_pixels": shot * screens,
        "full_tree_tokens": len(ENC.encode(full_tree)),
        "viewport_tree_tokens": len(ENC.encode("\n".join(vis["lines"]))),
        "interactive_total": vis["total"],
        "interactive_in_view": len(vis["lines"]),
    }

with sync_playwright() as pw:
    proxy = {"server": os.environ["HTTPS_PROXY"]} if os.environ.get("HTTPS_PROXY") else None
    browser = pw.chromium.launch(proxy=proxy)
    page = browser.new_page(viewport=VIEW)
    rows = []
    for u in URLS:
        try:
            rows.append(measure(page, u))
        except Exception as e:
            rows.append({"url": u, "error": str(e)[:80]})
    browser.close()

print(json.dumps(rows, indent=1))
Enter fullscreen mode Exit fullscreen mode

Run it with pip install playwright tiktoken and playwright install chromium. Pass your own URLs as arguments.

The results

Measured on November 17, 2026, at a 1280x800 viewport. All numbers are tokens.

Page 1 screenshot Whole page as screenshots Full tree Viewport tree Interactive in view / total
Wikipedia article 1,365 10,920 (8 screens) 12,711 578 76 / 723
Hacker News front page 1,365 2,730 (2 screens) 9,905 1,516 167 / 229
GitHub repo page 1,365 15,015 (11 screens) 13,361 907 95 / 274
Python docs (asyncio) 1,365 5,460 (4 screens) 3,772 229 26 / 78
Stripe pricing 1,365 40,950 (30 screens) 19,737 903 98 / 445
BBC News 1,365 10,920 (8 screens) 8,144 260 33 / 119

What the numbers say

1. The full tree is not the cheap option people assume. It cost between 2.8 and 14.5 times one screenshot. On Hacker News the full tree was 9,905 tokens, while scrolling the entire page as screenshots was 2,730. Dense link pages are terrible as trees.

2. For reading a whole long page, the tree usually wins. On 4 of 6 pages the full tree beat scrolling through screenshots. Stripe's pricing page is the extreme: 19,737 tokens as a tree against 40,950 as 30 screenshots.

3. The viewport tree is the best default for acting. It beat a single screenshot on 5 of 6 pages: 229 tokens against 1,365 on the Python docs, a 6x saving. The one loss was Hacker News, where 167 links fit in one viewport and the list (1,516 tokens) cost slightly more than the image.

The part tokens do not show

Cost is half the story. The other half is what the agent does next.

With a screenshot, the model answers "click at (873, 659)". With a tree, it answers "click element 41". In our own browser automation runs, coordinate clicks went wrong whenever the window size changed between two steps: the same coordinates landed on a different button. References to tree nodes kept working, because they point at the element and not at a place on the screen.

That is the real argument for tools like agent-desktop. A tree is not only cheaper on most pages. It gives the agent handles that survive a layout change.

What I would build from this

  • Default to the viewport tree. Numbered, interactive elements only, re-read after every action.
  • Fall back to a screenshot when the tree is poor. Canvas apps, charts, image-heavy pages and custom widgets with no accessible names.
  • Fall back when the list is longer than the image. If the viewport tree passes roughly 1,400 tokens, the screenshot is cheaper. Hacker News is the proof.
  • Never send the full tree by default. Fetch it only when the task is to read the whole page.

Caveats

  • Six pages, one viewport size, one run each. Treat the ratios as a guide, not a law.
  • Image tokens use Anthropic's estimate. Text tokens were counted with OpenAI's o200k_base tokenizer as a stand-in, so the two columns are close but not from the same tokenizer.
  • The viewport tree drops static text on purpose. It is for acting, not for reading.

If you are building agents that drive real software, the loop around this matters more than the view itself. I wrote about that here: the agent loop architecture we use in production. And if you want an agent like this built for your product, that is what we do at Axionry.

Sources

Top comments (1)

Collapse
 
zhang_jimmy_54a0414571400 profile image
zhang jimmy •

The Wikipedia contrast really stuck with me: a 578-token viewport tree against a 12,711-token full tree, while a single screenshot stays at 1,365 tokens regardless of content. That makes your default — viewport tree first, re-read after every action — feel like the right trade-off for acting, not just a cheaper one. I also liked the Hacker News exception, where the viewport tree at 1,516 tokens actually costs more than a screenshot. Have you tried a hybrid fallback that samples the tree first and only switches to pixels when the visible interactive count crosses a threshold, rather than waiting for the token estimate? That could catch dense pages earlier.