We run a lot of clickops through Claude Code: reading dashboards, checking settings, pulling a number out of a web app. There are two ways we do it. claude-in-chrome drives Chrome and can return screenshots. opencli drives the same logged-in Chrome but returns the page as text.
We wanted to know what each one costs in tokens, so we measured it.
What we measured
We went through our own Claude Code session logs: 7,500+ browser tool calls. For each call, we took the token counts the API reported before and after it. The difference is how many tokens that call's result added to the context.
These are real API counts, not character-based estimates.
Screenshots are the bill
| claude-in-chrome call type | Share of calls | Median tokens per call |
|---|---|---|
| Returned a screenshot | 23% | 1,855 |
| Returned text only | 77% | 373 |
A screenshot call cost about 5x a text call. The 23% of calls that returned screenshots made up 49% of all claude-in-chrome tokens.
An opencli call had a median of 397 tokens. That is about the same as a claude-in-chrome text call. The difference between the two tools comes from screenshots, not from the text.
Same sites, both tools
Nine websites had at least 10 calls with each tool. Here is the change in median tokens per call when we used opencli instead of claude-in-chrome:
| Site | claude-in-chrome | opencli | Change |
|---|---|---|---|
| telegram.org | 1,879 | 275 | −85% |
| github.com | 580 | 336 | −42% |
| cloudflare.com | 578 | 378 | −35% |
| google.com | 605 | 437 | −28% |
| linkedin.com | 736 | 594 | −19% |
| youtube.com | 542 | 470 | −13% |
| lokerdollar.com | 432 | 386 | −11% |
| nvidia.com | 363 | 384 | +6% |
| bing.com | 270 | 353 | +31% |
opencli won on 7 of the 9 sites, about 27% fewer tokens per call when weighted by call count. The big wins were logged-in web apps like Telegram Web, GitHub and the Cloudflare dashboard. On simple public pages the two tools were even, and on Bing opencli used more. The nvidia.com and bing.com rows have under 30 calls each, so take them loosely.
Our clickops rule now
- Use the API or curl when one exists.
- Read the page as text with opencli.
- Take a screenshot only to check something visual.
Limits of this measurement
The site comparison matches calls by website, not by identical task. We did not run the same task twice with each tool, so this is a field measurement, not a lab A/B. Screenshot results also stay in context and get re-read on later turns. We left that out because session length differed between the tools, so the real cost of screenshots is likely higher than shown here.
Top comments (4)
Useful numbers, thanks for publishing the real API counts instead of character estimates. The 23% of calls making up 49% of tokens matches what we see.
One thing your "re-read on later turns" caveat hints at: for clickops you repeat (the same dashboard number every morning), the per-call cost matters less than whether the model is in the loop at all. What has worked for us is running it once with the model, keeping the steps it took, and replaying them deterministically next time, with the model only called back in when a step fails because the UI changed. A text-vs-screenshot win then applies to the first run and the repairs, not every run.
Did you see a split between one-off lookups and recurring checks in those 7,500 calls? Curious how much of the total is repeat work.
(Disclosure: I'm on the Auten team, we build that replay approach into a computer-use MCP server.)
Thanks, good question. I ran a rough split on the same logs. I counted a call as repeat work when the same page (URL with IDs and query strings removed) showed up in 3 or more separate sessions.
By that measure, about 26% of the 7,500 calls and 25% of the tokens were repeat work. The other three quarters were one-off lookups. It's a proxy, since the same page can serve different questions.
The repeat pages are mostly dashboards: Cloudflare build pages, Gmail and Search Console. Our scheduled production checks mostly run as cron jobs and API probes, not in a browser, so they don't appear in these numbers.
We're also running a same-task test now, with each tool doing identical tasks, and will post the results here.
Thanks for running the split, that is a cleaner proxy than I expected anyone to bother with.
A quarter of tokens on repeat dashboard visits fits what we see. Those are also the easiest case to take the model out of: the navigation is identical every time, and the part that changes is only the number you read at the end, which is a text read, not a judgment call. The model is only worth calling when that number looks off or the page layout moved.
Looking forward to the same-task results, especially whether the text tool still wins on pages with charts.
The same-task results are up as their own post: dev.to/vousmeevoyez/same-task-both...
Short version: five read-only tasks, three runs per tool, a fresh headless session per run with only one browser tool allowed. opencli cost 62% less and used 42% fewer tokens, at about the same accuracy. The gap was only 9% on a text-heavy GitHub page and 48–62% on the other four.
On charts: we didn't test a chart page, so I can't answer that yet. My guess is a text read does fine when the chart's numbers are in the page's HTML and badly when they're only drawn on a canvas. That's the next task we'll add.