If you use Claude in Chrome (Chrome MCP) for browser automation, you have probably hit some of these:
- Clicks land next to the target, not on it
- The pointer drifts right of the element, and retrying never fixes it
- You take a screenshot, and moments later the page dimensions have changed
- Double-clicks don't enter edit mode — you get a row selection instead
I run 600+ web tools built and QA'd through AI-driven browser automation, so I hit these constantly. I burned hours suspecting the page elements. They were fine.
The real problem: the assumptions behind coordinate-based clicking were breaking. It comes down to exactly two causes.
Cause 1: the window width settles late, so coordinates drift right
Right after a page loads, innerWidth can report a smaller value than the real window. On my machine it started at 1664. Even after document.readyState hits complete, it keeps widening for 2–3 seconds until it reaches the true width (1920). During that window, outerWidth / outerHeight sometimes return 0.
Now take a screenshot during that settling period and click using its coordinates. The page has widened since the capture, so your click lands to the right of the target. 1664→1920 is a 1.15× stretch — the further right the element, the bigger the miss.
That's the whole mystery: "the page changed size after my screenshot" and "the pointer keeps drifting" are the same bug.
This is not limited to exotic canvas apps. Ordinary HTML pages with lazy loading or dynamic layout do it too. Some of my "this input just won't click" sessions were nothing more than coordinates swinging at empty air.
How to diagnose it
Read the viewport with javascript_tool and check whether it has stabilized:
({ url: location.href, readyState: document.readyState,
dpr: window.devicePixelRatio,
innerW: window.innerWidth, innerH: window.innerHeight,
outerW: window.outerWidth, outerH: window.outerHeight })
Four rules. If outerW is 0, the window size hasn't settled — do not interact yet. Read again after a moment; if innerW changed, it's still settling — keep waiting. Two consecutive identical innerW readings with a non-zero outerW and the size is clear. The fourth rule guards against a different failure entirely: dpr has to match whatever it was when you took your screenshot. Drag the window onto a monitor with different DPI scaling and every coordinate rescales — same math as the 1664→1920 stretch — except nothing ever looks unsettled, because the new values are correct and stable from the very first read. An innerW-only check waves it straight through. (Pointed out by @skillselion in the comments — see the update note below.)
Cause 2: double_click fires too slowly and becomes two single clicks
The computer tool's double_click action can leave a gap between the two clicks that exceeds the app's double-click threshold. The app interprets it as two single clicks — so instead of entering text-edit mode, you get a row selection or nothing.
The nasty part: the tool believes it sent a double-click. The logs look correct.
You cannot see this failure in any log.
Fixes
Plan A (preferred): stop using coordinates, use element references
On a normal HTML page this is the reliable path. Drop coordinate clicks entirely. Use find (natural-language element search) or read_page (accessibility tree) to get element references, then click and fill through those references (form_input for values). Element references survive layout shifts.
They're not a free pass, though. A ref is a pointer to one specific node and nothing more — it knows nothing about where that node sits on the page or what it currently means to the user. Replace the node in a rerender and the ref dies with an explicit error. Move the node, or let it take on a different role, and the ref keeps right on working: it reports success and clicks whatever that node has become, wherever it now lives. That second case is a silent miss with a clean log. Both reproduce in plain vanilla JS, so there's no framework magic involved — a React rerender just does the replacing for you. The fix is the same either way: treat refs as disposable. Grab one right before you use it, use it once, and never carry it across a navigation or a form submit (credit to @skillselion in the comments for the single-use rule). One more caveat I'd rather admit than bury: I've watched a perfectly fresh ref click no-op silently while the window was still settling, which tells me ref clicks get resolved to coordinates somewhere under the hood. Plan A shrinks the coordinate problem. It does not make it go away.
The exception: canvas/Flutter apps (the Rive editor, for example). Their UI doesn't exist as HTML elements, so find and read_page return nothing. Then Plan B is your only option.
Plan B: if you must use coordinates, pair "wait for settle" with "rapid double-fire"
- Wait for the size to settle before doing anything — use the diagnostic snippet above until
innerWstabilizes, then take your screenshot - Don't let time pass between screenshot and click. Capture → click on those fresh coordinates, as one unit. If anything intervened, re-capture
- Never use
double_click. Build your own with two rapidleft_clicks at the same coordinates viabrowser_batch— the gap gets tight enough to pass the threshold:
browser_batch([
{ name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
{ name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
])
- After every click,
zoominto the target and verify the expected state change (selection, edit mode, highlight) before moving on. One unnoticed miss desyncs everything after it
Cheat sheet
| Symptom | First suspect |
|---|---|
| Clicks miss | Size not settled (Cause 1) — wait |
| Double-click doesn't work | Switch to rapid double-fire (Cause 2) |
| Stuck on a normal HTML page | Switch to Plan A (element refs) |
| Ref click errors out or hits the wrong element | Stale ref — re-find immediately before the interaction (refs are single-use) |
| Canvas/Flutter app | Plan B only — there are no elements |
Limits and caveats
The numbers here (1664→1920, 2–3 seconds) are from my setup (Windows, 1920×1080). Display config and DPI scaling will change them. Take the principle, not the numbers: wait for settle before you capture.
Claude in Chrome ships updates frequently. If the tool ever starts waiting for size-settle itself, this whole workaround becomes unnecessary. And coordinate-based automation is inherently fragile — whenever element references are available, they win.
Update (2026-07-11)
@skillselion left a comment below that made this article better, and I've revised Plan A and the settle rules accordingly. I didn't take either point on faith — both got re-verified in plain vanilla JS before I touched the text. Refs really do track node identity and nothing else: kill the node and the ref dies loud, move it and the ref follows it silently into whatever it has become. And the innerW-only settle check really is blind to DPI changes — by design, since a rescaled-but-stable viewport is indistinguishable from a settled one if all you compare is innerW. I couldn't physically test the cross-monitor move (one display here), but my own setup argued the point for me: the same physical 1920px screen that settled at innerWidth 1920 in June now reports 1745 at a fractional dpr of 1.1. The full back-and-forth is in the comments.
Verified: June 2026; Plan A and settle conditions re-verified July 2026 (dpr 1.1 / 110% scaling environment). Environment: Windows 11 / Chrome + Claude in Chrome extension / 1920×1080, including automation of the Rive editor (Flutter/canvas).
Top comments (2)
Plan A has its own version of Cause 1 that is worth flagging: element references go stale on rerender. On a React app, a route-level state change can remount the whole subtree, and a ref captured before that mutation clicks the old node, or nothing, with the same silent-success logs as your double-click case. I ended up treating refs as single-use: find immediately before the interaction, never cached across a navigation or a form submit.
One addition for the diagnostic snippet: track devicePixelRatio against its value at screenshot time. Moving the window to a display with different DPI mid-session rescales coordinates exactly like your 1664 to 1920 stretch, but it never "settles" because the new value is correct - the screenshot is just from the other monitor's scale. Two identical innerW readings pass that check while every click still lands off-target.
Thanks for the comment. I have no React experience, so I can't speak to it directly — but the rescaling behavior you described tracked what I'd measured closely enough that I suspected the phenomenon wasn't framework-specific, so I tried to reproduce it in plain vanilla JS.
Both of your cases reproduced there. When the node gets replaced (replaceChild), the cached ref dies loud in Claude in Chrome — "No element found with reference" — and nothing gets clicked. When the node moves or changes role (I captured a ref to the top button of two, then swapped their order), the cached ref reports success and clicks the old node in its new position: genuine success log, wrong logical target. That's your silent-success case, with no React involved. So refs track node identity, not role — my "element references survive layout shifts" and your "clicks the old node" turn out to be the same property seen from opposite sides. And since this reproduces in the bare DOM, I suspect what you're seeing in React is this same mechanism — the rerender is just performing the replacement/move for you. Your single-use rule (find immediately before the interaction, never cache across a nav or submit) covers both cases; I'm adopting it.
On devicePixelRatio: conceded — that's a real hole in my settle check. It only compares consecutive innerW readings, so a DPI change passes as "settled" while every coordinate has rescaled. I couldn't reproduce the cross-monitor move (single display here), but I found supporting evidence closer to home: the machine I re-tested on today reports innerWidth 1745 at a fractional dpr of 1.1 — same physical 1920px display as the article, different CSS resolution — so identical-innerW checks don't even transfer across my own environment over time. I also caught one fresh, valid ref click silently no-opping while outerWidth was 0, which suggests ref-based clicks still resolve through coordinates somewhere. Element refs shrink the coordinate problem; they don't eliminate it. I'm adding "dpr unchanged since the screenshot" as a settle condition.
I'll fold both points into the article with credit. This is exactly the kind of comment I hoped the post would get.