If you use Claude in Chrome (Chrome MCP) for browser automation, you have probably hit some of these:
- Clicks land next to the target, not on it
- The pointer drifts right of the element, and retrying never fixes it
- You take a screenshot, and moments later the page dimensions have changed
- Double-clicks don't enter edit mode — you get a row selection instead
I run 600+ web tools built and QA'd through AI-driven browser automation, so I hit these constantly. I burned hours suspecting the page elements. They were fine.
The real problem: the assumptions behind coordinate-based clicking were breaking. It comes down to exactly two causes.
Cause 1: the window width settles late, so coordinates drift right
Right after a page loads, innerWidth can report a smaller value than the real window. On my machine it started at 1664. Even after document.readyState hits complete, it keeps widening for 2–3 seconds until it reaches the true width (1920). During that window, outerWidth / outerHeight sometimes return 0.
Now take a screenshot during that settling period and click using its coordinates. The page has widened since the capture, so your click lands to the right of the target. 1664→1920 is a 1.15× stretch — the further right the element, the bigger the miss.
That's the whole mystery: "the page changed size after my screenshot" and "the pointer keeps drifting" are the same bug.
This is not limited to exotic canvas apps. Ordinary HTML pages with lazy loading or dynamic layout do it too. Some of my "this input just won't click" sessions were nothing more than coordinates swinging at empty air.
How to diagnose it
Read the viewport with javascript_tool and check whether it has stabilized:
({ url: location.href, readyState: document.readyState,
dpr: window.devicePixelRatio,
innerW: window.innerWidth, innerH: window.innerHeight,
outerW: window.outerWidth, outerH: window.outerHeight })
Three rules. If outerW is 0, the window size hasn't settled — do not interact yet. Read again after a moment; if innerW changed, it's still settling — keep waiting. Two consecutive identical innerW readings with a non-zero outerW means you're clear.
Cause 2: double_click fires too slowly and becomes two single clicks
The computer tool's double_click action can leave a gap between the two clicks that exceeds the app's double-click threshold. The app interprets it as two single clicks — so instead of entering text-edit mode, you get a row selection or nothing.
The nasty part: the tool believes it sent a double-click. The logs look correct.
You cannot see this failure in any log.
Fixes
Plan A (preferred): stop using coordinates, use element references
On a normal HTML page this is the reliable path. Drop coordinate clicks entirely. Use find (natural-language element search) or read_page (accessibility tree) to get element references, then click and fill through those references (form_input for values). Element references survive layout shifts, so the settling problem simply doesn't apply.
The exception: canvas/Flutter apps (the Rive editor, for example). Their UI doesn't exist as HTML elements, so find and read_page return nothing. Then Plan B is your only option.
Plan B: if you must use coordinates, pair "wait for settle" with "rapid double-fire"
- Wait for the size to settle before doing anything — use the diagnostic snippet above until
innerWstabilizes, then take your screenshot - Don't let time pass between screenshot and click. Capture → click on those fresh coordinates, as one unit. If anything intervened, re-capture
- Never use
double_click. Build your own with two rapidleft_clicks at the same coordinates viabrowser_batch— the gap gets tight enough to pass the threshold:
browser_batch([
{ name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
{ name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
])
- After every click,
zoominto the target and verify the expected state change (selection, edit mode, highlight) before moving on. One unnoticed miss desyncs everything after it
Cheat sheet
| Symptom | First suspect |
|---|---|
| Clicks miss | Size not settled (Cause 1) — wait |
| Double-click doesn't work | Switch to rapid double-fire (Cause 2) |
| Stuck on a normal HTML page | Switch to Plan A (element refs) |
| Canvas/Flutter app | Plan B only — there are no elements |
Limits and caveats
The numbers here (1664→1920, 2–3 seconds) are from my setup (Windows, 1920×1080). Display config and DPI scaling will change them. Take the principle, not the numbers: wait for settle before you capture.
Claude in Chrome ships updates frequently. If the tool ever starts waiting for size-settle itself, this whole workaround becomes unnecessary. And coordinate-based automation is inherently fragile — whenever element references are available, they win.
Verified: June 2026. Environment: Windows 11 / Chrome + Claude in Chrome extension / 1920×1080, including automation of the Rive editor (Flutter/canvas).
Top comments (0)