DEV Community

Mxhlix
Mxhlix

Posted on

Claude in Chrome keeps missing clicks. I measured why — there are exactly two causes.

If you use Claude in Chrome (Chrome MCP) for browser automation, you have probably hit some of these:

  • Clicks land next to the target, not on it
  • The pointer drifts right of the element, and retrying never fixes it
  • You take a screenshot, and moments later the page dimensions have changed
  • Double-clicks don't enter edit mode — you get a row selection instead

I run 600+ web tools built and QA'd through AI-driven browser automation, so I hit these constantly. I burned hours suspecting the page elements. They were fine.

The real problem: the assumptions behind coordinate-based clicking were breaking. It comes down to exactly two causes.

Cause 1: the window width settles late, so coordinates drift right

Right after a page loads, innerWidth can report a smaller value than the real window. On my machine it started at 1664. Even after document.readyState hits complete, it keeps widening for 2–3 seconds until it reaches the true width (1920). During that window, outerWidth / outerHeight sometimes return 0.

Now take a screenshot during that settling period and click using its coordinates. The page has widened since the capture, so your click lands to the right of the target. 1664→1920 is a 1.15× stretch — the further right the element, the bigger the miss.

That's the whole mystery: "the page changed size after my screenshot" and "the pointer keeps drifting" are the same bug.

This is not limited to exotic canvas apps. Ordinary HTML pages with lazy loading or dynamic layout do it too. Some of my "this input just won't click" sessions were nothing more than coordinates swinging at empty air.

How to diagnose it

Read the viewport with javascript_tool and check whether it has stabilized:

({ url: location.href, readyState: document.readyState,
   dpr: window.devicePixelRatio,
   innerW: window.innerWidth, innerH: window.innerHeight,
   outerW: window.outerWidth, outerH: window.outerHeight })
Enter fullscreen mode Exit fullscreen mode

Three rules. If outerW is 0, the window size hasn't settled — do not interact yet. Read again after a moment; if innerW changed, it's still settling — keep waiting. Two consecutive identical innerW readings with a non-zero outerW means you're clear.

Cause 2: double_click fires too slowly and becomes two single clicks

The computer tool's double_click action can leave a gap between the two clicks that exceeds the app's double-click threshold. The app interprets it as two single clicks — so instead of entering text-edit mode, you get a row selection or nothing.

The nasty part: the tool believes it sent a double-click. The logs look correct.

You cannot see this failure in any log.

Fixes

Plan A (preferred): stop using coordinates, use element references

On a normal HTML page this is the reliable path. Drop coordinate clicks entirely. Use find (natural-language element search) or read_page (accessibility tree) to get element references, then click and fill through those references (form_input for values). Element references survive layout shifts, so the settling problem simply doesn't apply.

The exception: canvas/Flutter apps (the Rive editor, for example). Their UI doesn't exist as HTML elements, so find and read_page return nothing. Then Plan B is your only option.

Plan B: if you must use coordinates, pair "wait for settle" with "rapid double-fire"

  1. Wait for the size to settle before doing anything — use the diagnostic snippet above until innerW stabilizes, then take your screenshot
  2. Don't let time pass between screenshot and click. Capture → click on those fresh coordinates, as one unit. If anything intervened, re-capture
  3. Never use double_click. Build your own with two rapid left_clicks at the same coordinates via browser_batch — the gap gets tight enough to pass the threshold:
browser_batch([
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
])
Enter fullscreen mode Exit fullscreen mode
  1. After every click, zoom into the target and verify the expected state change (selection, edit mode, highlight) before moving on. One unnoticed miss desyncs everything after it

Cheat sheet

Symptom First suspect
Clicks miss Size not settled (Cause 1) — wait
Double-click doesn't work Switch to rapid double-fire (Cause 2)
Stuck on a normal HTML page Switch to Plan A (element refs)
Canvas/Flutter app Plan B only — there are no elements

Limits and caveats

The numbers here (1664→1920, 2–3 seconds) are from my setup (Windows, 1920×1080). Display config and DPI scaling will change them. Take the principle, not the numbers: wait for settle before you capture.

Claude in Chrome ships updates frequently. If the tool ever starts waiting for size-settle itself, this whole workaround becomes unnecessary. And coordinate-based automation is inherently fragile — whenever element references are available, they win.


Verified: June 2026. Environment: Windows 11 / Chrome + Claude in Chrome extension / 1920×1080, including automation of the Rive editor (Flutter/canvas).

Top comments (0)