DEV Community

Cover image for Your browser automation clicks are lying to you
Azan Hyder
Azan Hyder

Posted on

Your browser automation clicks are lying to you

#javascript #webdev #automation #tutorial

I spent an evening last month watching an automation script click a button that was definitely there. There was no error. The screenshot showed the cursor parked right on top of it. The script reported success and moved on. Nothing had happened. The page had not received a click at all, and every tool I was using to check said otherwise.

If you have written browser automation against a modern web app, you have hit this. Here is the fix up front (the send helper is defined in use case 1), then the full explanation of why the obvious approach silently does nothing.

// The fix: click through the browser's input pipeline
// (Chrome DevTools Protocol), not through JS.
await send("Input.dispatchMouseEvent", {
  type: "mousePressed", x, y, button: "left", clickCount: 1,
});
await send("Input.dispatchMouseEvent", {
  type: "mouseReleased", x, y, button: "left", clickCount: 1,
});
Enter fullscreen mode Exit fullscreen mode

That pair of events is indistinguishable from a physical mouse to the page. Everything below is about why the .click() you are probably sending instead is not.

What the old hack looks like

Every automation script starts like this:

document.querySelector("button.publish").click();
Enter fullscreen mode Exit fullscreen mode

It is in a thousand Stack Overflow answers and it works right up until it does not. The failure shows up on exactly the apps you most want to automate: editors, dashboards, anything with components listening on pointerdown or mousedown, controls gated on trusted input, or framework-managed form fields. Three separate things go wrong.

First, click() dispatches a synthetic event. Every synthetic event carries isTrusted: false, and the page can see that. Plenty of apps simply ignore untrusted events on sensitive controls.

Second, click() only fires the click event (form controls also run their activation behavior, but that is the exception, not the rule). Real input produces a sequence: pointerdown, mousedown, focus, pointerup, mouseup, then click. A growing number of components hang their behavior on the earlier events in that chain. Fire only the last one and the component never wakes up.

Third, and this is the nasty one, the failure is silent. There is no exception and no rejected promise. Your script reports success because nothing told it otherwise. The only honest signal is whether the DOM changed afterward, and almost nobody checks that.

Morpheus meme: what if I told you your click event never happened

Use case 1: clicking a button that ignores you

The setup is a raw WebSocket to the Chrome DevTools Protocol (CDP). Launch Chrome with --remote-debugging-port=10086, grab the page's webSocketDebuggerUrl from http://localhost:10086/json, and add a tiny helper:

// ws = new WebSocket(webSocketDebuggerUrl)
let id = 0;
const pending = new Map();

function send(method, params = {}) {
  const msg = { id: ++id, method, params };
  ws.send(JSON.stringify(msg));
  return new Promise((resolve, reject) => pending.set(msg.id, { resolve, reject }));
}

ws.onmessage = (e) => {
  const msg = JSON.parse(e.data);
  if (msg.id && pending.has(msg.id)) {
    const { resolve, reject } = pending.get(msg.id);
    pending.delete(msg.id);
    msg.error ? reject(msg.error) : resolve(msg.result);
  }
};
Enter fullscreen mode Exit fullscreen mode

Now the click. The one non-negotiable step is asserting the element actually has a size before you click its center. A display: none element returns a rect full of zeros, and clicking coordinates (0, 0) is how you click the wrong thing with total confidence. Note the behavior: "instant" on the scroll: without it, a page with smooth scrolling animates the scroll and you measure the rect mid-flight, then click stale coordinates.

async function clickForReal(selector) {
  const { result } = await send("Runtime.evaluate", {
    expression: `(() => {
      const el = document.querySelector(${JSON.stringify(selector)});
      if (!el) return null;
      el.scrollIntoView({ block: "center", behavior: "instant" });
      const r = el.getBoundingClientRect();
      return { x: r.x + r.width / 2, y: r.y + r.height / 2, w: r.width, h: r.height };
    })()`,
    returnByValue: true,
  });

  const rect = result.value;
  if (!rect || rect.w === 0 || rect.h === 0) {
    throw new Error(`not clickable: ${selector}`);
  }

  // A real mouse moves before it clicks; hover-dependent UI expects it too.
  await send("Input.dispatchMouseEvent", {
    type: "mouseMoved", x: rect.x, y: rect.y,
  });
  for (const type of ["mousePressed", "mouseReleased"]) {
    await send("Input.dispatchMouseEvent", {
      type, x: rect.x, y: rect.y, button: "left", clickCount: 1,
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

You need both mousePressed and mouseReleased. A release with no press is not a click. It is a shrug, and most apps treat it like one. One more caveat: this asserts the element has size, not that it receives the click. A modal overlay or cookie banner sitting on those coordinates will silently eat it. If overlays are in play, add a document.elementFromPoint(x, y) check before dispatching.

Use case 2: typing into a controlled input

The same trap exists for text. Setting value does nothing the framework can see:

// Looks right, does nothing. React never fires onChange for this,
// and may snap the value back on the next render.
document.querySelector("textarea").value = "hello";
Enter fullscreen mode Exit fullscreen mode

The classic workaround reaches past the framework to the native setter, then dispatches an input event so React notices:

const el = document.querySelector("textarea");
const setter = Object.getOwnPropertyDescriptor(
  window.HTMLTextAreaElement.prototype, "value"  // HTMLInputElement for <input>
).set;
setter.call(el, "hello");
el.dispatchEvent(new Event("input", { bubbles: true }));
Enter fullscreen mode Exit fullscreen mode

This works often enough to be worth knowing, but it is still a synthetic event with all the caveats from above. The real fix is the same pipeline as the click:

await send("Input.insertText", { text: "hello" });
Enter fullscreen mode Exit fullscreen mode

It is a single method call, it is trusted, and it respects whatever element has focus. Focus the field first with a real CDP click and the text lands the way typed text lands. One boundary to know: insertText fires no keydown or keyup. For Enter-to-submit, keyboard shortcuts, or editors with custom key handlers, you need Input.dispatchKeyEvent with rawKeyDown and keyUp instead.

Use case 3: verify state, not screenshots

Once the input path is honest, the remaining lie is your success check. "Take a screenshot and look at it" is the default in agent-style automation, and it fails in both directions: pixels can look right while nothing happened, and a render hiccup can make a success look broken.

The DOM always knows. Define success as a state transition before you act, then assert it after. Poll with a deadline instead of sleeping a fixed 800ms, and handle the case where a successful action removes the element entirely:

// Publishing from an editor: success is the composer clearing or unmounting.
async function waitFor(expression, timeoutMs = 5000) {
  const deadline = Date.now() + timeoutMs;
  while (Date.now() < deadline) {
    const { result } = await send("Runtime.evaluate", {
      expression, returnByValue: true,
    });
    if (result.value) return;
    await new Promise((r) => setTimeout(r, 150));
  }
  throw new Error("publish did not happen");
}

await clickForReal("button.publish");
await waitFor(`(() => {
  const ta = document.querySelector("textarea");
  return !ta || ta.value.length === 0;   // cleared or gone: both are success
})()`);
Enter fullscreen mode Exit fullscreen mode

Pick whatever transition your app actually guarantees: the composer empties, the row appears in the table, the URL changes. If you cannot name a DOM transition for an action, you do not yet know whether the action works.

The limits, honestly

Coordinates are a snapshot. The layout can shift between measuring the rect and dispatching the click, so measure immediately before clicking and never cache coordinates across actions. Coordinates are also frame-relative: for elements inside iframes you have to add each frame's offset on the way up. CDP is also Chromium territory. Firefox's equivalent is WebDriver BiDi, which is converging on the same model but is not a drop-in swap today.

And if you can install Playwright, install Playwright. It does all of this plus auto-waiting for visibility and stability, and its actionability checks are the checklist above written by people better at it than us. The raw approach earns its place when you are driving a browser that is already open, attached to a real user profile, or calling from a runtime where pulling in a driver is not an option.

The takeaway

Two greps worth running over any automation code you own:

grep -rn '\.click()' scripts/          # synthetic clicks, silent no-ops waiting
grep -rn 'screenshot' scripts/         # success checks that can lie in both directions
Enter fullscreen mode Exit fullscreen mode

Every hit is a place where your script can report success while the page disagrees. Trust the input pipeline and trust the DOM. Everything in between is a rumor.

If you have a favorite automation lie, the click that looked fine and never happened, I want to hear it. These failure modes are collectable.

Sources and further reading:

Top comments (0)