DEV Community

lamingsrb
lamingsrb

Posted on Originally published at bizflowai.io

I/O 2026 Broke 11 Of My 37 Scrapers On A Tuesday

I/O 2026 Broke 11 Of My 37 Scrapers On A Tuesday

On November 4th, eleven of my thirty-seven headless Chrome scrapers started returning null. No errors, no stack traces — just clean nulls piped into a Telegram channel I wasn't watching closely enough. If you run any agent that touches a real browser in production, this hit you too. You probably haven't noticed yet.

Here's the exact DOM diff, the selector that died, and the forty-line patch that brought them back.

What Google shipped at I/O 2026 that nobody read carefully

Google I/O 2026 pushed three Chrome updates aimed at developers. Every recap covered them the same way — one-paragraph summaries, no operator angle:

  • Modern Web Guidance — Gemini writes your CSS.
  • DevTools for agents — Claude and other agents can drive a browser through a structured protocol.
  • AI assistance in DevTools — a Gemini sidebar inside the inspector so a junior dev can ask why their flexbox collapsed.

The third one is where the damage lives. AI assistance in DevTools ships with a runtime attribute layer. Chrome now injects data-devtools-* attributes onto DOM nodes so the Gemini sidebar can annotate them. On paper, harmless — attributes are read-only decorations, right?

Wrong. In shipped Chrome, some of those annotations arrive as sibling nodes, not as attributes on the target element. Which means every nth-child selector written before November 4th is now pointing one index off. Roughly ninety percent of the scraper tutorials on YouTube use nth-child. Ninety percent of the scrapers you inherited from a freelancer use nth-child. Mine did too.

The exact failure: six hours of silent nulls

The stack: home server, WSL Ubuntu, headless Chrome behind Puppeteer, cron every four hours, thirty-seven targets for price monitoring. Chrome auto-updated inside the container on November 4th around the time I was asleep. Eleven scrapers started returning null on the price field on the very next run.

The alerting fired. But nulls happen — a target is slow, a CDN throws a 503, a page ships an A/B variant. My channel gets 2-3 false empties a week. I logged the alerts as noise. It took six hours of stale data before I actually opened the diff.

The selector was:

await page.$eval(
  'div.product-info > div:nth-child(4)',
  el => el.textContent.trim()
);
Enter fullscreen mode Exit fullscreen mode

Post-update, the real price div had moved to nth-child(5). Chrome had injected a sibling annotation node ahead of it — something like:

<div class="product-info">
  <div data-devtools-annotation="price-region"></div>  <!-- NEW -->
  <div class="label">Price</div>
  <div class="currency">USD</div>
  <div class="tax-note">incl. tax</div>
  <div class="price">$248.00</div>  <!-- was nth-child(4), now nth-child(5) -->
</div>
Enter fullscreen mode Exit fullscreen mode

Puppeteer's query returned the tax-note div. That element has no text on this template. trim() returned "", my downstream handler cast empty to null, and the pipeline shrugged.

The lesson before we get to the fix: an "empty" scrape and a "wrong selector" scrape look identical downstream. If you don't distinguish them at the source, you get quiet corruption instead of loud failure.

The 40-line patch: strip, pin, canary

The fix is three things, none of them clever. Together they took about ninety minutes and haven't broken since.

1. Strip DevTools attributes before every query. Walk the subtree, drop any attribute starting with data-devtools. If Chrome ships more attribute variants next release, extend the prefix list.

// puppeteer-hooks/strip-devtools.js
async function stripDevtoolsAttrs(page, rootSelector = 'body') {
  await page.evaluate((sel) => {
    const root = document.querySelector(sel);
    if (!root) return;
    const walker = document.createTreeWalker(root, NodeFilter.SHOW_ELEMENT);
    let node = walker.currentNode;
    while (node) {
      // Remove injected annotation nodes entirely
      if (node.hasAttribute && node.hasAttribute('data-devtools-annotation')) {
        const next = walker.nextSibling();
        node.remove();
        node = next;
        continue;
      }
      // Strip decorative attrs on real nodes
      if (node.attributes) {
        [...node.attributes]
          .filter(a => a.name.startsWith('data-devtools'))
          .forEach(a => node.removeAttribute(a.name));
      }
      node = walker.nextNode();
    }
  }, rootSelector);
}

module.exports = { stripDevtoolsAttrs };
Enter fullscreen mode Exit fullscreen mode

Wire it into your scrape function so it runs after page.goto and before any $eval:

await page.goto(url, { waitUntil: 'networkidle2' });
await stripDevtoolsAttrs(page, 'div.product-info');
const price = await page.$eval('div.product-info > div.price', el => el.textContent.trim());
Enter fullscreen mode Exit fullscreen mode

While you're in there, switch off nth-child for anything you care about. Class selectors, data-* attributes on the target site, or :has() are all more stable. nth-child is a positional selector against a DOM you don't control — it's borrowed time.

2. Pin Chrome in your Dockerfile. Auto-update in a container is the actual root cause. Puppeteer bundles its own Chromium — use it and freeze the Puppeteer version, or install a specific Chrome build and disable updates.

# Dockerfile
FROM node:20-slim

# Pin puppeteer to a version whose bundled Chromium you have tested
RUN npm install puppeteer@23.9.0

# If you install Chrome directly, pin the exact build
ARG CHROME_VERSION=130.0.6723.116-1
RUN apt-get update && apt-get install -y \
    google-chrome-stable=${CHROME_VERSION} \
 && apt-mark hold google-chrome-stable
Enter fullscreen mode Exit fullscreen mode

Chrome versions are now something you diff in a pull request, not something the container decides for you at 3am.

3. One canary selector test per target, in CI. Pick a stable public page for each scraper. Assert a known value. Fail the build on drift.

// tests/canary.spec.js
const targets = require('../config/targets.json');

for (const t of targets) {
  test(`canary: ${t.name}`, async () => {
    const page = await browser.newPage();
    await page.goto(t.canaryUrl);
    await stripDevtoolsAttrs(page, t.rootSelector);
    const value = await page.$eval(t.priceSelector, el => el.textContent.trim());
    expect(value).toMatch(t.canaryPattern);   // e.g. /^\$\d/
  });
}
Enter fullscreen mode Exit fullscreen mode

That single test suite would have caught November 4th in the pipeline instead of in production. Total cost: about ninety seconds per CI run.

The three practices, at a glance

  • Strip DevTools attributes in a pre-query hook — defensive against future Google injections.
  • Pin the browser version in the Dockerfile — no silent updates.
  • Canary one selector per target in CI — fail loud on drift.

Treat Chrome as a dependency, not as furniture

This isn't really a story about eleven scrapers. It's about the mental model.

If Chrome is a tool your agent calls, Chrome is a dependency. Not infrastructure, not "the environment," not something the OS just provides. A dependency. You pin it, diff it, canary it — same discipline you apply to a Python package or a model version.

The teams I work with that came through November 4th cleanly all had the same three things: version-pinned browser, attribute-stripping pre-query hook, one canary per target. That's it. No exotic observability platform, no vendor lock-in, no $40k/yr contract.

Here's a rough operator scorecard I now run through with every client before we ship a browser-driven agent:

Concern Furniture mindset Dependency mindset
Browser version Whatever the container pulls Pinned in Dockerfile, held with apt-mark
Selector strategy nth-child copy-pasted from Stack Overflow Class / attribute / :has(), DOM-stripped before query
Failure signal Null in the output = "site was slow" Distinguish empty, selector_miss, network_error
Update cadence Auto PR + diff + canary run
CI coverage Unit tests only Canary against a real public page per target

The reason I/O 2026 hurt so many agent operators is exactly this gap. Everyone treated the browser as furniture. It updates in the background, it's fine, Google handles it. That model was already wrong. This update just made it expensive.

Why Google will keep breaking your agents

Hot take. AI assistance in DevTools is a net negative for anyone building autonomous agents, and Google knows it. They shipped it anyway because the audience at I/O is humans writing code with Gemini, not agents running Chrome as a tool.

That gap — between what Google optimizes for and what agent operators need — is going to widen every release. Chrome's product surface is now competing with itself: help human devs debug faster (inject helpful annotations) vs. keep the DOM predictable for programmatic clients (don't touch the tree). Guess which one wins in the keynote demo.

Plan for it. Assume every major Chrome release ships something that will bite a browser-driven agent. Budget one afternoon per release cycle to run your canaries against the new build in a sandbox before you promote the pinned version. That's the deal now.

Also worth reading directly, not through recaps: the Chrome release notes and Puppeteer's supported Chromium matrix. Those two pages tell you more about your production risk than any I/O keynote will.

Where bizflowai.io fits in

For clients running lead-gen, price monitoring, or competitive-intel pipelines, bizflowai.io ships the browser layer with these three practices baked in — pinned Chromium, a pre-query DOM hook, and per-target canaries wired into deploys. It's the boring part of a browser agent stack, but it's the part that turns a Tuesday morning Chrome update from a refund conversation into a non-event.

The 5-minute audit for your own stack

Before you close this tab, actually check:

  1. Grep your scrapers for nth-child. Anything you find is a ticking timer.
  2. docker exec into your container and run google-chrome --version. If you can't tell me the exact build number that shipped last deploy, you don't have a pinned browser.
  3. Look at your last 30 days of scraper alerts. How many "empty result" alerts did you dismiss? Any of them stack in a suspicious pattern around November 4th?
  4. Write one canary test today for your most valuable target. Just one. Ship it to CI. You'll add the others when the first one saves you.

Six hours of stale data on my server was annoying. Six hours on a client's pipeline is a refund conversation. Ninety minutes of work prevents both.


Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.

Top comments (0)