When a Site Blocks Your Scraper, Read the Browser Tab You Already Have Open

Originally published on hexisteme notes.

I needed live listing prices from a property portal for a personal buy-decision tool. The portal does everything a site does to stop a scraper: it disallows the paths in robots.txt, it fingerprints headless clients, it hides behind a login, and the prices don't even exist in the fetched HTML — they're painted in by JavaScript after the page loads. I spent an afternoon losing to all four defenses. Then I stopped fighting them, because the data was already sitting on my screen: I had the page open in Chrome.

For a personal tool, the move past robots.txt, bot detection, login walls, and JS-rendered single-page apps is not a better headless scraper — it's a ~40-line AppleScript bridge that reads the DOM out of the authenticated Chrome tab you already have open. You inherit the human's session and the fully-rendered content, and you add zero HTTP requests, so there is no scraper for the site to detect. It sidesteps all four walls at once because a human already solved the hard parts. It is also strictly a solo, read-only, human-scale technique — and that boundary is the point, not a limitation to route around.

The wall: the data is behind everything that stops a scraper

Every defense a site can raise was stacked on this one. A plain HTTP fetch of the listing URL got redirected to a login page. Sending cookies got past that, until the anti-bot layer noticed a client with no browser fingerprint and started serving challenges. And even a "successful" fetch was useless, because the response was an almost-empty HTML shell — the actual numbers are rendered client-side by the app's JavaScript, so the price I wanted simply wasn't in the bytes I'd downloaded. Each of these is solvable on its own with enough effort (headless browser, session replay, a rendering engine), and stacking all the solutions together is how a weekend project becomes an unmaintained pile of anti-detection hacks that breaks every time the site ships.

The tell that I was solving the wrong problem: I was reconstructing, badly, a thing that already existed and worked perfectly. A logged-in browser session that renders the page and shows me the price. I had one open in front of me.

The move: you already have the page open

The reframe is small and it changes everything. Don't fetch the page — read the page you already fetched by opening it. When the portal is open in a Chrome tab, that tab has already passed the login, already survived the bot check (it's a real browser doing real navigation), and already run the JavaScript that paints the prices into the DOM. The numbers are right there in document.body.innerText. The only thing missing is a way to reach into that tab from a script.

On macOS, that way is Apple Events. Chrome can evaluate JavaScript inside any open tab on request (a one-time setting, View → Developer → Allow JavaScript from Apple Events), and a JXA script — JavaScript for Automation, run via osascript — can drive it. The whole bridge is about forty lines: enumerate the windows and tabs, find the first tab whose URL contains a substring you pass in, and evaluate an expression inside it.

// read-tab.js  —  osascript -l JavaScript read-tab.js "new.land.naver.com" "document.body.innerText"
function run(argv) {
  const [pattern, jsExpr = "document.body.innerText"] = argv;
  const Chrome = Application("Google Chrome");
  let tab = null;
  for (const w of Chrome.windows())
    for (const t of w.tabs())
      if (t.url().indexOf(pattern) !== -1) { tab = t; break; }
  if (!tab) throw new Error("no open tab matching: " + pattern);

  // Evaluate inside the live, authenticated, already-rendered tab.
  const wrapped =
    "(function(){try{const v=(" + jsExpr + ");" +
    "return typeof v===\"string\"?v:JSON.stringify(v);}" +
    "catch(e){return \"JS_ERROR: \"+(e&&e.message||e);}})()";
  return tab.execute({ javascript: wrapped });
}

That's the entire mechanism. Point it at a URL substring, hand it any expression, and it returns the result as text. Pass document.body.innerText for the rendered page, a specific querySelectorAll(...) to pull just the fields you want, or document.documentElement.outerHTML to get the post-render HTML for a downstream parser. From a Python collector it's one subprocess call; the deterministic pipeline downstream never knows the data came from a browser tab instead of an API.

Why it sidesteps every wall at once

The reason this feels like cheating is that it doesn't defeat any of the defenses — it makes all of them irrelevant, because the human already dealt with each one before the script ever runs.

Wall	Why it's already gone
Login / session	You read the tab under the user's authenticated session — no credentials in your code, nothing to store or refresh.
JS-rendered SPA	You read the DOM after the app's JavaScript ran, so you see the painted numbers, not the empty shell a fetch returns.
Bot detection	You issue no HTTP request at all. There is no headless client to fingerprint; the traffic already happened, made by a real browser a human drove.
robots.txt / rate limits	You aren't crawling. You read one page the user chose to open, at the rate a person clicks — the politest possible traffic profile.

Every one of those problems is hard to solve in a scraper and free to inherit from a browser. That's the whole trade: you give up automation-at-scale and get correctness-without-a-fight.

The honest edges

This is a sharp tool with a narrow blade, and pretending otherwise is how you get burned.

• Reading is reliable; writing is a different, harder problem. It's tempting to also fill search boxes or click through pagination the same way. Don't assume it'll work: Apple Events JavaScript runs in an isolated world, and modern frameworks override the native value setter, so a plain el.value = "..." is silently ignored and no input/change event fires — the app never sees your input. Writing has to go through the native value setter plus manually dispatched events, and it's fragile enough that I keep the reader strictly read-only and treat interaction as a separate concern.
• It is single-user, single-machine, by design. This works because a person has the page open on their desktop. It does not run in the cloud, does not scale, and is not a commercial scraper. If you need server-side or high-volume collection, this isn't your tool — and you should be honoring the site's terms and rate limits, not looking for a bypass.
• It's coupled to the browser being open and permissioned. The one-time Apple Events permission, the specific browser, the tab actually being open — these are real dependencies. The script fails loudly ("no open tab matching…") when they're not met, which is the right failure: it tells you to go open the page, not silently return stale or empty data.

## When to reach for it

The fit is narrow and clear: a personal, low-volume tool that needs data from a site that fights scrapers, where you are already an authenticated human looking at the page. Live prices for your own buy decision, a number off a dashboard you have access to, a value from a portal you log into anyway. In that slot it's unbeatable, because you're not extracting anything you couldn't already see — you're just saving yourself the copy-paste. Outside that slot — scale, headless, other people's data, evading a wall you're not entitled to be past — it's the wrong tool, and the boundary isn't a limitation to engineer around. It's what keeps the technique honest: it only ever reads a page a person chose to open.

---

More notes at hexisteme.github.io/notes.