A scraper that works locally can still fail in production with a 403, a CAPTCHA loop, or a blank page after login. The proxy is fine. The headers look normal. navigator.webdriver is patched. The problem is often simpler: the browser fingerprint does not describe one believable device.
Modern bot systems do not rely on one signal. They combine canvas rendering, WebGL strings, timezone, locale, screen size, fonts, TLS handshake shape, IP geolocation, request timing, cookies, and storage state. Any one value can look harmless by itself. The mismatch is what gives you away.
Do not randomize everything
A common first attempt is to randomize every visible browser property on every request. That usually makes detection easier.
A real user does not switch from a UK timezone to a US locale to an Indian IP across three page loads in the same session. A real Chrome browser does not send Firefox-like TLS handshakes while claiming a Chrome user agent. A real GPU does not produce a new canvas fingerprint every time toDataURL() runs on the same page.
The goal is not maximum entropy. The goal is a coherent profile.
For one browser session, these values should agree:
- Proxy country
- Browser timezone
navigator.language- Accept-Language header
- User agent
- Screen size
- WebGL vendor and renderer
- TLS fingerprint
- Cookies and localStorage
If you rotate, rotate the whole profile between sessions, not individual fields inside a session.
Patch the obvious automation flags, but do not stop there
Stealth plugins help with basic checks. For Puppeteer, that usually means puppeteer-extra-plugin-stealth:
npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
This patches common signals like navigator.webdriver, missing Chrome runtime objects, and some plugin inconsistencies. That is useful, but it is not a full fingerprint strategy.
The edge case shows up when the target validates signals together. You might hide navigator.webdriver, but still send:
- A datacenter IP in Virginia
-
Asia/Kolkataas the browser timezone -
en-GBas the locale - A user agent for Chrome on Windows
- A TLS fingerprint that does not match Chrome
That request does not look like a privacy-conscious user. It looks assembled.
Validate geo alignment before you scrape
You can catch a lot of failures before hitting the target page. Run a small check inside the browser context and compare it with the proxy you selected.
async function validateBrowserGeo(page, expected) {
return page.evaluate((expected) => {
const actual = {
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
language: navigator.language,
languages: navigator.languages,
platform: navigator.platform
};
const errors = [];
if (actual.timezone !== expected.timezone) {
errors.push(`timezone=${actual.timezone}, expected=${expected.timezone}`);
}
if (!actual.language.startsWith(expected.languagePrefix)) {
errors.push(`language=${actual.language}, expected prefix=${expected.languagePrefix}`);
}
return {
ok: errors.length === 0,
actual,
errors
};
}, expected);
}
const result = await validateBrowserGeo(page, {
timezone: 'Europe/London',
languagePrefix: 'en-GB'
});
if (!result.ok) {
throw new Error(`Fingerprint geo mismatch: ${result.errors.join('; ')}`);
}
A failure here is better than a mysterious 403 later:
Fingerprint geo mismatch: timezone=America/New_York, expected=Europe/London; language=en-US, expected prefix=en-GB
If your workflow needs browser execution, proxy country selection, and session state to stay aligned across many requests, Wire treats those as parts of the same extraction profile rather than separate settings you have to keep in sync yourself.
Canvas and WebGL spoofing need stable noise
Canvas fingerprinting hashes pixel-level rendering differences. WebGL exposes values like the unmasked renderer and vendor. Spoofing these can help, but unstable spoofing creates a different problem.
This kind of patch injects small canvas noise:
await page.addInitScript(() => {
const original = HTMLCanvasElement.prototype.toDataURL;
HTMLCanvasElement.prototype.toDataURL = function (...args) {
const ctx = this.getContext('2d');
if (ctx && this.width > 0 && this.height > 0) {
const image = ctx.getImageData(0, 0, this.width, this.height);
for (let i = 0; i < image.data.length; i += 4) {
image.data[i] += 1; // red
image.data[i + 1] += 0; // green
image.data[i + 2] -= 1; // blue
}
ctx.putImageData(image, 0, 0);
}
return original.apply(this, args);
};
});
The important part is not the exact noise. The important part is stability. If the same session produces a different canvas hash on each call, the noise itself becomes a fingerprint.
A better approach is to generate deterministic noise from a session seed and keep it stable for that site and session. Rotate it only when you rotate the full browser profile.
TLS fingerprints are outside JavaScript
JavaScript patches cannot fix TLS fingerprints. JA3 and JA4-style fingerprints come from cipher suite ordering and TLS extensions during the handshake. The server can inspect them before your HTTP headers arrive.
This matters when you use browser automation through a proxy or custom HTTP client. If your headers claim Chrome but your TLS handshake looks like a generic Node.js client, some systems will block before your page script runs.
You usually do not want to invent TLS fingerprints. Use a browser or client that impersonates real browser handshakes, then keep the user agent and browser version consistent with that choice.
Session state is part of the fingerprint
Authenticated scraping adds another layer. Login flows create cookies, localStorage entries, CSRF tokens, feature flags, and sometimes device identifiers. If you log in on every request, clear storage between pages, or reuse cookies with a different fingerprint, you create patterns that normal browsing does not produce.
A practical session flow looks more like this:
- Open the home page.
- Navigate to login.
- Submit credentials.
- Visit one or two normal intermediate pages.
- Store cookies and localStorage.
- Reuse the same browser profile for later target pages.
When extraction requires warmed sessions, Wire can keep browser-backed jobs tied to persisted session state so retries do not accidentally look like new devices.
What to test before calling it fixed
Run the same workflow at least 10 times and record the fingerprint values you care about:
- IP country
- Timezone
- Locale
- User agent
- WebGL vendor and renderer
- Canvas hash
- TLS profile, if your tooling exposes it
- Status code
- CAPTCHA presence
- Redirects to bot-check pages
You are looking for two things: consistency inside a session and believable variation between sessions.
Also be honest about the boundary. Fingerprint evasion can violate a site's terms of service, and authenticated scraping has different legal and contractual risks than collecting public pages. Check that before you build around it.
A good next step is to add a fingerprint validation script to CI or to your scraping job startup path. Fail fast when timezone, locale, proxy country, or storage state drift instead of debugging another unexplained 403 in production.
Top comments (0)