Running Puppeteer and Playwright Behind a Proxy

You wrote a scraper, it worked great on your laptop, and then you deployed it. Within an hour every request is a CAPTCHA wall, a 403, or an infinite "checking your browser" spinner. Nothing changed in your code. What changed is the IP. A headless browser firing hundreds of requests from a single server address is one of the easiest things in the world to fingerprint. Rate limits trip, reputation scores drop, and the target site starts treating you like a bot because, from its side of the wire, you look exactly like one.

Why one IP gets you blocked

Real users are spread across thousands of residential and mobile addresses. Your scraper is one datacenter IP hammering the same endpoints on a tight loop. Sites detect this in a few ways: request volume per address, known datacenter ranges, and the absence of the organic browsing patterns a normal visitor produces. Once that single IP is flagged, every context, session, and browser instance sharing it inherits the block. The fix is not a smarter user agent string. It is spreading traffic across many clean addresses so no single one crosses a threshold.

Wiring a proxy into Puppeteer

Puppeteer takes the proxy at launch time through a Chromium flag. If the proxy needs credentials, you supply them per page with authenticate before navigating.

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: ['--proxy-server=http://proxy.host:8080'],
  });
  const page = await browser.newPage();
  await page.authenticate({ username: 'user', password: 'pass' });

  await page.goto('https://example.com');
  console.log(await page.title());
  await browser.close();
})();

The --proxy-server flag applies to the whole browser process, so every page in that instance routes through the same address. To rotate, launch separate browsers (or restart with a fresh proxy) per batch of work.

Per-context proxies in Playwright

Playwright is more flexible. You can set a proxy globally at launch, or scope a different one to each browser context, which lets a single process run many isolated sessions on many addresses at once.

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch();
  const context = await browser.newContext({
    proxy: {
      server: 'http://proxy.host:8080',
      username: 'user',
      password: 'pass',
    },
  });
  const page = await context.newPage();
  await page.goto('https://example.com');
  console.log(await page.title());
  await browser.close();
})();

Because the proxy lives on the context, you can spin up several contexts, each with its own address, and run them in parallel. That is the pattern you want for scale: many lightweight sessions, each looking like a separate visitor, instead of one loud stream from one IP. SOCKS5 works the same way here, just swap the scheme in the server field.

Where to get proxies that survive detection

Free proxy lists are dead on arrival. They are shared, already blocklisted, and often log everything you send. For anything serious you want private addresses that rotate, so each session presents a clean IP the target has not seen before.

That is the case for a service like WinGate proxies for headless browsers, which offers private IPv4 and SOCKS5 with rotation, a worldmix pool, and unlimited traffic. It supports HTTP, HTTPS, and SOCKS5, handles up to 5,000 threads for large parallel jobs, and drops straight into both the Puppeteer and Playwright snippets above without any code changes beyond the connection string.

The practical move is to test before you commit. WinGate offers a free 2-hour trial so you can point your existing automation at it, watch the CAPTCHA rate fall, and confirm the rotation behaves the way your workload needs. Wire it in, run your real scraper against your real targets, and see whether the blocks go away before you spend anything.