Puppeteer Behind a Proxy: A Practical Setup

#puppeteer #node #webscraping #automation

Puppeteer drives a real Chromium, which is exactly why people use it for scraping and testing sites that a plain HTTP client cannot handle. It is also why one address gets you blocked so fast: a headless browser loads every asset on the page, so a single run generates a burst of requests that looks nothing like casual browsing. Routing Puppeteer through a rotating proxy is what keeps it usable at scale. Here is a practical setup.

The basic proxy launch

Puppeteer takes the proxy as a Chromium launch argument. Every request the browser makes then goes out through it.

const puppeteer = require("puppeteer");

const browser = await puppeteer.launch({
  args: ["--proxy-server=http://proxy.host:8080"],
  headless: "new",
});
const page = await browser.newPage();
await page.goto("https://example.com", { waitUntil: "domcontentloaded" });

Note that --proxy-server does not carry credentials. If your proxy needs a username and password, the launch arg alone gives you a 407, and you handle auth separately.

Handling authenticated proxies

For a proxy that requires credentials, call page.authenticate before you navigate. Chromium then answers the proxy's auth challenge for you.

const page = await browser.newPage();
await page.authenticate({ username: "user", password: "pass" });
await page.goto("https://example.com");

This is the step most people miss, and it is why "the proxy works in curl but not in Puppeteer" is such a common complaint. The browser needs the credentials handed to it explicitly.

Rotation without restarting the browser

Launch args are fixed for the life of a browser process, so if you rotate by changing the launch arg you have to restart Chromium for every address, which is slow. The better pattern with a rotating provider is to point the launch arg at one rotating endpoint and let the pool change the exit per request underneath. For cases where you want a fresh address per identity, launch a separate browser context or process per profile, each pinned to the endpoint.

One rotating endpoint, many pages. Let the pool cycle exits while the browser stays up.
One context per identity. When each session must look like a separate person, isolate cookies and storage per context and keep it on its own exit.
Pace the navigation. A browser is heavy. Do not open fifty pages at once against one target, or rotation will not save you.

What the pool needs to handle a browser

A headless browser is demanding on the address behind it, so the pool matters more here than with a plain HTTP client.

WinGate fits this well. It provides private proxies with SOCKS5 and standard user and password auth, so page.authenticate has real credentials to present, and the addresses are private rather than shared, so you are not inheriting someone else's flags. The rotating pool cycles exits automatically from a worldmix, traffic is unlimited so asset-heavy page loads do not meter you, and it handles up to 5000 threads for many parallel contexts. It speaks HTTP, HTTPS, and SOCKS5, so the launch arg and authenticate call above work unchanged.

An honest note: a proxy stops the address from being the reason Puppeteer gets blocked, it does not hide the fact that you are automating a browser. Fingerprinting, pacing, and respecting the site's terms still matter. There is a free 2 hour test, so wire the launch arg and authenticate call into a small run and watch how many pages load clean before you scale contexts up.