Using proxies in Puppeteer is clearly a topic that many developers need. I’ve put together this post with small code examples for most common cases - static proxies, per-page routing, authentication, and rotation.
Table of Contents
- Introduction
- Static Proxy with
--proxy-server
- Page-Level Proxy / Request Interception
- Handling Proxy Authentication
- Proxy Rotation
- Troubleshooting
- Notes for Devs
Static Proxy with --proxy-server
Use a single proxy for all pages. This is simple and native.
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch({
args: ['--proxy-server=http://HOST:PORT']
});
const page = await browser.newPage();
await page.goto('https://httpbin.org/ip');
console.log(await page.evaluate(() => document.body.innerText));
await browser.close();
This will show the IP of your proxy instead of your local IP.
Page-Level Proxy / Request Interception
Puppeteer doesn’t support per-page proxies natively. You can use puppeteer-page-proxy
to route requests individually.
npm install puppeteer puppeteer-page-proxy
Example:
import puppeteer from 'puppeteer';
import useProxy from 'puppeteer-page-proxy';
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on('request', req => useProxy(req, 'http://user:password@host:port));
await page.goto('https://httpbin.org/ip');
await browser.close();
Each request goes through the proxy you specify.
Handling Proxy Authentication
This method of authentication won’t work:
const browser = await puppeteer.launch({
args: ['--proxy-server=http://host:port]
});
Use the previous example with puppeteer-page-proxy, or for HTTP/HTTPS proxies that require login, use page.authenticate()
.
const browser = await puppeteer.launch({
args: ['--proxy-server=http://host:port]
});
const page = await browser.newPage();
await page.authenticate({ username: 'USER', password: 'PASS' });
await page.goto('https://httpbin.org/ip');
console.log(await page.evaluate(() => document.body.innerText));
await browser.close();
Always authenticate before navigating to the page.
Proxy Rotation
Rotate proxies to avoid bans. Combine with random user-agents for better results.
const pool = [
'http://user:password@host1:port',
'http://user:password@host2:port',
'http://user:password@host3:port'
];
let i = 0;
function getNextProxy() {
return pool[i++ % pool.length];
}
const proxy = getNextProxy();
const browser = await puppeteer.launch({ args: [`--proxy-server=${proxy}`] });
const page = await browser.newPage();
await page.goto('https://httpbin.org/ip');
console.log(await page.evaluate(() => document.body.innerText));
await browser.close();
You can expand this with headers rotation or retry logic for blocked proxies. Here’s a link to a list of latest User Agents.
Troubleshooting
Most common errors:
-
407 Proxy Authentication Required
. Check username/password and callpage.authenticate()
. This can also happen if you used the wrong authentication method. -
ERR_PROXY_CONNECTION_FAILED
. Test host and port, try connecting with curl first. - 403/CAPTCHA. Rotate proxies, headers, or user-agent.
Notes for Devs:
Full Guide on How to Set Up Proxy in Puppeteer
More examples on Github
Join our Discord
If you want any examples I might have missed, leave a comment and I’ll add them.
Top comments (0)