Playwright is the standard tool for browser automation in 2026. It handles Chromium, Firefox, and WebKit from a single API, supports async/await natively, and integrates cleanly with modern Python and Node.js workflows. But without proxies, Playwright automation hits rate limits and blocks quickly on any platform with bot detection.
This guide covers every aspect of integrating NodeMaven residential proxies with Playwright: proxy configuration for all three browser engines, authentication handling, rotating and sticky session patterns, and a CAPTCHA detection strategy. All code is production-ready and tested against real-world targets.
Prerequisites
Install Playwright and its browser binaries:
pip install playwright
playwright install
You will need NodeMaven proxy credentials from your dashboard. Copy them in {host}:{port}:{username}:{password} format. NodeMaven supports both HTTP and SOCKS5 protocols. Use port 8080 for HTTP and port 1080 for SOCKS5.
Basic Proxy Setup: Chromium
Playwright accepts proxy configuration at the browser level via the proxy parameter in launch(). The server field takes the full proxy URL including protocol.
import asyncio
from playwright.async_api import async_playwright
PROXY_HOST = "gate.nodemaven.com"
PROXY_PORT = "8080"
PROXY_USER = "your_nodemaven_username"
PROXY_PASS = "your_nodemaven_password"
async def basic_chromium():
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
}
)
page = await browser.new_page()
await page.goto("https://httpbin.org/ip")
content = await page.content()
print(content)
await browser.close()
asyncio.run(basic_chromium())
The proxy applies to all requests made by the browser instance. Every page opened under this browser uses the same proxy configuration.
Proxy Setup: Firefox and WebKit
The proxy configuration syntax is identical across all three engines. Swap p.chromium for p.firefox or p.webkit:
async def multi_engine_example():
async with async_playwright() as p:
proxy_config = {
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
}
browser_cr = await p.chromium.launch(proxy=proxy_config)
browser_ff = await p.firefox.launch(proxy=proxy_config)
browser_wk = await p.webkit.launch(proxy=proxy_config)
for browser, name in [
(browser_cr, "Chromium"),
(browser_ff, "Firefox"),
(browser_wk, "WebKit"),
]:
page = await browser.new_page()
await page.goto("https://httpbin.org/ip")
print(f"{name}: {await page.inner_text('body')}")
await browser.close()
asyncio.run(multi_engine_example())
For scraping workflows targeting sites that fingerprint browser engine type, use Chromium. For sites that explicitly check for non-Chrome browsers, Firefox gives you a different engine fingerprint. WebKit is useful for testing how iOS Safari users see content.
SOCKS5 Proxy Configuration
NodeMaven supports SOCKS5 proxies, which handle all traffic types, not just HTTP. SOCKS5 is available at port 1080. For more on the SOCKS5 protocol and use cases, see nodemaven.com/proxies/socks5-proxy-server/.
async def socks5_example():
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"socks5://{PROXY_HOST}:1080",
"username": PROXY_USER,
"password": PROXY_PASS,
}
)
page = await browser.new_page()
await page.goto("https://httpbin.org/ip")
print(await page.inner_text("body"))
await browser.close()
asyncio.run(socks5_example())
Rotating Sessions: New IP Per Browser Instance
For scraping tasks that need a different IP on each run, instantiate a new browser for each request batch. NodeMaven assigns a new IP from the 30M+ residential pool on each new connection when no session parameter is set.
import asyncio
import random
from playwright.async_api import async_playwright
TARGETS = [
"https://www.amazon.com/s?k=laptop",
"https://www.amazon.com/s?k=monitor",
"https://www.amazon.com/s?k=keyboard",
]
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/124.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 Chrome/124.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0",
]
async def scrape_with_rotation(url: str):
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
}
)
context = await browser.new_context(
user_agent=random.choice(USER_AGENTS),
viewport={"width": 1366, "height": 768},
)
page = await context.new_page()
try:
await page.goto(url, wait_until="domcontentloaded", timeout=30000)
title = await page.title()
print(f"OK: {title[:60]} | {url}")
return await page.content()
except Exception as e:
print(f"Failed: {url} | {e}")
return None
finally:
await browser.close()
async def main():
for url in TARGETS:
html = await scrape_with_rotation(url)
await asyncio.sleep(random.uniform(2, 5))
asyncio.run(main())
Each call to scrape_with_rotation() opens a new browser and gets a fresh IP from the pool. NodeMaven's 95% clean IP rate and 99.54% average success rate mean the vast majority of these fresh IPs succeed on the first attempt.
Sticky Sessions: Same IP Across Multiple Pages
For workflows requiring session continuity — login flows, checkout sequences, account management — configure sticky sessions in your NodeMaven dashboard before starting. The same IP holds for up to 7 days on residential proxies.
async def login_and_scrape():
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
},
headless=True,
)
context = await browser.new_context(
user_agent=(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/124.0.0.0 Safari/537.36"
),
viewport={"width": 1440, "height": 900},
)
page = await context.new_page()
# Step 1: login page
await page.goto("https://example-site.com/login", wait_until="networkidle")
await page.fill("#username", "your_account_username")
await page.fill("#password", "your_account_password")
await page.click("#login-button")
await page.wait_for_load_state("networkidle")
# Step 2: navigate to protected page - same IP throughout
await page.goto("https://example-site.com/dashboard")
print("Dashboard loaded, IP consistent across all steps")
cookies = await context.cookies()
print(f"Saved {len(cookies)} cookies")
await browser.close()
asyncio.run(login_and_scrape())
Context-Level Proxy Override
Playwright supports setting proxy at the context level, which lets you run multiple contexts within one browser instance, each with a different proxy:
async def multi_context_proxies():
async with async_playwright() as p:
browser = await p.chromium.launch()
context_us = await browser.new_context(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": f"{PROXY_USER}-country-us",
"password": PROXY_PASS,
}
)
context_uk = await browser.new_context(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": f"{PROXY_USER}-country-gb",
"password": PROXY_PASS,
}
)
page_us = await context_us.new_page()
page_uk = await context_uk.new_page()
await page_us.goto("https://httpbin.org/ip")
await page_uk.goto("https://httpbin.org/ip")
print("US IP:", await page_us.inner_text("body"))
print("UK IP:", await page_uk.inner_text("body"))
await browser.close()
asyncio.run(multi_context_proxies())
This pattern is useful for geo-comparison tasks: checking how a page renders for US vs UK users in parallel, or running ad verification across multiple markets simultaneously.
CAPTCHA Detection and Handling
NodeMaven's 95% clean IP rate reduces CAPTCHA frequency significantly. But CAPTCHAs still appear occasionally. Detect them and rotate to a fresh IP rather than retrying the same address:
CAPTCHA_SIGNALS = [
"captcha", "recaptcha", "hcaptcha",
"are you a robot", "verify you are human",
"unusual traffic", "automated queries", "g-recaptcha",
]
async def is_captcha_page(page) -> bool:
try:
content = (await page.content()).lower()
title = (await page.title()).lower()
return any(s in content or s in title for s in CAPTCHA_SIGNALS)
except Exception:
return False
async def scrape_with_captcha_handling(url: str, max_retries: int = 3):
for attempt in range(max_retries):
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
}
)
page = await (await browser.new_context()).new_page()
try:
await page.goto(url, wait_until="domcontentloaded", timeout=30000)
if await is_captcha_page(page):
print(f"CAPTCHA on attempt {attempt + 1}, rotating IP...")
await browser.close()
await asyncio.sleep(random.uniform(2, 5))
continue
content = await page.content()
await browser.close()
return content
except Exception as e:
print(f"Error on attempt {attempt + 1}: {e}")
await browser.close()
await asyncio.sleep(random.uniform(1, 3))
return None
Each retry opens a new browser and draws a fresh IP from the pool. Since the pool is 30M+ pre-filtered IPs, successive retries are unlikely to encounter the same flagged address.
Bandwidth Saving: Blocking Unnecessary Assets
Playwright's route() API lets you intercept requests before they go out through the proxy. Block images, fonts, and stylesheets to reduce bandwidth consumption on text-heavy scraping targets:
async def scrape_with_request_filtering(url: str):
async with async_playwright() as p:
browser = await p.chromium.launch(
proxy={
"server": f"http://{PROXY_HOST}:{PROXY_PORT}",
"username": PROXY_USER,
"password": PROXY_PASS,
}
)
page = await browser.new_page()
await page.route(
"**/*",
lambda route: route.abort()
if route.request.resource_type in ["image", "font", "media", "stylesheet"]
else route.continue_()
)
await page.goto(url, wait_until="domcontentloaded")
content = await page.content()
await browser.close()
return content
Since NodeMaven is priced per GB at $2.20/GB, reducing unnecessary asset downloads translates directly to lower costs on large-scale runs.
Quick Reference: Playwright Proxy Patterns
| Pattern | Config level | Use case |
|---|---|---|
| Single browser proxy | launch(proxy=...) |
All pages use same proxy |
| Context-level proxy | new_context(proxy=...) |
Different proxy per context in one browser |
| Rotating IP | New browser per task | SERP scraping, price monitoring |
| Sticky session | Dashboard session config | Login flows, account management |
| SOCKS5 |
server: socks5:// port 1080 |
All traffic types, not just HTTP |
| CAPTCHA retry | Close browser, open new | Any anti-bot target |
| Bandwidth saving | page.route() |
Text scraping at scale |
Getting Started
NodeMaven residential proxies integrate with Playwright's launch() and new_context() proxy parameters directly. 30M+ IPs across 190+ countries, 95% clean IP rate, 99.54% average success rate, sticky sessions up to 24 hours, HTTP and SOCKS5 support. From $2.20/GB with traffic rollover and cashback on used bandwidth. Trial at $3.50 for 750MB.
Get credentials from the dashboard at nodemaven.com/proxies/residential-proxies/ and paste into the PROXY_USER and PROXY_PASS variables in any of the examples above.
Top comments (0)