DEV Community

Elen Simonian
Elen Simonian

Posted on

Playwright with Proxies: Full Integration Guide for 2026

Playwright is the standard tool for browser automation in 2026. It handles Chromium, Firefox, and WebKit from a single API, supports async/await natively, and integrates cleanly with modern Python and Node.js workflows. But without proxies, Playwright automation hits rate limits and blocks quickly on any platform with bot detection.

This guide covers every aspect of integrating NodeMaven residential proxies with Playwright: proxy configuration for all three browser engines, authentication handling, rotating and sticky session patterns, and a CAPTCHA detection strategy. All code is production-ready and tested against real-world targets.

Prerequisites

Install Playwright and its browser binaries:

pip install playwright
playwright install
Enter fullscreen mode Exit fullscreen mode

You will need NodeMaven proxy credentials from your dashboard. Copy them in {host}:{port}:{username}:{password} format. NodeMaven supports both HTTP and SOCKS5 protocols. Use port 8080 for HTTP and port 1080 for SOCKS5.

Basic Proxy Setup: Chromium

Playwright accepts proxy configuration at the browser level via the proxy parameter in launch(). The server field takes the full proxy URL including protocol.

import asyncio
from playwright.async_api import async_playwright

PROXY_HOST = "gate.nodemaven.com"
PROXY_PORT = "8080"
PROXY_USER = "your_nodemaven_username"
PROXY_PASS = "your_nodemaven_password"

async def basic_chromium():
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": PROXY_USER,
                "password": PROXY_PASS,
            }
        )
        page = await browser.new_page()
        await page.goto("https://httpbin.org/ip")
        content = await page.content()
        print(content)
        await browser.close()

asyncio.run(basic_chromium())
Enter fullscreen mode Exit fullscreen mode

The proxy applies to all requests made by the browser instance. Every page opened under this browser uses the same proxy configuration.

Proxy Setup: Firefox and WebKit

The proxy configuration syntax is identical across all three engines. Swap p.chromium for p.firefox or p.webkit:

async def multi_engine_example():
    async with async_playwright() as p:
        proxy_config = {
            "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
            "username": PROXY_USER,
            "password": PROXY_PASS,
        }

        browser_cr = await p.chromium.launch(proxy=proxy_config)
        browser_ff = await p.firefox.launch(proxy=proxy_config)
        browser_wk = await p.webkit.launch(proxy=proxy_config)

        for browser, name in [
            (browser_cr, "Chromium"),
            (browser_ff, "Firefox"),
            (browser_wk, "WebKit"),
        ]:
            page = await browser.new_page()
            await page.goto("https://httpbin.org/ip")
            print(f"{name}: {await page.inner_text('body')}")
            await browser.close()

asyncio.run(multi_engine_example())
Enter fullscreen mode Exit fullscreen mode

For scraping workflows targeting sites that fingerprint browser engine type, use Chromium. For sites that explicitly check for non-Chrome browsers, Firefox gives you a different engine fingerprint. WebKit is useful for testing how iOS Safari users see content.

SOCKS5 Proxy Configuration

NodeMaven supports SOCKS5 proxies, which handle all traffic types, not just HTTP. SOCKS5 is available at port 1080. For more on the SOCKS5 protocol and use cases, see nodemaven.com/proxies/socks5-proxy-server/.

async def socks5_example():
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            proxy={
                "server": f"socks5://{PROXY_HOST}:1080",
                "username": PROXY_USER,
                "password": PROXY_PASS,
            }
        )
        page = await browser.new_page()
        await page.goto("https://httpbin.org/ip")
        print(await page.inner_text("body"))
        await browser.close()

asyncio.run(socks5_example())
Enter fullscreen mode Exit fullscreen mode

Rotating Sessions: New IP Per Browser Instance

For scraping tasks that need a different IP on each run, instantiate a new browser for each request batch. NodeMaven assigns a new IP from the 30M+ residential pool on each new connection when no session parameter is set.

import asyncio
import random
from playwright.async_api import async_playwright

TARGETS = [
    "https://www.amazon.com/s?k=laptop",
    "https://www.amazon.com/s?k=monitor",
    "https://www.amazon.com/s?k=keyboard",
]

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0",
]

async def scrape_with_rotation(url: str):
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": PROXY_USER,
                "password": PROXY_PASS,
            }
        )
        context = await browser.new_context(
            user_agent=random.choice(USER_AGENTS),
            viewport={"width": 1366, "height": 768},
        )
        page = await context.new_page()
        try:
            await page.goto(url, wait_until="domcontentloaded", timeout=30000)
            title = await page.title()
            print(f"OK: {title[:60]} | {url}")
            return await page.content()
        except Exception as e:
            print(f"Failed: {url} | {e}")
            return None
        finally:
            await browser.close()

async def main():
    for url in TARGETS:
        html = await scrape_with_rotation(url)
        await asyncio.sleep(random.uniform(2, 5))

asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode

Each call to scrape_with_rotation() opens a new browser and gets a fresh IP from the pool. NodeMaven's 95% clean IP rate and 99.54% average success rate mean the vast majority of these fresh IPs succeed on the first attempt.

Sticky Sessions: Same IP Across Multiple Pages

For workflows requiring session continuity — login flows, checkout sequences, account management — configure sticky sessions in your NodeMaven dashboard before starting. The same IP holds for up to 7 days on residential proxies.

async def login_and_scrape():
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": PROXY_USER,
                "password": PROXY_PASS,
            },
            headless=True,
        )
        context = await browser.new_context(
            user_agent=(
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/124.0.0.0 Safari/537.36"
            ),
            viewport={"width": 1440, "height": 900},
        )
        page = await context.new_page()

        # Step 1: login page
        await page.goto("https://example-site.com/login", wait_until="networkidle")
        await page.fill("#username", "your_account_username")
        await page.fill("#password", "your_account_password")
        await page.click("#login-button")
        await page.wait_for_load_state("networkidle")

        # Step 2: navigate to protected page - same IP throughout
        await page.goto("https://example-site.com/dashboard")
        print("Dashboard loaded, IP consistent across all steps")

        cookies = await context.cookies()
        print(f"Saved {len(cookies)} cookies")
        await browser.close()

asyncio.run(login_and_scrape())
Enter fullscreen mode Exit fullscreen mode

Context-Level Proxy Override

Playwright supports setting proxy at the context level, which lets you run multiple contexts within one browser instance, each with a different proxy:

async def multi_context_proxies():
    async with async_playwright() as p:
        browser = await p.chromium.launch()

        context_us = await browser.new_context(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": f"{PROXY_USER}-country-us",
                "password": PROXY_PASS,
            }
        )
        context_uk = await browser.new_context(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": f"{PROXY_USER}-country-gb",
                "password": PROXY_PASS,
            }
        )

        page_us = await context_us.new_page()
        page_uk = await context_uk.new_page()

        await page_us.goto("https://httpbin.org/ip")
        await page_uk.goto("https://httpbin.org/ip")

        print("US IP:", await page_us.inner_text("body"))
        print("UK IP:", await page_uk.inner_text("body"))
        await browser.close()

asyncio.run(multi_context_proxies())
Enter fullscreen mode Exit fullscreen mode

This pattern is useful for geo-comparison tasks: checking how a page renders for US vs UK users in parallel, or running ad verification across multiple markets simultaneously.

CAPTCHA Detection and Handling

NodeMaven's 95% clean IP rate reduces CAPTCHA frequency significantly. But CAPTCHAs still appear occasionally. Detect them and rotate to a fresh IP rather than retrying the same address:

CAPTCHA_SIGNALS = [
    "captcha", "recaptcha", "hcaptcha",
    "are you a robot", "verify you are human",
    "unusual traffic", "automated queries", "g-recaptcha",
]

async def is_captcha_page(page) -> bool:
    try:
        content = (await page.content()).lower()
        title   = (await page.title()).lower()
        return any(s in content or s in title for s in CAPTCHA_SIGNALS)
    except Exception:
        return False

async def scrape_with_captcha_handling(url: str, max_retries: int = 3):
    for attempt in range(max_retries):
        async with async_playwright() as p:
            browser = await p.chromium.launch(
                proxy={
                    "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                    "username": PROXY_USER,
                    "password": PROXY_PASS,
                }
            )
            page = await (await browser.new_context()).new_page()
            try:
                await page.goto(url, wait_until="domcontentloaded", timeout=30000)
                if await is_captcha_page(page):
                    print(f"CAPTCHA on attempt {attempt + 1}, rotating IP...")
                    await browser.close()
                    await asyncio.sleep(random.uniform(2, 5))
                    continue
                content = await page.content()
                await browser.close()
                return content
            except Exception as e:
                print(f"Error on attempt {attempt + 1}: {e}")
                await browser.close()
                await asyncio.sleep(random.uniform(1, 3))
    return None
Enter fullscreen mode Exit fullscreen mode

Each retry opens a new browser and draws a fresh IP from the pool. Since the pool is 30M+ pre-filtered IPs, successive retries are unlikely to encounter the same flagged address.

Bandwidth Saving: Blocking Unnecessary Assets

Playwright's route() API lets you intercept requests before they go out through the proxy. Block images, fonts, and stylesheets to reduce bandwidth consumption on text-heavy scraping targets:

async def scrape_with_request_filtering(url: str):
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            proxy={
                "server": f"http://{PROXY_HOST}:{PROXY_PORT}",
                "username": PROXY_USER,
                "password": PROXY_PASS,
            }
        )
        page = await browser.new_page()
        await page.route(
            "**/*",
            lambda route: route.abort()
            if route.request.resource_type in ["image", "font", "media", "stylesheet"]
            else route.continue_()
        )
        await page.goto(url, wait_until="domcontentloaded")
        content = await page.content()
        await browser.close()
        return content
Enter fullscreen mode Exit fullscreen mode

Since NodeMaven is priced per GB at $2.20/GB, reducing unnecessary asset downloads translates directly to lower costs on large-scale runs.

Quick Reference: Playwright Proxy Patterns

Pattern Config level Use case
Single browser proxy launch(proxy=...) All pages use same proxy
Context-level proxy new_context(proxy=...) Different proxy per context in one browser
Rotating IP New browser per task SERP scraping, price monitoring
Sticky session Dashboard session config Login flows, account management
SOCKS5 server: socks5:// port 1080 All traffic types, not just HTTP
CAPTCHA retry Close browser, open new Any anti-bot target
Bandwidth saving page.route() Text scraping at scale

Getting Started

NodeMaven residential proxies integrate with Playwright's launch() and new_context() proxy parameters directly. 30M+ IPs across 190+ countries, 95% clean IP rate, 99.54% average success rate, sticky sessions up to 24 hours, HTTP and SOCKS5 support. From $2.20/GB with traffic rollover and cashback on used bandwidth. Trial at $3.50 for 750MB.

Get credentials from the dashboard at nodemaven.com/proxies/residential-proxies/ and paste into the PROXY_USER and PROXY_PASS variables in any of the examples above.

Top comments (0)