Selenium is still the workhorse for browser automation and testing, and adding a proxy to it should be simple. It usually is, until you need authentication, or you switch from Chrome to Firefox and the same code stops working. The proxy story differs by browser, and the auth story is where most people get stuck. Here is a practical setup for both Chrome and Firefox, including the credential problem that trips everyone up.
Chrome with a proxy
For Chrome you pass the proxy as a command-line argument on the options object.
from selenium import webdriver
opts = webdriver.ChromeOptions()
opts.add_argument("--proxy-server=http://proxy.host:8080")
driver = webdriver.Chrome(options=opts)
driver.get("https://example.com")
This works cleanly for a proxy with no auth. The catch is the same one Chromium always has: --proxy-server does not carry a username and password, so an authenticated proxy gives you a login popup that Selenium cannot type into.
The authentication problem
When the proxy needs credentials, the browser throws up a native auth dialog, and native dialogs are outside the DOM, so Selenium cannot fill them. There are two common ways around it.
-
A small extension. Package a tiny Chrome extension that supplies the credentials through the
webRequest.onAuthRequiredhandler, and load it at launch. It is the most reliable method for Chrome auth. - Credentials in the endpoint via a local forwarder. Point Selenium at a local proxy that itself holds the upstream credentials, so the browser never sees an auth challenge.
Neither is elegant, but both are stable. The extension approach is the one most scraping setups settle on.
Firefox with a proxy
Firefox does not take a launch arg. You set proxy preferences on the profile, which is more verbose but gives you fine control.
from selenium import webdriver
opts = webdriver.FirefoxOptions()
opts.set_preference("network.proxy.type", 1)
opts.set_preference("network.proxy.http", "proxy.host")
opts.set_preference("network.proxy.http_port", 8080)
opts.set_preference("network.proxy.ssl", "proxy.host")
opts.set_preference("network.proxy.ssl_port", 8080)
driver = webdriver.Firefox(options=opts)
driver.get("https://example.com")
Firefox has the same native-dialog problem for auth, handled the same way, with a helper that answers the challenge.
Rotation and isolation
Whichever browser, the anti-blocking rules are the same. A headless browser generates a burst of requests, so one address gets flagged fast. Point the proxy setting at a rotating endpoint so exits cycle, and when a session must stay logged in, pin it to one stable address for its life. Do not launch many drivers against one target through a single exit, or rotation will not save you.
What the pool needs
Selenium leans on the address pool the same way any browser automation does. WinGate provides private proxies with SOCKS5 and standard user and password auth, so your extension or forwarder has real credentials to present, and the addresses are private rather than shared. The rotating pool cycles exits from a worldmix, traffic is unlimited for asset-heavy page loads, and it supports up to 5000 threads for many parallel drivers. It speaks HTTP, HTTPS, and SOCKS5, so both the Chrome arg and the Firefox preferences work with the same account.
An honest note: a proxy stops the address from being the reason Selenium gets blocked, it does not hide that you are driving a browser. Fingerprinting, pacing, and the site's terms still matter. There is a free 2 hour test, so wire the proxy into a small Chrome and Firefox run and confirm auth works before you scale drivers up.
The takeaway: Chrome takes a launch arg, Firefox takes profile preferences, and both need a helper to answer the native auth dialog. Point them at a rotating private pool, isolate stateful sessions on stable exits, and Selenium automates reliably across both browsers without getting blocked on the first burst.
Top comments (0)