DEV Community

Miller James
Miller James

Posted on

Building a Regional Request Router With Residential Proxies in Python

By the Proxy001 engineering team · Last reviewed April 2026
Disclosure: Proxy001 sponsors this content. All technical specifications are verified from proxy001.com as of April 2026.


Most tutorials stop at "here's how to pass a proxy dict to requests.get()." That's fine if you're routing a single request through a single IP. But if you're building a service that sends traffic through a US exit for one task, a DE exit for another, and a JP exit for a third — all from the same Python process — you need a router that maps country codes to proxy sessions and manages them for you.

This tutorial builds a RegionalRouter class that does exactly that: maps country codes to residential proxy exits, manages a session pool, handles retries and errors cleanly, and can be wrapped into a lightweight HTTP service with basic access control. You'll have something fully runnable before the end of the page.


Prerequisites

Before you write a single line of router code, confirm the following:

  • Python 3.8+ — f-strings and typing improvements used throughout
  • requests library: pip install requests
  • urllib3 ≥ 1.26.0 — required for the allowed_methods parameter on Retry (ships with requests 2.26+); earlier versions use the deprecated method_whitelist instead
  • SOCKS5 support (optional, but recommended): pip install "requests[socks]" — this pulls in PySocks, which is needed if your provider offers SOCKS5 endpoints
  • python-dotenv for credential management: pip install python-dotenv
  • A residential proxy account that supports country-level geo-targeting — your provider must allow you to embed a country code in the proxy username. This is the mechanism the entire router relies on. If your provider only offers random rotation without country selection, the geo-routing won't work regardless of how the code is written.

Your proxy account should give you four things: a gateway hostname, a port, a username, and a password. Keep those handy.


How Does Country-Binding Work in a Residential Proxy?

Most residential proxy providers expose a single gateway — one hostname, one port. Country selection doesn't happen by pointing to a different server; it happens by encoding the target country code directly into the proxy username.

Bright Data's documentation describes the pattern clearly: to route through a US exit, you append a country flag to your zone username, producing something like customer-XXXX-zone-resi-country-us. Oxylabs uses the same username-embedding convention with a cc parameter. The resulting proxy URL takes this form:

http://user-country-US:password@gateway.example.com:10000
Enter fullscreen mode Exit fullscreen mode

Change US to DE, JP, or any ISO 3166-1 alpha-2 code, and the provider routes your request through an exit node in that country. Same gateway, same port — only the username string changes. That's what makes a dynamic router feasible: you're generating per-country proxy URLs at runtime through string interpolation, not maintaining 50 separate connections to 50 different servers.

The exact username format varies by provider. Check your dashboard's "Proxy Setup" or "Integration" tab and look for a field labeled something like "username with location" or a code example showing country targeting — that string is your template.


Step 1 — Build the Country-to-Proxy URL Mapping

Start by loading your credentials from environment variables. If your password contains characters like @, :, #, or %, you must URL-encode it before embedding it in the proxy URL — an unescaped @ will break URL parsing and you'll get a 407 with no obvious explanation.

Create a .env file:

PROXY_HOST=gateway.yourprovider.com
PROXY_PORT=10000
PROXY_USER=your_base_username
PROXY_PASS=your_p@ssword!
Enter fullscreen mode Exit fullscreen mode

Now build the mapping. The build_proxy_url function below handles encoding and constructs the country-specific username:

import os
from urllib.parse import quote
from dotenv import load_dotenv

load_dotenv()

PROXY_HOST = os.environ["PROXY_HOST"]
PROXY_PORT = os.environ["PROXY_PORT"]
PROXY_USER = os.environ["PROXY_USER"]
PROXY_PASS = quote(os.environ["PROXY_PASS"], safe="")  # encode ALL special chars


def build_proxy_url(country_code: str) -> str:
    """
    Construct a country-targeted residential proxy URL.
    Country code must be ISO 3166-1 alpha-2 (e.g. 'US', 'DE', 'JP').
    Adjust the username suffix to match your provider's exact syntax.
    """
    username = f"{PROXY_USER}-country-{country_code.upper()}"
    return f"http://{username}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"


# Pre-build a mapping for the countries you'll route through
SUPPORTED_COUNTRIES = ["US", "DE", "GB", "JP", "FR", "SG", "BR"]

COUNTRY_PROXIES: dict[str, dict] = {
    cc: {"http": build_proxy_url(cc), "https": build_proxy_url(cc)}
    for cc in SUPPORTED_COUNTRIES
}
Enter fullscreen mode Exit fullscreen mode

Provider syntax note: The -country-XX suffix above follows the pattern used by Bright Data and several other major providers. If you're using Proxy001, their geo-targeting uses the same username-parameter model — check the "Proxy Setup" section of your dashboard for the exact suffix format for your zone type. Proxy001 supports country, city, and carrier-level targeting across 200+ regions, so the same build_proxy_url logic applies once you have the correct template string.


Step 2 — Build the RegionalRouter Class With a Session Pool

Creating a new requests.Session for every request wastes TCP connection setup time and can exhaust file descriptors under sustained load. The right approach is to pre-build one session per country and reuse it — a session pool.

One thing to be aware of upfront: pre-building sessions for every supported country means holding that many open connections. For 7–15 countries, the memory cost is negligible. If you're routing through 30+ countries or running in a constrained environment like AWS Lambda, lazy initialization (building sessions on first use rather than at startup) is worth considering — you'd replace the __init__ loop with a _get_or_create_session(cc) method. For most services, the pre-built approach below is simpler and performs better.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry  # requires urllib3 >= 1.26.0


class RoutingError(Exception):
    """Raised when a regional request cannot be completed."""
    pass


class RegionalRouter:
    def __init__(
        self,
        country_proxies: dict,
        timeout: tuple = (10, 30),
        max_retries: int = 3,
        backoff_factor: float = 0.5,
    ):
        """
        country_proxies: dict mapping ISO country codes to proxy dicts,
                         e.g. {"US": {"http": "http://...", "https": "http://..."}}
        timeout:         (connect_timeout, read_timeout) in seconds
        max_retries:     retry attempts on connection errors and 5xx responses
        backoff_factor:  sleep between retries = backoff_factor * (2 ** (retry_n - 1))
        """
        self.timeout = timeout
        self._sessions: dict[str, requests.Session] = {}

        # raise_on_status=False here means Retry won't raise mid-retry cycle;
        # the final response's raise_for_status() is called in route() instead,
        # so callers always get a RoutingError with the status code included.
        retry_strategy = Retry(
            total=max_retries,
            backoff_factor=backoff_factor,
            status_forcelist=[429, 500, 502, 503, 504],
            allowed_methods=["GET", "POST", "HEAD"],  # urllib3 >= 1.26.0
            raise_on_status=False,
        )
        adapter = HTTPAdapter(
            max_retries=retry_strategy,
            pool_connections=len(country_proxies),  # one pool per country
            pool_maxsize=10,
        )

        for cc, proxy_dict in country_proxies.items():
            session = requests.Session()
            session.proxies.update(proxy_dict)
            session.mount("http://", adapter)
            session.mount("https://", adapter)
            self._sessions[cc.upper()] = session

    def route(self, url: str, country: str, **kwargs) -> requests.Response:
        """
        Send a GET request through the residential exit for the given country.

        url:     Target URL
        country: ISO 3166-1 alpha-2 country code (e.g. 'US', 'DE')
        kwargs:  Passed through to session.get() (headers, params, etc.)
        """
        cc = country.upper()
        session = self._sessions.get(cc)
        if session is None:
            raise ValueError(
                f"Country '{cc}' is not in the supported list: "
                f"{list(self._sessions.keys())}"
            )

        kwargs.setdefault("timeout", self.timeout)

        try:
            response = session.get(url, **kwargs)
            response.raise_for_status()
            return response
        except requests.exceptions.ProxyError as e:
            raise RoutingError(f"[{cc}] Proxy connection failed: {e}") from e
        except requests.exceptions.ConnectTimeout as e:
            raise RoutingError(f"[{cc}] Connection timed out: {e}") from e
        except requests.exceptions.HTTPError as e:
            raise RoutingError(
                f"[{cc}] HTTP {e.response.status_code} from target: {url}"
            ) from e
        except requests.exceptions.RequestException as e:
            raise RoutingError(f"[{cc}] Request failed: {e}") from e
Enter fullscreen mode Exit fullscreen mode

Wire it together with the mapping from Step 1:

router = RegionalRouter(country_proxies=COUNTRY_PROXIES)
Enter fullscreen mode Exit fullscreen mode

The Retry object handles transient 429s and 5xx responses with exponential backoff. Setting pool_connections to the number of countries gives each country its own connection pool rather than having all sessions compete for a shared one.


Step 3 — Add Graceful Error Handling

The route() method wraps all proxy-layer and HTTP-layer exceptions into RoutingError. Callers don't need to catch five different exception types, the country code is always included in the message, and the original exception is preserved via from e for stack traces. In practice that last point matters more than it sounds — when a ProxyError bubbles up in a log aggregator at 2am, having the original exception chained makes the difference between a 5-minute fix and a 30-minute debugging session.

For your calling code, a typical pattern looks like this:

try:
    resp = router.route("https://example.com/api/data", country="DE")
    print(resp.json())
except ValueError as e:
    # Unsupported country — likely a bug in the calling code
    print(f"Config error: {e}")
except RoutingError as e:
    # Proxy or network failure — log and handle gracefully
    print(f"Routing failed: {e}")
Enter fullscreen mode Exit fullscreen mode

One design decision worth flagging: raise_for_status() is called inside route(), which means 4xx and 5xx responses from the target site also become RoutingError. That's intentional for most use cases — you want to know if the target returned a 403, not silently process an empty body. If you'd rather inspect the response yourself, remove response.raise_for_status() from route() and check response.status_code in the caller.


How Do I Confirm the Router Is Using the Right Country Exit?

Run this verification before you point the router at your real targets. It uses ip-api.com/json, which returns the geographic location of the requesting IP:

def verify_routing(router: RegionalRouter, countries: list[str]) -> None:
    """
    Check that each country exit is actually routing through the target country.
    Prints a pass/fail line per country code.
    """
    for cc in countries:
        try:
            resp = router.route("http://ip-api.com/json", country=cc)
            data = resp.json()
            actual_cc = data.get("countryCode", "UNKNOWN")
            status = "✓ PASS" if actual_cc == cc else f"✗ FAIL (got {actual_cc})"
            print(f"  [{cc}] {status} — exit IP: {data.get('query')}")
        except RoutingError as e:
            print(f"  [{cc}] ✗ ERROR — {e}")

verify_routing(router, SUPPORTED_COUNTRIES)
Enter fullscreen mode Exit fullscreen mode

Expected output:

  [US] ✓ PASS — exit IP: 104.28.x.x
  [DE] ✓ PASS — exit IP: 185.220.x.x
  [GB] ✓ PASS — exit IP: 51.36.x.x
  ...
Enter fullscreen mode Exit fullscreen mode

Once verify_routing() passes for all target countries, it's worth adding to your CI pipeline or startup health check — catching a misconfigured credential or expired account at deploy time is far better than debugging a geo-mismatch in production at request time.

That said: if you see a consistent FAIL for one specific region despite the code looking correct, it's almost always a provider plan limitation rather than a bug. The curl test in Troubleshooting #3 below confirms this in about 30 seconds, without touching Python at all.


Troubleshooting: 3 Real Pitfalls

These aren't hypothetical edge cases pulled from documentation. The socks5h DNS issue is particularly nasty because requests succeed — they just silently exit through the wrong country, or connect to the wrong resolved IP, and nothing in the logs tells you why. The password encoding problem is the most common "it works in curl but not in Python" symptom we've seen. Here's what to look for in each case.


1. socks5:// Resolves DNS Locally — Use socks5h:// Instead

Symptom: Requests succeed, but the exit IP doesn't match the target country — or you get connection errors on endpoints that expect hostname-based routing.

Cause: With socks5://, the Python requests library resolves the hostname on the client side before forwarding the connection through the proxy. The proxy server sees an IP address, not a hostname. Some residential proxy providers route by hostname at their infrastructure layer, so when they receive a bare IP, the country-targeting logic has nothing to work with.

Fix: Switch the proxy URL prefix from socks5:// to socks5h://:

# Wrong — DNS resolves locally, country routing may silently fail
"socks5://user-country-US:pass@gateway.example.com:10000"

# Correct — DNS resolves on the proxy side
"socks5h://user-country-US:pass@gateway.example.com:10000"
Enter fullscreen mode Exit fullscreen mode

This requires pip install "requests[socks]" with PySocks installed. Update build_proxy_url to use socks5h:// instead of http:// if your provider's endpoint is SOCKS5.


2. Special Characters in the Password Cause 407

Symptom: 407 Proxy Authentication Required even though the credentials are confirmed correct in a browser or curl test.

Cause: Characters like @, :, #, and % are URL delimiters. When they appear unencoded in the password portion of a proxy URL, the library misparses where the password ends and the hostname begins. The result is a malformed authentication header that the proxy server rejects.

Fix: urllib.parse.quote(password, safe="") encodes every special character, including /:

from urllib.parse import quote

raw_password = "p@ss:w0rd#!"
encoded = quote(raw_password, safe="")
# Result: "p%40ss%3Aw0rd%23%21"

proxy_url = f"http://user-country-US:{encoded}@gateway.example.com:10000"
Enter fullscreen mode Exit fullscreen mode

The build_proxy_url function in Step 1 already applies this via PROXY_PASS = quote(os.environ["PROXY_PASS"], safe=""). The risk arises if you construct proxy URLs anywhere else in your codebase without going through that function — worth a quick grep for @{PROXY_HOST} or @gateway to catch any direct string construction.


3. Country Code Is Ignored — Exit IP Doesn't Match

Symptom: verify_routing() reports FAIL (got XX) for one or more countries despite the code running without errors.

Cause: Not all residential proxy providers support country-level geo-targeting on all plans or pool types. Entry-level plans, shared ISP pools, and some trial accounts route randomly regardless of the username parameter — the provider accepts the credential but silently ignores the country suffix.

Fix — diagnose in three steps:

  1. Rule out the code first. Run the equivalent curl command directly:
   curl -x "http://user-country-DE:pass@gateway:port" http://ip-api.com/json
Enter fullscreen mode Exit fullscreen mode

If curl also returns the wrong country, it's a provider or plan issue, not Python.

  1. Check your plan's feature flags. Geo-targeting is usually a toggle in the provider dashboard — look for a "Location" or "Targeting" section in your zone settings. It may need to be explicitly enabled.

  2. Confirm the username suffix format. Some providers use -country-US, others use -cc-US, and some require a different entry-point hostname per country rather than a username parameter. The format in build_proxy_url may need adjusting. Your provider's integration documentation (not the generic proxy setup page, but the language-specific or zone-specific docs) is the right place to check.


Optional: Expose the Router as a Lightweight HTTP Service

If other services, cron jobs, or CLI tools need to use the router over HTTP, wrapping it in a FastAPI endpoint is straightforward. The snippet below includes a minimal API key check — this matters more than it might seem, because an unauthenticated /fetch endpoint lets anyone with network access route arbitrary traffic through your proxy quota.

First, add the API key to your .env:

ROUTER_API_KEY=your-secret-key-here
Enter fullscreen mode Exit fullscreen mode

Then:

from fastapi import FastAPI, HTTPException, Security
from fastapi.security import APIKeyHeader
from pydantic import BaseModel

app = FastAPI()

ROUTER_API_KEY = os.environ["ROUTER_API_KEY"]
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=True)


def verify_api_key(key: str = Security(api_key_header)) -> None:
    if key != ROUTER_API_KEY:
        raise HTTPException(status_code=403, detail="Invalid API key")


class FetchRequest(BaseModel):
    url: str
    country: str


@app.post("/fetch")
def fetch(req: FetchRequest, _: None = Security(verify_api_key)):
    try:
        resp = router.route(req.url, req.country)
        return {
            "status_code": resp.status_code,
            "content": resp.text,
        }
    except ValueError as e:
        raise HTTPException(status_code=400, detail=str(e))
    except RoutingError as e:
        raise HTTPException(status_code=502, detail=str(e))
Enter fullscreen mode Exit fullscreen mode

Run with: uvicorn main:app --reload

Call it:

curl -X POST http://localhost:8000/fetch \
  -H "X-API-Key: your-secret-key-here" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "country": "JP"}'
Enter fullscreen mode Exit fullscreen mode

For rate limiting before any external exposure, slowapi wraps FastAPI in about 10 lines — their README has the decorator-based pattern. Both auth and rate limiting are worth adding before you expose this beyond localhost.


Start Routing With a Free Trial

You now have a complete RegionalRouter: country-to-proxy URL mapping from environment variables, a pre-built session pool with per-country connection pools and exponential-backoff retry, clean error handling with chained exceptions, a verify_routing() function to confirm geo-exits before deployment, troubleshooting paths for the three most common failures, and an HTTP wrapper with API key authentication.

The only external dependency is a residential proxy account with geo-targeting enabled. Proxy001 covers 200+ regions with 100M+ IPs and supports country, city, and carrier-level targeting — the same username-parameter model this router is built around. The free trial gives you 500 MB without a credit card, which is enough to run verify_routing() across your full country list and load-test the session pool before committing to a plan. Pricing runs from $2.00/GB for small volumes down to $0.70/GB at higher tiers (as of April 2026; verify current rates on their pricing page).

(Disclosure: Proxy001 sponsors this content. Technical specifications verified from proxy001.com as of April 2026.)

Top comments (0)