How to Rotate Proxies in Python with Requests (2026 Guide)

#automation #python #tutorial #webscraping

Originally published on UnlimitedResidentialProxy.

Websites may apply rate limits or access controls to automated requests. For permitted data collection, residential proxy rotation can be one useful configuration alongside careful pacing, retries, and error handling.

Only automate requests that you are authorized to make. Respect the target website's terms, robots directives, rate limits, and applicable law.

Why rotate residential proxies?

Sending every request through one static exit can make a long-running collection job fragile. Rotating residential sessions distributes authorized requests across available exits. It does not make automation undetectable, guarantee access, or replace responsible request pacing.

Install Requests

python -m pip install requests

Configure the proxy gateway

Rotation is handled by the gateway, so your Python code does not need to maintain and validate a large list of proxy IPs.

import requests

PROXY_HOST = "<YOUR_GATEWAY_HOST>"
PROXY_PORT = "7777"
PROXY_USER = "USERNAME"
PROXY_PASS = "PASSWORD"

proxy_url = (
    f"http://{PROXY_USER}:{PROXY_PASS}@"
    f"{PROXY_HOST}:{PROXY_PORT}"
)

proxies = {
    "http": proxy_url,
    "https": proxy_url,
}

target_url = "https://httpbin.org/ip"

for attempt in range(5):
    try:
        response = requests.get(
            target_url,
            proxies=proxies,
            timeout=10,
        )
        response.raise_for_status()

        exit_ip = response.json().get("origin", "unknown")
        print(f"Request {attempt + 1}: IP is {exit_ip}")

    except requests.exceptions.RequestException as error:
        print(f"Request {attempt + 1} failed: {error}")

The timeout prevents a request from waiting indefinitely. raise_for_status() turns unsuccessful HTTP responses into exceptions, while .get("origin", "unknown") avoids a KeyError if the response shape changes.

Use SOCKS5 when needed

Install Requests with SOCKS support:

python -m pip install 'requests[socks]'

Then change the proxy URL scheme:

proxy_url = (
    f"socks5h://{PROXY_USER}:{PROXY_PASS}@"
    f"{PROXY_HOST}:{PROXY_PORT}"
)

Using socks5h:// sends DNS resolution through the proxy. The rest of the request and error-handling structure stays the same.

Rotation, sticky sessions, and targeting

The unlimited plans provide access to a shared rotating residential pool advertised at 4M+ IPs. Rotation and sticky-session behavior depend on your session configuration and live exit availability, so a new or unique IP is not guaranteed for every request.

For country targeting, the gateway username can include a country parameter such as -country-us. Use the exact syntax supplied in your dashboard and verify availability for the location you need.

Practical troubleshooting

Confirm the gateway host, port, username, and password.
URL-encode special characters in credentials when necessary.
Keep timeouts enabled and add bounded retries with backoff.
Test through an IP-check endpoint before running a larger job.
Reduce concurrency when the target begins returning rate-limit responses.
Do not assume that every request must return a different exit IP.

Choosing bandwidth for larger workloads

Unlimited plans are priced by aggregate throughput and duration rather than by traffic volume. If you are comparing throughput tiers, use the residential proxy cost calculator or review the unlimited residential proxy plans.

For the maintained version of this tutorial, see the original Python proxy rotation guide.

DEV Community