DEV Community: Squid Proxies

Proxy Authentication & Session Persistence in Python: Sticky Sessions, Rotating Sessions, and When to Use Each

Squid Proxies — Wed, 16 Sep 2026 15:07:16 +0000

Most proxy tutorials cover one thing: how to rotate. Fewer cover the harder question: when not to rotate. If you're logging into a site, filling a multi-step form, or building up a cart before checkout, swapping your IP mid-flow can look exactly like account takeover to the site you're hitting, and you'll get flagged or logged out.

This post covers both authentication mechanics and the rotating-vs-sticky decision, since they usually come up together. Code samples below use <PROXY_HOST>:<PROXY_PORT> placeholders, swap in credentials from whatever provider you're using; the examples assume a standard user:pass@host:port format, which is what most providers expect.

Two ways to authenticate a proxy

Credentials in the URL is the simplest and most common approach:

import requests

proxy = "http://username:password@<PROXY_HOST>:<PROXY_PORT>"
proxies = {"http": proxy, "https": proxy}

resp = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
print(resp.json())

requests parses the username:password@ portion automatically and handles the handshake for you.

The Proxy-Authorization header is the alternative, and some proxy providers or corporate setups require it explicitly rather than accepting inline credentials:

import base64
import requests

def build_proxy_auth_header(username, password):
    credentials = base64.b64encode(f"{username}:{password}".encode()).decode()
    return {"Proxy-Authorization": f"Basic {credentials}"}

proxy = "http://<PROXY_HOST>:<PROXY_PORT>"
proxies = {"http": proxy, "https": proxy}
headers = build_proxy_auth_header("username", "password")

resp = requests.get("https://httpbin.org/ip", proxies=proxies, headers=headers, timeout=10)

Functionally these do the same thing, both send Basic auth to the proxy, but if inline credentials ever silently fail against a particular proxy setup, the header version is the fallback worth trying.

Rotating vs. sticky: the actual decision

	Rotating (new proxy per request)	Sticky (same proxy for a duration)
Use for	Scraping many independent pages, price checks, rank tracking	Logins, multi-step checkouts, anything with server-side session state
Why	Spreads load, avoids per-IP rate limits	Consistent IP across a flow avoids triggering "session hijack" style detection
Risk if you get it backwards	Sticking to one proxy for bulk scraping gets that IP rate-limited fast	Rotating mid-login or mid-checkout gets the session invalidated or flagged

The rule of thumb: rotate when each request stands alone, stay sticky when requests belong to the same logical session.

Building a sticky session manager

The pattern below maps an arbitrary "session key" — a user ID, task ID, or account handle, whatever makes sense for your workload, to a fixed proxy for a configurable time-to-live. Once the TTL expires, it gets reassigned:

import time
import random

class StickySessionManager:
    def __init__(self, proxies, ttl_seconds=600):
        self.proxies = proxies
        self.ttl_seconds = ttl_seconds
        self._assignments = {}  # session_key -> (proxy, assigned_at)

    def get_proxy(self, session_key):
        now = time.time()
        assignment = self._assignments.get(session_key)

        if assignment:
            proxy, assigned_at = assignment
            if now - assigned_at < self.ttl_seconds:
                return proxy

        proxy = random.choice(self.proxies)
        self._assignments[session_key] = (proxy, now)
        return proxy

    def release(self, session_key):
        self._assignments.pop(session_key, None)

Calling get_proxy("user-42") twice within the TTL window returns the same proxy both times. Call release("user-42") once that flow is done (checkout completed, login task finished) to free the assignment up rather than waiting out the full TTL.

Wiring it into requests.Session

Combine the manager with requests.Session so cookies persist alongside the fixed proxy, this is what actually keeps a login or cart flow coherent end to end:

import requests

PROXIES = [
    "http://user:pass@proxy1.example.com:8000",
    "http://user:pass@proxy2.example.com:8000",
    "http://user:pass@proxy3.example.com:8000",
]

sticky = StickySessionManager(PROXIES, ttl_seconds=600)

def fetch_with_sticky_session(session_key, url, **kwargs):
    proxy = sticky.get_proxy(session_key)
    proxies = {"http": proxy, "https": proxy}
    session = requests.Session()
    session.proxies = proxies
    return session.get(url, **kwargs)

# Login, cart, and checkout for "user-42" all hit the same proxy
resp1 = fetch_with_sticky_session("user-42", "https://example.com/login")
resp2 = fetch_with_sticky_session("user-42", "https://example.com/cart")
resp3 = fetch_with_sticky_session("user-42", "https://example.com/checkout")

sticky.release("user-42")

One thing worth flagging: creating a fresh requests.Session() per call in this example means cookies aren't actually shared across resp1/resp2/resp3 above, only the proxy assignment is. If you need both a persistent proxy and persistent cookies across a flow, keep one Session object alive for the duration of that session key instead of creating a new one per call, and swap its .proxies only when the TTL expires.

When TTL should match your workload, not a default

600 seconds is a reasonable starting point, but the right TTL depends on what you're doing:

Login + immediate action: short TTL (60–120s) is often enough
Multi-step checkout flows: match your longest expected flow duration, with some buffer
Long-running authenticated sessions: consider extending the TTL on each successful request instead of using a fixed expiry, so an active session doesn't get reassigned mid-use

Wrapping up

Authentication is the easy half of this, inline credentials or a header, pick whichever your proxy setup accepts. The part worth actually thinking through is matching your proxy strategy to the shape of your traffic: rotate for independent requests, stay sticky for anything that depends on continuity. Getting this backwards is a common reason "the proxies are fine but I keep getting logged out" bugs show up.

If you're shopping for a provider that supports both patterns cleanly, Squid Proxies' docs and pricing are worth a look, residential and datacenter proxies on one dashboard, no separate accounts to juggle for sticky vs. rotating use cases.

Rotating Proxies with Python Requests: A Copy-Paste Starter Kit

Squid Proxies — Mon, 07 Sep 2026 11:38:30 +0000

If you're scraping, monitoring prices, or checking rank data at any real volume, a single IP will get rate-limited or blocked fast. Rotating proxies across requests is the standard fix, but most tutorials either hand-wave the retry logic or skip authentication entirely.

This is a copy-paste starter kit: four increasingly useful patterns for rotating proxies with Python's requests library, from "just make it work" to "make it survive failures."

Prerequisites

pip install requests

You'll need a pool of proxy addresses. Any provider that gives you host:port (or user:pass@host:port for authenticated proxies) works with the patterns below — for these examples we're using the format <PROXY_HOST>:<PROXY_PORT>, which is what you'd swap in from a provider like Squid Proxies.

1. The absolute minimum: one proxy, one request

import requests

proxy = "http://<PROXY_HOST>:<PROXY_PORT>"

proxies = {
    "http": proxy,
    "https": proxy,
}

response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
print(response.json())

That's it, requests routes both HTTP and HTTPS traffic through the same proxy dict. If your proxy requires auth, embed the credentials directly in the URL:

proxy = "http://username:password@<PROXY_HOST>:<PROXY_PORT>"

2. Rotating across a pool

One proxy isn't rotation, it's just a detour. Here's a pool with round-robin cycling using itertools.cycle:

import requests
from itertools import cycle

PROXIES = [
    "http://user:pass@proxy1.squidproxies.com:8000",
    "http://user:pass@proxy2.squidproxies.com:8000",
    "http://user:pass@proxy3.squidproxies.com:8000",
]

def get_proxy_pool(proxies):
    return cycle(proxies)

pool = get_proxy_pool(PROXIES)

for _ in range(5):
    proxy = next(pool)
    proxies = {"http": proxy, "https": proxy}
    resp = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
    print(proxy, "->", resp.json())

cycle() loops the list indefinitely, so next(pool) always gives you the next proxy in sequence: proxy1, proxy2, proxy3, proxy1, proxy2, ...

If you'd rather pick randomly than in sequence (useful for avoiding predictable request-timing patterns), swap in random.choice(PROXIES) instead.

3. Rotation with retries and backoff

The real reason to rotate is that individual proxies fail, they get temporarily blocked, time out, or hit a rate limit on the target site. A rotation pattern without retry logic just fails louder. Here's a version that retries through the pool on failure, with exponential backoff between attempts:

import time
import random
import requests

def make_request(url, proxy, timeout=10):
    proxies = {"http": proxy, "https": proxy}
    try:
        resp = requests.get(url, proxies=proxies, timeout=timeout)
        resp.raise_for_status()
        return resp
    except requests.exceptions.RequestException as e:
        print(f"Request failed via {proxy}: {e}")
        return None

def fetch_with_rotation(url, proxies, max_retries=3):
    tried = []
    for attempt in range(max_retries):
        proxy = random.choice(proxies)
        tried.append(proxy)
        resp = make_request(url, proxy)
        if resp is not None:
            return resp
        time.sleep(2 ** attempt)  # 1s, 2s, 4s...
    raise RuntimeError(f"All {max_retries} attempts failed. Tried: {tried}")

response = fetch_with_rotation("https://httpbin.org/ip", PROXIES)
print(response.json())

A few things worth calling out:

raise_for_status() turns HTTP error codes (403, 429, 503) into exceptions, so a "successful" request that actually got blocked doesn't silently pass through as a 200.
Exponential backoff (2 ** attempt) gives a rate-limited proxy a moment to cool down instead of hammering it again immediately.
tried in the error message makes debugging which proxies are consistently failing much faster than a bare "all retries failed."

4. A reusable session class

If you're making many requests in the same script, wrapping this in a class keeps connection pooling intact (via requests.Session) while still rotating the proxy per call:

import random
import requests

class RotatingProxySession:
    def __init__(self, proxies):
        self.proxies = proxies
        self.session = requests.Session()

    def get(self, url, **kwargs):
        proxy = random.choice(self.proxies)
        self.session.proxies = {"http": proxy, "https": proxy}
        return self.session.get(url, **kwargs)

    def post(self, url, **kwargs):
        proxy = random.choice(self.proxies)
        self.session.proxies = {"http": proxy, "https": proxy}
        return self.session.post(url, **kwargs)


client = RotatingProxySession(PROXIES)

for i in range(5):
    resp = client.get("https://httpbin.org/ip")
    print(f"Request {i}: {resp.json()}")

This is the pattern I'd actually reach for in a real scraping job, it's a small enough abstraction that you can drop in retry logic from section 3, custom headers, or per-domain proxy assignment without rewriting the calling code.

Wrapping up

Rotation solves the "one IP gets blocked" problem; retries with backoff solve the "one proxy in the pool is temporarily bad" problem. You want both, not one or the other.

If you're pulling proxies from a provider dashboard rather than hardcoding a list, most APIs return the pool as JSON, you can drop that straight into the PROXIES list in any of the examples above.

Next up in this series: the same patterns in async Python with aiohttp, for when synchronous requests become the bottleneck.