Timeouts Are an Exit Property: Per-Proxy Time Budgets in asyncio

Every scraping framework I have reviewed ships the same line somewhere near the top of its request layer:

timeout = aiohttp.ClientTimeout(total=30)

It is copied from a blog post, it survives code review because thirty seconds feels "generous but not crazy," and then it quietly decides the economics of your whole pipeline. The problem is not the number. The problem is the shape of the abstraction. A single global timeout treats every request as if it exits your network from the same place. The moment your traffic rotates through residential IPs, that assumption is false: each request leaves through a different exit, and exits fail in ways that origins do not.

Timeouts are an exit property. The budget belongs to the exit — or to the exit class you can actually observe: geography, ASN, carrier tier — and it should be learned from your own connect-time data, not inherited from someone else's defaults page.

Why fresh residential exits have fat tails

A datacenter IP connects to an origin over a few routed hops with a stable RTT. A residential exit is a different animal. Your CONNECT request traverses carrier-grade NAT, lands on a mobile gateway or a home router behind a shared uplink, then performs a full TLS handshake through that relay path. Add contention — you are one of many tenants sharing that IP pool — and the distribution of connect times stops looking Gaussian.

In practice this means something like: p50 connect through a fiber residential exit around 300ms, p95 maybe a second, and p99 through a mobile exit stretching into several seconds. Those numbers are illustrative, but the shape is not. The right tail is heavy, and it is populated by two very different kinds of requests:

• Slow but healthy exits that will deliver a good response if you let them finish the handshake.
• Dead or saturated exits that will never deliver anything, and will sit there burning a worker slot until your timeout says stop.

One fixed timeout cannot serve both. Set it low and you amputate healthy mobile exits, throwing away paid GB and good data. Set it high and you wait seconds for corpses. The only way out is to stop guessing a single number and instead maintain a budget per exit class, derived from what those classes actually did on your last few thousand requests.

The anatomy of aiohttp's ClientTimeout

aiohttp.ClientTimeout has four fields and most people use one:

Field	Covers	Fires when
total	Entire request lifecycle	Anything takes too long overall
connect	TCP + TLS + proxy CONNECT phase	The path to the origin is the bottleneck
sock_connect	Socket connect only (no proxy negotiation budget)	Raw socket establishment stalls
sock_read	Gap between reads once the socket is open	Origin is slow to send bytes

The distinction that matters for proxy work is connect versus sock_read, and the reason to bound them separately is that they diagnose different faults:

• A connect timeout almost always means the exit is dead, saturated, or blocked by the target. The origin was never meaningfully involved.
• A sock_read timeout means bytes crossed the wire and then stalled — usually a slow origin, rate limiting, or a heavyweight page, with an exit that may be perfectly fine.

A total=30 collapses these into one indistinguishable event. You lose the signal that tells you whether to rotate or to back off. Separate the budgets: a tight, per-exit-class connect, a looser sock_read, and a total that is the sum of your intentions, not a round number from Stack Overflow.

Learning budgets from your own data

Here is a controller that keeps a rolling window of connect times per exit class and computes a budget from its recent p95. The exit class is whatever you can observe cheaply — country code, ASN from your provider's metadata, or mobile/fiber tier.

import asyncio
import math
import random
import time
from collections import deque
from dataclasses import dataclass, field

@dataclass
class ExitProfile:
    """Rolling window of recent connect times (seconds) for one exit class."""
    window_size: int = 50
    connect_times: deque = field(default_factory=lambda: deque(maxlen=50))

    def record_connect(self, elapsed: float) -> None:
        self.connect_times.append(elapsed)

    def p95(self) -> float | None:
        if len(self.connect_times) < 10:      # not enough evidence yet
            return None
        samples = sorted(self.connect_times)
        idx = math.ceil(0.95 * len(samples)) - 1
        return samples[idx]


@dataclass
class TimeoutController:
    floor: float = 0.8    # never below this: protects slow-but-healthy exits
    cap: float = 8.0      # never above this: dead exits get cut off
    k: float = 1.6        # slack multiplier over p95
    profiles: dict[str, ExitProfile] = field(default_factory=dict)

    def profile(self, exit_class: str) -> ExitProfile:
        return self.profiles.setdefault(exit_class, ExitProfile())

    def connect_budget(self, exit_class: str) -> float:
        p = self.profile(exit_class)
        est = p.p95()
        if est is None:
            # Cold start: use a conservative probe budget and learn.
            return min(self.cap, max(self.floor, 2.0))
        return max(self.floor, min(self.cap, self.k * est))

    def timeouts(self, exit_class: str) -> "tuple[aiohttp.ClientTimeout, float]":
        connect = self.connect_budget(exit_class)
        return (
            aiohttp.ClientTimeout(
                total=connect + 20.0,   # connect phase budget + generous read phase
                connect=connect,
                sock_read=15.0,
            ),
            connect,
        )

The knobs are deliberate. The floor exists because a class whose recent traffic happened to be fast should not be allowed to strangle its own tail — one lucky window should not sentence the next unlucky handshake to death. The cap is where you decide how long you are willing to wait on a corpse, and it should be far below 30 seconds for rotating exits. The k multiplier is your risk appetite: lower it to fail faster and retry more, raise it if retries are expensive relative to waiting (say, a metered pay-per-GB plan where a retry after a connect timeout is cheap but a wasted download is not).

Now wire it into the fetch routine. Timings are recorded per exit class, and the budget for the next request through that class reflects what just happened:

import aiohttp

CONTROLLER = TimeoutController()

async def fetch(session: aiohttp.ClientSession,
                url: str,
                proxy: str,
                exit_class: str) -> str | None:
    timeout, connect_budget = CONTROLLER.timeouts(exit_class)
    t0 = time.perf_counter()
    try:
        async with session.get(url, proxy=proxy, timeout=timeout) as resp:
            connect_elapsed = time.perf_counter() - t0
            # Approximation: this includes TTFB, so only record it when the
            # response headers arrived; refine by splitting at resp.start_tls
            # or using a Trace if you need true connect-only timings.
            CONTROLLER.profile(exit_class).record_connect(connect_elapsed)
            if resp.status == 200:
                return await resp.text()
            return None
    except asyncio.TimeoutError:
        failed_at = time.perf_counter() - t0
        if failed_at <= connect_budget * 1.05:
            # Died inside the connect window => exit problem, cheap to retry
            # elsewhere. Do NOT record as a connect time; record as a fault.
            log_event("connect_timeout", exit_class, proxy)
        else:
            # Passed connect, stalled reading => origin problem.
            log_event("read_timeout", exit_class, proxy)
        return None

async def log_event(kind, exit_class, proxy):
    ...  # your metrics sink

async def crawl(urls, proxies):
    connector = aiohttp.TCPConnector(limit=200)
    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = [
            fetch(session, url, random.choice(proxies), classify(url, proxies))
            for url in urls
        ]
        return await asyncio.gather(*tasks)

(For production-grade separation of connect from read, use aiohttp.Trace with on_connection_create_start/end hooks — the sketch above is honest about being an approximation, and the approximation is good enough to drive per-class budgets.)

Different failures deserve different remedies

The reason connect and sock_read earn separate budgets is that their correct remedies are opposites:

Event	Likely cause	Right response	Wrong response
Connect timeout	Dead/saturated/blocked exit	Rotate to a different exit immediately; retry cheaply	Back off; the origin never saw you
sock_read timeout	Slow origin, rate limit, heavy page	Keep exit class, retry same exit or back off per-origin	Fan out across exits — burns GB
TLS reset post-connect	Fingerprint or partial block	Change request profile, not necessarily exit	Lower timeouts (hides it)

Retrying a read timeout across five fresh residential exits does not make the origin faster. It makes five different people's home connections download a partial page you throw away, and it multiplies your bandwidth bill by the retry count while leaving your success rate untouched. The exit is innocent; fire the exit only when the connect phase is what failed.

What adaptive budgets do to throughput

This is where the argument stops being about elegance and becomes about worker-seconds. Suppose you run 200 concurrent requests and 1% of your exits are dead, with a global total=30. Those dead exits sit on a worker slot for the full thirty seconds each. At even a modest dead-exit rate, slow requests dominate aggregate worker time — roughly, with 1% of requests taking 30s and 99% taking 1s, average latency is 0.99 × 1s + 0.01 × 30s ≈ 1.3s, but the variance means your effective throughput in the tail stalls: during any 30-second window a few percent of your pool is frozen, and as you raise concurrency to compensate, you mostly raise the number of slots frozen waiting on corpses. (Illustrative arithmetic, but the asymmetry is real: the mean hides that the tail owns your capacity.)

Now swap in the controller. Healthy us-mobile class learns a 2.5s cap; a dead exit inside that class is evicted from your worker pool 12× faster than the global timeout allows. Failing fast is not pessimism — under a fixed pool, every second a dead exit holds is a second a good request cannot start. Fast failure converts dead exits from capacity sinks into cheap retries on a different exit, and only then does raising concurrency actually convert into raising pages-per-hour. You can push limit upward precisely because the budget on each exit class is tight where it should be tight and generous where generosity pays — a single global number can never be both, which is why the naive version forces you to run conservatively at all times.

The mental model to keep: you are not tuning "how long a request takes." You are allocating an error budget per exit class, deciding how much patience each class has earned, and routing each timeout event to the remedy — rotate, back off, or re-profile — that matches what actually broke. Copy timeout=30 from a blog and you have outsourced that decision to someone whose traffic never left a datacenter.

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele