Slow Down Before the 429: EWMA Latency Inflation as Your Real Politeness Signal

The standard polite-scraping recipe is a fixed per-origin rate limit, or a header-driven one: read X-RateLimit-Remaining, and when it looks low, slow down. Both are reactive. By the time you see a 429 or a Retry-After, you have already crossed the line; by the time the rate-limit header shows trouble, the origin is already throttling you. And many origins — most, honestly — send you no rate-limit headers at all, so you're flying blind until the wall arrives.

There is an earlier signal hiding in plain sight: the origin's response time. When a server starts shedding load or applying soft throttling to your traffic, its p50 latency creeps up before it ever returns a 429. A search page that normally answers in 180ms starts answering in 240, then 310. That drift is the origin telling you — politely, in the only channel it always has — that you are contributing to its load. The core idea of this article: track a per-origin exponentially weighted moving average of response latency and use deviation from baseline as a continuous congestion signal that scales your request rate down smoothly, long before an explicit 429 fires.

Why latency is the honest signal

Response time is the one metric that is always present, always per-origin, and always measured at the exit where it matters. Unlike headers (which origins may omit), unlike error codes (which are a lagging, binary confirmation), and unlike your own semaphore count (which knows nothing about the origin's mood), latency reflects the origin's actual capacity to serve you. Under load, servers queue; queues show up as latency; and the latency you observe through your proxy exit is the latency of the origin plus your contribution to its queue.

There's a subtlety that makes latency better than the rate-limit header even when the header is present: it's measured at the true point of contention. The header may report a quota per API key while the origin's real bottleneck is per-IP at the edge, or vice versa. Your observed latency is unambiguous — it's how long this request, through this exit, to this origin, actually took. That is the number the polite-scraping problem is really about.

The baseline problem: absolute latency is meaningless

A raw 300ms means nothing on its own. One origin is a fast CDN-cached blog (300ms is a crisis), another is a database-backed search API whose p50 is 450ms on a calm day (300ms is a gift). You can't hardcode thresholds across origins. The fix is to make the signal relative to each origin's own recent behavior.

Track two EWMA per origin: a fast-moving average (reacts within seconds, catches the onset of slowdown) and a slow-moving average (the "calm baseline," takes minutes to move). The congestion signal is the ratio fast / slow. When the origin is at its normal pace, the ratio hovers near 1.0. When it starts throttling you, fast climbs while slow lags, and the ratio rises.

import time
import threading

class OriginPacer:
    """Per-origin adaptive politeness controller driven by latency EWMA.

    - `fast` reacts quickly (alpha ~0.3): catches onset of congestion.
    - `slow` is the baseline (alpha ~0.02): what 'normal' looks like.
    - signal = fast / slow.  signal < ~1.15 -> send faster;
      signal > ~2.0 -> crawl.
    We turn the signal into a token-bucket rate with a smooth transfer
    function, so the request rate never steps abruptly (abrupt ramps are
    themselves a bot tell).
    """
    def __init__(self, origin: str, base_rps: float = 4.0, max_rps: float = 8.0,
                 min_rps: float = 0.3):
        self.origin = origin
        self.base_rps = base_rps
        self.max_rps = max_rps
        self.min_rps = min_rps
        self._fast = None
        self._slow = None
        self._last = time.monotonic()
        self._tokens = base_rps
        self._lock = threading.Lock()
        self._alpha_fast = 0.30
        self._alpha_slow = 0.02

    def observe(self, latency_ms: float) -> None:
        """Call once per *successful* response with its wall-clock latency."""
        with self._lock:
            if self._fast is None:          # seed
                self._fast = self._slow = latency_ms
                return
            self._fast += self._alpha_fast * (latency_ms - self._fast)
            self._slow += self._alpha_slow * (latency_ms - self._slow)

    def signal(self) -> float:
        if not self._slow:
            return 1.0
        return round(self._fast / self._slow, 3)

    def target_rate(self) -> float:
        """Smooth transfer function mapping congestion signal -> requests/sec."""
        s = self.signal()
        if s <= 1.15:                       # origin comfortable -> allow burst up to max
            r = self.max_rps
        elif s >= 2.5:                      # heavy congestion -> crawl
            r = self.min_rps
        else:
            # linear interpolation between the two knees
            t = (s - 1.15) / (2.5 - 1.15)
            r = self.max_rps + t * (self.min_rps - self.max_rps)
        return r

    def acquire(self) -> float:
        """Block (return seconds to sleep) so caller respects the current rate."""
        with self._lock:
            now = time.monotonic()
            self._tokens = min(self.target_rate(),
                               self._tokens + (now - self._last) * self.target_rate())
            self._last = now
            if self._tokens >= 1:
                self._tokens -= 1
                return 0.0
            need = (1 - self._tokens) / self.target_rate()
            self._tokens = 0
            return need

The transfer function is deliberately piecewise-linear rather than a hard if/else on thresholds. A scraper that clamps instantly from 8 rps to 0.3 rps creates a discontinuity in its own traffic shape — a step function that is easy for an origin to spot. Creeping down as latency climbs produces a smooth ramp that resembles organic load tapering off.

Latency vs the timeout trap

One engineering trap to be honest about: you cannot treat every high latency as congestion. Some origins are genuinely slow at certain hours (nightly batch jobs, cache-cold pages). Some are slow because your exit is slow — a congested residential gateway can add 800ms of pure proxy latency that has nothing to do with the origin's willingness to serve you. Feeding contaminated latency into the pacer means you throttle yourself for your provider's bad routing.

The fix is to subtract the exit's overhead. You already have a cheap way to measure it: your egress canary / echo endpoint that returns instantly. Observe latency to the echo through the same exit at the same time you observe the origin, and pace on the origin latency minus exit baseline, not raw latency.

async def timed_request(session, url, exit_url, echo_url, pacer: OriginPacer):
    t0 = time.monotonic()
    r = await session.get(url, proxy=exit_url)
    origin_ms = (time.monotonic() - t0) * 1000

    t1 = time.monotonic()
    await session.get(echo_url, proxy=exit_url, timeout=(2, 5))
    exit_ms = (time.monotonic() - t1) * 1000

    net_overhead = min(exit_ms, origin_ms * 0.5)   # don't over-subtract
    pacer.observe(origin_ms - net_overhead)
    return r

Now the pacer sees the origin's queue, not the proxy's congestion. When the proxy itself is the problem, you'll see it in a different place (a rising timeout rate on the echo call), and the correct response there is rotate the exit, not throttle the origin.

Making latency one input among several

A good politeness controller doesn't rely on a single signal; it combines a cheap proactive one (latency) with cheap reactive confirmations. A clean design is a small weighted state machine:

• Latency signal (proactive) sets the baseline rate, continuously.
• 429 / Retry-After (reactive) clamps the rate hard and puts the exit in cooldown for the stated duration, then resumes the latency-driven ramp.
• Soft-block signals (200-with-thin-body, "slow down" banners) nudge the target down one level even without a 429, since those are the origin's polite warning shot.
• robots.txt Crawl-delay / Request-rate act as a ceiling you never exceed regardless of what the other signals say — respecting the published limit is the floor of good manners, and Request-rate: 1/5 1/1m should cap you no matter how fast the origin feels.

The latency pacer is the dial; the others are hard stops and warnings layered on top. The point of putting latency first is that it's the only signal that scales you down continuously and early — a thermostat, where 429s are just the smoke alarm.

What you get in production

Teams that move from "fixed rate, then react to 429" to latency-driven pacing consistently report the same outcome: the tail of incidents collapses. They still occasionally see a 429, but the volume of requests that trigger the severe tier — permanent IP bans, CAPTCHA walls, account locks — drops, because those escalate from sustained overload, and latency-based pacing never lets your rate reach sustained overload. You spend fewer proxy-GB (a throttled, smooth crawl is cheaper than a burst-then-cooldown sawtooth), you look less like a bot (smooth ramps, organic tapering), and you get more successful rows out of the same quota because you stopped hitting the wall at all.

There's a philosophical shift worth naming: polite scraping has traditionally meant "obey the explicit rules the origin publishes." But most origins publish almost nothing. The honest engineering move is to treat the origin's behavior — how fast it answers you — as a rule that is always being published, in a channel you can always read. Slow down before you are told to, and you'll rarely be told at all.

---

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele