Exponential Backoff and Request Budgets for Trading APIs in Python

Trading APIs fail in a predictable rhythm: fine for hours, then a burst of 429s exactly when your bot is busiest. The fix is not "retry harder" — it is a request budget with jittered exponential backoff, enforced in one place.

This post shows the minimal pattern I use for Polymarket CLOB/Data API work: a token bucket for steady-state pacing, backoff with jitter for bursts, and a hard ceiling so one bad hour does not take the scanner offline all day. The design rationale and observed limits are documented separately: https://brainresearchhub.net/research/sdk/rate-limit-design-patterns

The three components

1. Token bucket — enforces average rate with burst allowance, centrally.
2. Backoff with jitter — handles 429s without re-synchronizing your workers.
3. Retry cap + kill switch hook — fails loud instead of hammering forever.

import time, random, threading

class TokenBucket:
    """rate = tokens per second; capacity = max burst."""
    def __init__(self, rate: float, capacity: float):
        self.rate, self.capacity = rate, capacity
        self.tokens, self.ts = capacity, time.monotonic()
        self.lock = threading.Lock()

    def take(self, n: float = 1.0) -> float:
        with self.lock:
            now = time.monotonic()
            self.tokens = min(self.capacity, self.tokens + (now - self.ts) * self.rate)
            self.ts = now
            if self.tokens >= n:
                self.tokens -= n
                return 0.0
            return (n - self.tokens) / self.rate


def call(fn, bucket: TokenBucket, max_retry: int = 6,
         base: float = 0.5, cap: float = 45.0):
    """fn() returns an object with .status_code and .headers (requests-style)."""
    for attempt in range(max_retry + 1):
        wait = bucket.take()
        if wait > 0:
            time.sleep(wait)
        resp = fn()
        if resp.status_code != 429:
            return resp
        if attempt == max_retry:
            raise RuntimeError(f"budget exhausted after {max_retry} retries")
        ra = resp.headers.get("Retry-After")
        delay = float(ra) if ra else min(cap, base * 2 ** (attempt + 1))
        delay += random.uniform(0, delay * 0.2)      # jitter: de-synchronize workers
        time.sleep(delay)

Wiring it in

import requests

bucket = TokenBucket(rate=5.0, capacity=20)   # 5 rps sustained, 20 burst

def get_markets(slug: str):
    return call(
        lambda: requests.get(
            f"https://data-api.polymarket.com/markets?slug={slug}"),
        bucket,
    )

Rules I follow that the code cannot enforce:

• One bucket per process, not per module. Two components each "within limits" is how you exceed limits.
• Never retry non-idempotent writes blind. An order POST that times out may have succeeded; reconcile from the order channel instead of re-posting.
• Subscribe instead of polling. The market WebSocket channel kills most of your REST load; the stream architecture is covered here: https://brainresearchhub.net/research/sdk/websocket-stream-architecture
• The retry loop calls the kill switch, not the logger. When retries exhaust, halt order placement: https://brainresearchhub.net/research/sdk/kill-switch-design

Why jitter is non-negotiable

Without jitter, all your workers that hit the 429 at the same moment retry on the same beat, re-synchronize, and trip the limit again. Uniform jitter in delay * (1 ± 0.2) breaks the lockstep. It looks trivial and it halves your ban windows in practice.

Background reading: the generic mechanics of rate limiting (token bucket, leaky bucket) are well covered on Wikipedia: https://en.wikipedia.org/wiki/Rate_limiting — and for a regulated-venue comparison, the Kalshi API guide: https://brainresearchhub.net/research/sdk/kalshi-api-guide

Educational purposes only. Not financial advice.