If your integration is throwing 429 Too Many Requests, adding a retry loop usually makes it worse: every retry is another request against the same quota, and a fleet of workers all retrying at the same interval turns a small overage into a synchronized stampede. The fix is to stop sending too fast in the first place — a token bucket in front of the client — and to treat the server's Retry-After header as the authority whenever you do get throttled. Asking the vendor for a quota increase should be the last step, not the first.
This is the thing I get wrong most often when I move fast: I wire up an API, hit the limit under real traffic, wrap the call in try/except with a sleep(1), and ship it. It works in staging with one worker and falls apart the moment two processes run concurrently.
Why does adding retries make 429s worse?
Three failure modes, in the order I usually hit them.
Retries spend quota. A naive retry loop on a rate-limited endpoint sends attempt after attempt into a bucket that is already empty. If the limiter counts rejected requests (many do — the rejection still costs the gateway work), your retries actively delay recovery.
Fixed sleeps synchronize. Every worker that hits the limit at 12:00:00 and sleeps exactly one second wakes at 12:00:01 together. You've built a metronome. This is the classic thundering herd, and it's why jitter isn't optional decoration.
Retrying non-idempotent calls duplicates work. A POST /charges that returns 429 might have been rejected before any work happened — or it might have been rejected by a downstream limiter after the charge was created. Without an idempotency key, a retry is a coin flip on double-charging someone.
The takeaway: retries are the recovery path, not the rate control; if retries are your only rate control, you don't have any.
What should the client actually do with Retry-After?
Read it, and parse both forms. Retry-After is allowed to be either a delay in seconds or an HTTP-date, and I have been bitten by a client that assumed seconds, got a date string, and threw a ValueError inside its own error handler.
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
def retry_after_seconds(header: str | None, fallback: float) -> float:
"""Parse Retry-After (delay-seconds OR HTTP-date). Falls back on garbage."""
if not header:
return fallback
header = header.strip()
try:
return max(0.0, float(header))
except ValueError:
pass
try:
when = parsedate_to_datetime(header)
except (TypeError, ValueError):
return fallback
if when.tzinfo is None:
when = when.replace(tzinfo=timezone.utc)
return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())
Clamp the result before you use it. A server that is having a bad day can return Retry-After: 3600, and a worker that blindly sleeps for an hour looks exactly like a hung process to whatever is watching it. I cap at something like 60 seconds and let the job fail into a retry queue beyond that.
Beyond Retry-After, what you get is vendor-specific. Many APIs expose X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset; the IETF has been working on standardizing RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset, but as of mid-2026 that is still a draft, so read the docs for the specific API rather than assuming the header names. When Remaining is exposed, feeding it back into your limiter is the single cheapest improvement you can make — you stop guessing at the budget and start reading it.
The takeaway: treat Retry-After as authoritative but bounded, and treat every other rate-limit header as vendor-specific until you've read the docs.
How do I limit the client side without over-engineering it?
A token bucket is about twenty lines and covers most cases. It allows a burst up to the bucket capacity, then settles into a steady rate — which is what most APIs actually enforce.
import asyncio
import time
class TokenBucket:
def __init__(self, rate_per_sec: float, burst: int):
self.rate = rate_per_sec
self.capacity = float(burst)
self.tokens = float(burst)
self.updated = time.monotonic()
self._lock = asyncio.Lock()
async def acquire(self, tokens: float = 1.0) -> None:
while True:
async with self._lock:
now = time.monotonic()
self.tokens = min(
self.capacity, self.tokens + (now - self.updated) * self.rate
)
self.updated = now
if self.tokens >= tokens:
self.tokens -= tokens
return
wait = (tokens - self.tokens) / self.rate
await asyncio.sleep(wait)
Use time.monotonic(), not time.time(). A clock adjustment (NTP step, container migration) can move wall-clock time backwards, and a limiter that computes a negative elapsed interval will either stall or hand out free tokens.
One honest limitation: this implementation is not FIFO. Waiters wake up and re-race for the lock, so under heavy contention a request can wait longer than others that arrived after it. For a background worker that's fine. If you need fairness guarantees for user-facing latency, hand out ordered tickets instead of re-looping.
Wiring it together with retries, jitter, and a bounded sleep:
import random
import httpx
RETRYABLE = {429, 500, 502, 503, 504}
async def request_with_retry(
client: httpx.AsyncClient,
bucket: TokenBucket,
method: str,
url: str,
*,
max_attempts: int = 5,
**kwargs,
) -> httpx.Response:
for attempt in range(1, max_attempts + 1):
await bucket.acquire()
resp = await client.request(method, url, **kwargs)
if resp.status_code not in RETRYABLE or attempt == max_attempts:
return resp
backoff = min(60.0, 0.5 * 2 ** (attempt - 1))
delay = min(60.0, retry_after_seconds(resp.headers.get("Retry-After"), backoff))
await asyncio.sleep(delay + random.uniform(0.0, 0.3 * delay + 0.1))
raise AssertionError("unreachable")
Only retry POST/PATCH through this path if the request carries an idempotency key the server honors. Otherwise restrict retries to GET/PUT/DELETE and let write failures surface.
The takeaway: a token bucket plus exponential backoff with jitter, bounded by a hard ceiling, handles the overwhelming majority of third-party API throttling.
Which limiting strategy fits which problem?
| Strategy | What it controls | Burst behavior | Needs shared state | Use it when |
|---|---|---|---|---|
| Fixed window counter | Requests per calendar window | Allows 2× at window boundaries | Yes (if multi-process) | Simplest server-side enforcement |
| Sliding window log | Requests over a rolling window | Accurate, no boundary spike | Yes, and it's memory-hungry | You must match a strict rolling quota |
| Token bucket | Sustained rate + burst size | Burst up to capacity, then steady | Only for multi-process | Client-side pacing of a third-party API |
| Concurrency semaphore | In-flight requests | Unbounded rate, bounded parallelism | Per-process usually fine | The limit is really about connections or memory |
The distinction people miss: a concurrency limit and a rate limit are different constraints. Ten concurrent requests that each take 50ms is roughly 200 requests/second; ten concurrent requests that each take 5 seconds is 2 requests/second. If the API's limit is expressed in requests per second, a semaphore will not save you.
The takeaway: pick the strategy that matches how the limit is expressed, not the one that's easiest to implement.
What changes when you run more than one worker?
An in-process bucket controls one process. Run four replicas and you send four times your intended rate. Once you're multi-process, the counter has to live somewhere shared, and the usual answer is Redis with a small Lua script so the check-and-increment is atomic:
-- KEYS[1] = bucket key, ARGV[1] = limit, ARGV[2] = window in ms
local count = redis.call('INCR', KEYS[1])
if count == 1 then
redis.call('PEXPIRE', KEYS[1], ARGV[2])
end
if count > tonumber(ARGV[1]) then
return {0, redis.call('PTTL', KEYS[1])}
end
return {1, 0}
That's a fixed window, so it permits up to 2× the limit across a window boundary — acceptable when your client-side target is set below the real quota, not when you're pacing right at the ceiling. If you need a shared limiter from serverless functions where holding a TCP connection is awkward, Upstash Redis is the one that speaks HTTP and includes a rate-limiting helper, at the cost of per-request latency you wouldn't pay with a co-located Redis. If the traffic you need to shape is inbound rather than outbound, push the limit to the edge instead: Kong and Envoy both enforce rate limits at the proxy layer, which keeps the logic out of every service that sits behind them.
The takeaway: the moment you scale past one process, your rate limiter needs shared state or it silently stops being a limit.
FAQ
What does HTTP 429 Too Many Requests mean?
It means the server accepted your request as well-formed and authenticated, but you exceeded a rate limit, so it refused to process it. It is a client-side pacing problem, not an error in the request itself, and the response often carries a Retry-After header telling you how long to wait.
Should I automatically retry after a 429?
Yes, but only with backoff plus jitter, only up to a small number of attempts, and only for requests that are safe to repeat. Retrying a non-idempotent write without an idempotency key risks duplicating the operation.
Why am I still getting 429s after adding a rate limiter?
Almost always because the limiter is per-process and you're running multiple workers, or because the limit is enforced per-endpoint or per-resource rather than globally. Check whether your effective rate is your configured rate multiplied by your replica count.
Bottom line
If you're seeing 429s, put a token bucket in front of the client and set it a comfortable margin below the documented quota before you touch anything else. Honor Retry-After when it's present, clamp it so a bad value can't hang a worker, and add jitter so your fleet doesn't retry in lockstep. Move the counter into Redis the moment you run more than one replica, and push enforcement to a proxy like Kong or Envoy if you're limiting inbound traffic across many services. Ask for a quota increase only after your own numbers show you're pacing correctly and still hitting the ceiling.
Top comments (0)