Build a Production-Ready Rate Limiter in 40 Lines of Python

I've shipped rate limiters three times in my career. The first one broke production. The second one made our biggest customer angry. The third one — finally — worked without anyone noticing it was there.

That last part is the goal. A good rate limiter is invisible. It protects your API without ever getting in the way of legitimate traffic.

Here's what I learned, plus working code you can actually use.

The mistake everyone makes first

Your first instinct is a fixed window:

def is_allowed(user_id: str, limit: int, window: int = 60) -> bool:
    key = f"rl:{user_id}:{int(time.time() // window)}"
    count = redis.incr(key)
    redis.expire(key, window, nx=True)
    return count <= limit

This looks correct. In a 60-second window, each user can make limit requests. The counter resets every minute.

The problem is window boundaries. A user can make limit requests in the last second of one window, then limit more in the first second of the next. That's 2 * limit requests in about two seconds.

If your limit is 100 requests/minute, a burst of 200 is usually fine. But if your limit is 10 requests/minute and you're protecting an expensive LLM endpoint, that double-burst is exactly what takes you down.

Sliding window: the fix that's worth the extra lines

Instead of resetting the counter at a boundary, count only the requests in the last 60 seconds.

import time
import redis

def rate_limited(user_id: str, limit: int, window: int) -> bool:
    """Return True if the request should be allowed."""
    now = time.time()
    key = f"rl:{user_id}"
    pipe = redis.pipeline()

    # Drop entries older than the window
    pipe.zremrangebyscore(key, 0, now - window)
    # Add the current request, scored by its timestamp
    pipe.zadd(key, {str(now): now})
    # Count what's left
    pipe.zcard(key)
    # Expire the key so we don't leak memory
    pipe.expire(key, window)

    _, _, count, _ = pipe.execute()
    return count <= limit

This is a sliding window log implemented with a Redis sorted set. Each request is a member scored by its timestamp. Old entries get dropped, and we count what remains.

It's precise, and it's O(n) in the number of requests per window — which is fine for almost every real-world API.

The nuance nobody writes about

A rate limiter that just returns 429 is a bad rate limiter. Real users hit limits for three reasons:

1. They're legitimate but busy — a batch job, a retry loop
2. They made a mistake — bad pagination, missing caching
3. They're abusing your API

You can't tell these apart by counting alone. What you can do is communicate.

Always send these headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1720000000

And in the 429 response body, tell them when to retry:

{
  "error": "rate_limited",
  "retry_after_seconds": 18
}

The first version I shipped returned a bare 429 Too Many Requests with no body. Support tickets followed. It took me a full day to realize users were retrying instantly in a loop, which made the problem worse.

Three mistakes I still see in the wild

1. Using 503 instead of 429. A 503 means "the server is down." It triggers alerting, retries with backoff, and occasionally auto-scaling. A 429 means "you're asking too fast." These are different problems with different fixes. If your rate limiter returns 503, your on-call engineer is going to hate you.

2. Rate limiting on the wrong key. Rate limit per user, not per IP. NAT means a whole office can share one IP, so a per-IP limit will throttle a classroom full of students using your API legitimately. If you do limit by IP, read the X-Forwarded-For header — and never trust it blindly, because clients can set it.

3. Hard-coding limits in the code. Limits are a product decision, not a code decision. Read them from config. A customer paying for a higher tier shouldn't need a code deploy to get it.

Token bucket: when you want to smooth, not just block

A sliding window answers one question: "has this user made too many requests in the last N seconds?" That's enough for most cases.

A token bucket answers a different question: "can this user burst briefly, as long as their long-term average stays under control?" Tokens refill at a steady rate, and a request costs one token. The bucket can hold a small surplus, so a user who's been idle can burst.

import time

def token_bucket_allows(user_id, capacity, refill_rate):
    key = f"tb:{user_id}"
    now = time.time()
    # Lua keeps the read-modify-write atomic
    script = """
    local tokens = tonumber(redis.call('get', KEYS[1]) or ARGV[1])
    if tokens < 1 then return 0 end
    redis.call('set', KEYS[1], tokens - 1)
    return 1
    """
    ...

I've genuinely needed this exactly twice. Both times it was a write-heavy endpoint where clients legitimately burst (uploading a batch of records) but had to average out over an hour. If you can't name that scenario for your API, stick with the sliding window. It's simpler and easier to reason about.

Putting it together

Here's a minimal decorator that wires the sliding-window function into any function:

from functools import wraps

class RateLimited(Exception):
    def __init__(self, retry_after: int):
        self.retry_after = retry_after
        super().__init__(f"rate limited, retry in {retry_after}s")

def rate_limit(limit: int, window: int):
    def decorator(fn):
        @wraps(fn)
        def wrapper(*args, **kwargs):
            user_id = kwargs.get("user_id", "anonymous")
            if not rate_limited(user_id, limit, window):
                raise RateLimited(retry_after=window)
            return fn(*args, **kwargs)
        return wrapper
    return decorator

@rate_limit(limit=100, window=60)
def expensive_endpoint(user_id: str):
    return {"ok": True, "user": user_id}

The point isn't the exact code — it's the model. Count precisely, communicate clearly, and make the limit configurable.

What I'd do differently next time

• Start with sliding window; skip fixed window entirely. The extra complexity is one Redis call.
• Ship the headers on day one. They cost nothing and save you support tickets.
• Log every 429. Those logs are your early warning that a customer is about to churn because their integration is stuck in a retry loop.

Rate limiting feels like an infrastructure detail until the moment it isn't. The moment it isn't is usually 3 AM.