Rate Limiting in Practice: 4 Algorithms and When to Use Them

Why rate limiting matters

Every API service eventually hits the rate limiting question. Whether you're protecting your service from being overwhelmed, ensuring fair access for all clients, or preventing runaway code from a single buggy integration, rate limiting is one of the most fundamental backend engineering topics.

I've worked on several API services over the years, including a search API at SerpBase, and the rate limiting decision is one of the most consequential architecture choices you make. Get it wrong, and you either over-protect (legitimate users get throttled) or under-protect (your service falls over).

This post walks through 4 common rate limiting algorithms, their trade-offs, and how to choose between them.

Algorithm 1: Fixed window

The simplest approach. Count requests per fixed time window (e.g., per second or per minute). If the count exceeds the limit, reject.

import time
import redis

def allow_request_fixed_window(client_id, max_per_sec=1):
    window = int(time.time())
    key = f"limit:{client_id}:{window}"
    if redis.get(key):
        return False
    redis.setex(key, 1, 1)
    return True

Pros: Trivial to implement. One line of Redis.

Cons: The boundary case. A client can hit max_per_sec requests at 0.999s of one window, then max_per_sec more at 1.001s of the next window — effectively 2x the limit in 1 second.

When to use: Coarse-grained limits where exact precision doesn't matter (e.g., "1000 requests per hour" for a public API).

Algorithm 2: Sliding window

Improves on the fixed window by tracking a sliding time period. Each request checks the count over the past N seconds, optionally split into sub-windows for efficiency.

def allow_request_sliding(client_id, window_sec=1, sub_windows=10, max_requests=10):
    now = time.time()
    current_sub = int(now * sub_windows) % sub_windows

    total = 0
    for i in range(sub_windows):
        sub_key = f"limit:{client_id}:{(current_sub - i) % sub_windows}"
        total += int(redis.get(sub_key) or 0)

    if total < max_requests:
        redis.setex(f"limit:{client_id}:{current_sub}", 2, 1)
        redis.incr(f"limit:{client_id}:{current_sub}")
        return True
    return False

Pros: Smooth limit enforcement without the boundary case.

Cons: More complex, multiple Redis keys per client, slightly more memory.

When to use: Strict API quota enforcement where exact precision matters.

Algorithm 3: Token bucket

Each client has a bucket that holds tokens. Tokens refill at a steady rate. Each request consumes one token. If the bucket is empty, the request is rejected.

def allow_request_token_bucket(client_id, capacity=100, refill_rate=10):
    key = f"bucket:{client_id}"
    bucket = redis.hgetall(key)
    tokens = float(bucket.get("tokens", capacity))
    last_refill = float(bucket.get("last_refill", time.time()))

    elapsed = time.time() - last_refill
    tokens = min(capacity, tokens + elapsed * refill_rate)

    if tokens >= 1:
        tokens -= 1
        redis.hset(key, mapping={"tokens": tokens, "last_refill": time.time()})
        redis.expire(key, 3600)
        return True

    redis.hset(key, mapping={"tokens": tokens, "last_refill": time.time()})
    return False

Pros: Allows burst (a full bucket means up to capacity requests in a single moment) while the long-term average stays at refill_rate.

Cons: State needs persistence. In-process works for a single server; you need Redis (or similar) for a distributed system.

When to use: B2B APIs where clients occasionally need to burst (batch jobs, migrations, end-of-day syncs) but you still want to enforce a long-term average.

Algorithm 4: Leaky bucket

The mirror image of token bucket. Requests enter a queue, and the queue is processed at a steady rate. If the queue is full, the request is rejected.

import collections
import time

def leaky_bucket(capacity=100, rate=10):
    queue = collections.deque()

    def allow_request():
        now = time.time()
        # Process queued requests at the configured rate
        while queue and now - queue[0] >= 1.0 / rate:
            queue.popleft()

        if len(queue) < capacity:
            queue.append(now)
            return True
        return False

    return allow_request

Pros: Smooth, predictable downstream load.

Cons: No burst support. Even small bursts get queued.

When to use: Protecting downstream services that need predictable load (databases, payment processors, anything with limited throughput).

Choosing the right algorithm

Here's the decision matrix I use:

Use case	Algorithm
Public API, coarse limit ("1000/hour")	Fixed window
Strict API quota, exact precision	Sliding window
B2B API with occasional bursts	Token bucket
Protecting downstream service	Leaky bucket

At SerpBase, we serve a developer-facing search API where clients occasionally run batch jobs (a flurry of requests in a few seconds, then nothing for hours). Token bucket was the right call: fixed window had too many boundary issues, sliding window was overkill for a developer-friendly API, and leaky bucket would have forced our clients to throttle their batch jobs in ways that hurt their productivity.

4 common pitfalls

1. Confusing rate limits with quotas. Rate = "N requests per second". Quota = "N requests per month". Different concepts, different enforcement. Many APIs conflate them and surprise users.

2. Not returning Retry-After headers. When you throttle a client, tell them when to come back. Otherwise they retry blindly and amplify the problem. A 429 response without Retry-After is just "no" with extra steps.

3. Using in-process state in a distributed system. Each API node has its own counter. A client routed to 4 different nodes can effectively get 4x the limit. Always store rate limit state in a shared store (Redis is the standard choice).

4. Not monitoring rate limit triggers. If a client keeps hitting the limit, they probably have a bug or a misunderstanding. Reach out before they churn or write a frustrated blog post about your API.

Closing thoughts

Rate limiting looks simple but has surprising depth. The right algorithm depends on your traffic pattern, your client base, and what you're protecting against.

If you're building a new API service, start with fixed window for simplicity, then upgrade to token bucket or sliding window when you hit edge cases. Most production APIs I've seen end up at token bucket for B2B scenarios, with leaky bucket layered in front of any fragile downstream service.

For 429 response design — which headers to return, how to structure the body — the IETF draft draft-ietf-httpapi-ratelimit-headers is a good reference. Implementing it well is a small thing that makes your API feel professional to clients integrating against it.