Rate Limiting Basics Every Developer Should Know

What is Rate Limiting?

Rate limiting controls how many requests a client can make to an API or service within a given time window. It prevents abuse, ensures fair usage, and protects backend resources from being overwhelmed. Think of it as a bouncer at a club: only a certain number of people get in per minute.

Why You Need It

Without rate limiting, a single misbehaving client can degrade performance for everyone. Common scenarios:

• A buggy client sends thousands of requests per second.
• A malicious actor tries to brute-force endpoints.
• A scraper hammers your server for data.

Rate limiting mitigates these issues and helps maintain service stability.

Common Algorithms

Token Bucket

A bucket holds tokens, each token allows one request. Tokens are added at a fixed rate (e.g., 10 per second). If the bucket is empty, requests are rejected. This allows bursts up to the bucket size.

import time

class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate = rate
        self.capacity = capacity
        self.tokens = capacity
        self.last_refill = time.monotonic()

    def allow(self):
        now = time.monotonic()
        elapsed = now - self.last_refill
        self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
        self.last_refill = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

Leaky Bucket

Requests fill a bucket that leaks at a constant rate. If the bucket overflows, requests are throttled. This smooths out bursts and enforces a steady outflow.

Fixed Window

Count requests in a fixed time window (e.g., 100 requests per minute). Simple but can allow bursts at window boundaries.

Sliding Window Log

Track timestamps of requests. Count requests within the last N seconds. More accurate than fixed window but more memory intensive.

Practical Implementation with Redis

Redis is a popular choice for distributed rate limiting. Use INCR and EXPIRE for a simple fixed window:

import redis
import time

r = redis.Redis()

def is_rate_limited(client_id, limit=100, window=60):
    key = f"rate_limit:{client_id}:{int(time.time() // window)}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window)
    return count > limit

For sliding window, use sorted sets to track timestamps.

HTTP Headers

Communicate limits to clients via standard headers:

• X-RateLimit-Limit: maximum requests per window
• X-RateLimit-Remaining: remaining requests in current window
• X-RateLimit-Reset: time when the window resets (Unix timestamp)

Example response:

HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-Reset: 1620000000

When exceeded, return 429 Too Many Requests with a Retry-After header.

Handling on the Client Side

As a client, respect rate limits:

• Parse rate limit headers.
• Implement exponential backoff.
• Cache responses when possible.
• Queue requests if you anticipate hitting limits.

Common Pitfalls

• Using IP alone: Behind NAT, many users share an IP. Consider user-based limits (API keys).
• Not accounting for distributed systems: Use a centralized store like Redis.
• Too strict or too loose: Monitor usage patterns and adjust limits.
• Ignoring edge cases: What happens when the clock resets? Use monotonic time if possible.

Conclusion

Rate limiting is a fundamental tool for building resilient APIs. Choose an algorithm that fits your traffic patterns, implement with care, and always communicate limits to your users. Start simple with a fixed window and evolve as needed.