Rate Limiting Basics Every Developer Should Know

Why Rate Limiting Matters

Rate limiting is a technique to control how many requests a client can make to your API or server within a given time window. Without it, a single misbehaving client can overwhelm your resources, degrade service for everyone, or rack up costs on your cloud bill. It's not just about security (though it helps with brute-force attacks) - it's about protecting your system's stability and ensuring fair usage.

As developers, we often add rate limiting only after an incident. But understanding the basics early can save you a lot of pain. Let's break down the core concepts.

Key Concepts

1. Requests per Time Window

The simplest form is a fixed number of requests per second, minute, or hour. For example, allow 100 requests per minute per user. But "per user" can be tricky - you need to identify who the user is. Common identifiers are:

• IP address
• API key or token
• User ID after authentication

Each has tradeoffs. IP can be shared behind a NAT or proxy. API keys are better but can be leaked or shared. Choose based on your system and threat model.

2. Fixed Window vs Sliding Window

A fixed window algorithm counts requests in a calendar-like window (e.g., every minute from 12:00 to 12:01). It's simple but has a burst problem: a client can send 100 requests at 12:00:59 and another 100 at 12:01:01, effectively doubling the rate in a short span.

A sliding window algorithm uses a rolling window of time (e.g., last 60 seconds from any point). It smooths out bursts and is more accurate. Implementations can be memory-heavy, but there are approximations like the sliding window counter.

3. Token Bucket Algorithm

This is a popular and intuitive algorithm. Imagine a bucket that holds tokens. Each request consumes one token. Tokens are refilled at a fixed rate (e.g., 10 tokens per second). If the bucket is empty, the request is rejected. The bucket capacity allows short bursts up to its size.

Here's a simple in-memory implementation in JavaScript:

class TokenBucket {
  constructor(capacity, refillRatePerSecond) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillRate = refillRatePerSecond;
    this.lastRefill = Date.now();
  }

  take() {
    this._refill();
    if (this.tokens >= 1) {
      this.tokens -= 1;
      return true;
    }
    return false;
  }

  _refill() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}

// Usage
const bucket = new TokenBucket(10, 2); // capacity 10, refill 2 per second
if (bucket.take()) {
  // handle request
} else {
  // return 429 Too Many Requests
}

This is clean and works for a single server instance. For distributed systems, you need a shared store like Redis.

How to Respond When Limit Exceeded

The standard HTTP status code is 429 Too Many Requests. Include a Retry-After header so clients know when they can try again. For example:

HTTP/1.1 429 Too Many Requests
Retry-After: 30

Also consider returning a structured error body with details like the limit and remaining quota. This helps developers integrating with your API.

Where to Apply Rate Limiting

• API Gateway level: Easy to enforce centrally, but might not have deep business context.
• Application level: You can use user IDs and more fine-grained rules, but it adds code to your service.
• Database level: Not typical, but you can limit expensive queries per client to prevent resource exhaustion.

Often a combination works best: coarse limits at the gateway, fine-grained limits in the app.

Common Pitfalls

• Forgetting to identify clients correctly: Using IP only can lock out many users behind a corporate proxy. Prefer API keys or user IDs when possible.
• Not handling distributed state: In-memory counters don't work if you have multiple server instances. Use Redis with atomic operations (e.g., INCR and EXPIRE in Lua script).
• Not cleaning up state: If you track by IP, you'll accumulate memory. Use time-based expiration.
• Making limits too strict: You might block legitimate users. Monitor and adjust.
• Ignoring headers: If you rate limit, tell the client their current limits and remaining quota via headers like X-RateLimit-Limit and X-RateLimit-Remaining. This improves developer experience.

Testing Your Rate Limiter

Write tests to ensure your limiter works. For a token bucket, test that:

• A burst within capacity is allowed.
• Exceeding capacity returns false.
• Tokens refill over time.

Here's a quick test using Node's built-in test runner:

const test = require('node:test');
const assert = require('node:assert');

// Assume tokenBucket is defined elsewhere

test('allows requests up to capacity', () => {
  const bucket = new TokenBucket(3, 1);
  assert.equal(bucket.take(), true);
  assert.equal(bucket.take(), true);
  assert.equal(bucket.take(), true);
  assert.equal(bucket.take(), false);
});

Wrapping Up

Rate limiting is not a one-size-fits-all. Start with a simple fixed window or token bucket, then evolve based on your traffic patterns and business needs. Always monitor your rate limit rejections to spot legitimate users being blocked and adjust accordingly.

Remember, the goal is not to make your API unfriendly, but to keep it reliable. A well-designed rate limit with clear headers and error messages helps everyone build better integrations.