Rate Limiting Basics: Protecting Your API from Abuse

Why Rate Limiting Matters

Every public API eventually faces abuse. It could be a malicious script hammering your endpoints, a misconfigured client retrying in a loop, or just a sudden spike in traffic. Without rate limiting, your backend can get overwhelmed, leading to slow responses or crashes. Rate limiting protects your service, keeps costs predictable, and ensures fair usage among all consumers.

I remember my first production incident: a client accidentally sent thousands of requests per second to a search endpoint. The database CPU spiked to 100%, and the whole app became unresponsive for minutes. A simple rate limit would have prevented that entirely.

What Is Rate Limiting?

Rate limiting controls how many requests a client can make within a given time window. It's a policy that defines a threshold and an action when the threshold is exceeded. Common actions include rejecting the request with 429 Too Many Requests or delaying it.

The key concepts are:

• Limit: Maximum requests allowed in a window (e.g., 100 requests per minute).
• Window: The time period for the limit (e.g., 1 minute, 1 hour).
• Identifier: Who is being limited? Usually an IP address, API key, or user ID.

Simple Fixed Window Algorithm

The simplest approach is the fixed window. You track requests per identifier in a time bucket. For example, allow 10 requests per minute. When a request comes in, you increment the counter for the current minute. If the counter exceeds 10, reject.

Here's a minimal implementation in Node.js using an in-memory map:

const rateLimit = (limit, windowMs) => {
  const requests = new Map();

  return (req, res, next) => {
    const key = req.ip;
    const now = Date.now();
    const windowStart = Math.floor(now / windowMs) * windowMs;

    if (!requests.has(key)) {
      requests.set(key, { count: 0, windowStart });
    }

    const entry = requests.get(key);

    if (entry.windowStart !== windowStart) {
      entry.count = 0;
      entry.windowStart = windowStart;
    }

    entry.count += 1;

    if (entry.count > limit) {
      return res.status(429).json({ error: 'Too many requests' });
    }

    next();
  };
};

app.use('/api', rateLimit(10, 60000));

This works for simple cases, but has a flaw: a client can burst at the end of one window and the start of the next, effectively doubling the rate. For example, 10 requests at 59 seconds, then 10 more at 61 seconds.

Sliding Window Log

A more accurate approach is the sliding window log. You store timestamps of each request and count how many fall within the last windowMs. This avoids the burst issue but uses more memory.

const slidingWindow = (limit, windowMs) => {
  const timestamps = new Map();

  return (req, res, next) => {
    const key = req.ip;
    const now = Date.now();

    if (!timestamps.has(key)) {
      timestamps.set(key, []);
    }

    const list = timestamps.get(key);
    // Remove timestamps outside the window
    while (list.length > 0 && list[0] <= now - windowMs) {
      list.shift();
    }

    if (list.length >= limit) {
      return res.status(429).json({ error: 'Too many requests' });
    }

    list.push(now);
    next();
  };
};

This is more precise but can be slow if the list grows large. For high-traffic APIs, you'd want a more efficient structure like a token bucket.

Token Bucket Algorithm

The token bucket is popular because it allows bursts while smoothing out long-term rate. Think of a bucket that holds up to capacity tokens. Each request consumes one token. Tokens are added at a fixed rate (e.g., 1 token per second). If the bucket is empty, the request is rejected.

Here's a simple implementation:

class TokenBucket {
  constructor(capacity, refillRate) {
    this.capacity = capacity;
    this.refillRate = refillRate; // tokens per second
    this.tokens = capacity;
    this.lastRefill = Date.now();
  }

  tryConsume() {
    this.refill();
    if (this.tokens >= 1) {
      this.tokens -= 1;
      return true;
    }
    return false;
  }

  refill() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}

// Usage per key
const buckets = new Map();

function rateLimit(req, res, next) {
  const key = req.ip;
  if (!buckets.has(key)) {
    buckets.set(key, new TokenBucket(10, 1)); // 10 burst, 1 per second
  }
  const bucket = buckets.get(key);
  if (bucket.tryConsume()) {
    next();
  } else {
    res.status(429).json({ error: 'Too many requests' });
  }
}

This allows a client to make 10 requests immediately, then 1 per second afterwards. It's a good balance between responsiveness and protection.

Real-World Considerations

In production, you rarely write your own rate limiter. Libraries like express-rate-limit for Node, or services like Redis-based limiters (e.g., rate-limiter-flexible) handle distributed scenarios. But understanding the basics helps you choose the right one.

Key points to consider:

• Distributed systems: In-memory maps don't work across multiple instances. Use a shared store like Redis with atomic operations.
• Identify clients properly: IP addresses can be shared (NAT, proxies). Prefer API keys or user IDs when available.
• Response headers: Include X-RateLimit-Limit, X-RateLimit-Remaining, and Retry-After so clients know how to behave.
• Graceful degradation: When a limit is hit, return a clear error with a Retry-After header so clients can back off.

Testing Your Rate Limiter

Always write tests. For a simple limiter, test that:

• Requests within the limit pass.
• Requests over the limit get 429.
• The window resets correctly.

Here's a quick test using Node's built-in test runner:

const test = require('node:test');
const assert = require('node:assert');

// Assume your limiter is a function that returns middleware
const limiter = rateLimit(3, 1000);

const mockReq = (ip) => ({ ip });
const mockRes = () => {
  const res = {};
  res.status = (code) => {
    res.statusCode = code;
    return res;
  };
  res.json = (data) => { res.body = data; };
  return res;
};

test('allows up to limit requests', () => {
  const res = mockRes();
  limiter(mockReq('1.2.3.4'), res, () => {});
  limiter(mockReq('1.2.3.4'), res, () => {});
  limiter(mockReq('1.2.3.4'), res, () => {});
  assert.strictEqual(res.statusCode, undefined);
});

test('blocks over limit', () => {
  const res = mockRes();
  for (let i = 0; i < 4; i++) {
    limiter(mockReq('5.6.7.8'), res, () => {});
  }
  assert.strictEqual(res.statusCode, 429);
});

Final Thoughts

Rate limiting is a fundamental tool for any API developer. Start with a simple fixed window if you're prototyping, but for production, consider a token bucket or a battle-tested library. The most important part is being intentional: know your limits, document them, and handle the 429s gracefully on the client side as well.

Your future self, and your users, will thank you when the inevitable traffic spike doesn't take down your service.