DEV Community

Timevolt
Timevolt

Posted on

Designing a Rate Limiter Like a Time Traveler in *Back to the Future*

#systemdesign #architecture #backend #programming

The Quest Begins (The "Why")

Picture this: I’m sipping coffee, watching our API get hammered by a sudden traffic spike—think Black Friday meets a flash mob. Requests start queuing, latency creeps up, and the dreaded 503s appear like unwanted plot twists in a sitcom. We needed a way to say “slow down, buddy” without turning our service into a bouncer at a club that’s either too lax or too brutal.

I first tried the naïve approach: each service instance kept an in‑memory counter, reset every minute. It worked fine on my laptop, but in production the counters diverged like siblings arguing over the last slice of pizza. Some users got through, others got blocked unfairly, and we had no global view of the load. Honestly, it felt like trying to steer a ship with a broken compass—frustrating and futile.

That’s when I remembered a line from Back to the Future: “Where we’re going, we don’t need roads.” In our case, we didn’t need each node to guess the limit; we needed a single source of truth that could travel instantly across the cluster. Enter Redis.

The Revelation (The Insight)

The big “aha!” moment was realizing that a rate limiter isn’t just about counting requests; it’s about making the count atomic and observable across all nodes. If we can guarantee that the increment‑and‑check operation happens as one indivisible step, we eliminate the race conditions that plagued the per‑instance counters.

Redis gives us exactly that with its single‑threaded model and Lua scripting. By wrapping the increment, expiry‑set, and comparison in a Lua script, we ensure that no two requests can interleave—no matter how many API gateways are calling at once. The insight? Treat the limiter as a tiny, self‑contained transaction that lives in Redis, not as a scattered set of local counters.

Trade‑offs popped up immediately:

  • Latency vs. Consistency – Every request now does a round‑trip to Redis (usually sub‑millisecond if it’s in‑same‑AZ). We traded a bit of network latency for perfect global consistency.
  • Complexity – Adding a Lua script feels like learning a new spell, but it’s a one‑time cost that pays off in correctness.
  • Throughput – Redis can handle hundreds of thousands of ops/sec; for most APIs that’s plenty. If you truly need millions, you might shard or look at a specialized solution, but for the typical rate‑limiting use case it’s more than enough.

Here’s a quick ASCII diagram to visualize the flow:

+-----------+        +-------------------+        +--------+
|  Client   | --->   | API Gateway /    | --->   | Redis  |
| (Browser) |        | Rate Limiter Svc |        | (Lua)  |
+-----------+        +-------------------+        +--------+
        ^                         |
        |                         v
   (response)            +-------------------+
                         |  Application Logic|
                         +-------------------+
Enter fullscreen mode Exit fullscreen mode

The gateway calls the limiter service; the service runs the Lua script against Redis; Redis returns “allow” or “deny”; the gateway proceeds or returns 429.

Wielding the Power (Code & Examples)

The Struggle (Before) 

# Naive per‑instance limiter (Python‑like pseudocode)
import time
from collections import defaultdict

_counts = defaultdict(int)
_RESET_TIME = 60  # seconds

def allow_request(user_id):
    now = int(time.time())
    bucket = _counts[user_id]
    if now - bucket['timestamp'] > _RESET_TIME:
        bucket['count'] = 0
        bucket['timestamp'] = now

    if bucket['count'] >= LIMIT:
        return False   # reject
    bucket['count'] += 1
    return True        # allow
Enter fullscreen mode Exit fullscreen mode

Problems:

  • Each instance has its own _counts; no global view.
  • The check‑then‑increment isn’t atomic—two threads could both see count == LIMIT-1 and both increment, bursting over the limit.
  • Resetting relies on local clock drift.

The Victory (After)

We moved to Redis with a Lua script:

-- rate_limiter.lua
-- KEYS[1] = redis key for this user (e.g., "rate:user:123")
-- ARGV[1] = limit (e.g., 100)
-- ARGV[2] = window size in seconds (e.g., 60)

local current = redis.call('GET', KEYS[1])
if current == false then
    -- first request in this window
    redis.call('SET', KEYS[1], 1)
    redis.call('EXPIRE', KEYS[1], ARGV[2])
    return 1   -- allowed
end

if tonumber(current) >= tonumber(ARGV[1]) then
    return 0   -- deny
end

local newval = redis.call('INCR', KEYS[1])
return newval  -- allowed (new count)
Enter fullscreen mode Exit fullscreen mode

And the thin wrapper in Python:

import redis
import hashlib

r = redis.Redis(host='redis-service', port=6379, db=0)

def allow_request(user_id, limit=100, window=60):
    # Create a stable key per user; you could also include endpoint, IP, etc.
    key = f"rate:{hashlib.sha256(user_id.encode()).hexdigest()}"
    # Load the Lua script once (in practice, cache the SHA)
    script = open("rate_limiter.lua").read()
    result = r.eval(script, 1, key, limit, window)
    return bool(result)   # 1 -> allowed, 0 -> denied
Enter fullscreen mode Exit fullscreen mode

Why this beats the before version:

  • Atomicity – The Lua script runs as one command; no interleaving.
  • Global view – All gateways see the same count.
  • Automatic expiry – EXPIRE ensures old keys disappear, no manual cleanup.
  • Simple to tune – Just change limit and window; no redeploy needed.

Common traps to avoid:

  1. Calling GET then INCR separately – Reintroduces the race condition. Always keep them inside Lua (or use Redis’ built‑in INCR with a subsequent EXPIRE check, but that still has a tiny window; Lua is safest).
  2. Forgetting to set an expiry – Keys will linger forever, wasting memory. The EXPIRE call in the script guarantees cleanup.
  3. Using a separate SETNX + EXPIRE pattern – Two round‑trips; again, not atomic.

Why This New Power Matters

Now that we’ve got a rock‑solid, distributed rate limiter, our API can survive traffic spikes without turning away good users or letting bad actors hog resources. We can safely expose public endpoints, offer tiered plans (different limits per API key), and even embed the same limiter inside microservices that talk to each other—think of it as a universal speed‑limit sign that every car on the highway obeys.

The best part? The pattern is reusable. Swap the Lua script for a token‑bucket algorithm, a fixed‑window counter, or a sliding‑window log, and you’ve got a building block for everything from API protection to election‑style vote throttling. It’s like discovering a new spell that works in any dungeon you crawl.

So go ahead—grab your favorite Redis instance, drop in that Lua script, and watch your service go from “frantic traffic cop” to “calm, composed conductor.” Ever felt like you were stuck in a loop of over‑engineering and under‑delivering? This is the shortcut that finally lets you ship with confidence.

Your turn: Try implementing a sliding‑window version using Redis’ ZADD/ZREM/ZCOUNT inside Lua. Share your results or any gotchas you hit—I'd love to hear how your quest went! 🚀

Top comments (0)