The Empire Strikes Back: Choosing Between Microservices and Monoliths for a Rate Limiter

The Quest Begins (The "Why")

Imagine you’re building a shiny new API that lets users upload cat pictures. Everything’s going great until you notice a rogue client hammering the endpoint with a thousand requests per second. Your servers start sweating, latency spikes, and the cats look… unhappy. You need a rate limiter, fast.

At first glance the problem seems simple: count how many requests each IP (or API key) makes in a sliding window and block the excess. You slap a quick in‑memory counter on each service instance, call it a day, and move on.

Then reality hits. Your architecture isn’t a single monolith anymore; you’ve split the API into a handful of microservices running behind a load balancer. Each instance now has its own private counter, so the limit is effectively per‑instance instead of global. A sneaky client can spread its traffic across three instances and slip through the net, tripling the allowed rate.

That moment felt like facing the Death Star’s trench run without a targeting computer—you know the goal, but your tools are blind to the bigger picture. I spent a few hours debugging why my limiter kept “failing open,” and when the numbers finally added up, I felt like I’d just blown up the reactor.

The Revelation (The Insight)

The core insight hit me like a lightsaber to the throat: rate limiting must be a shared, atomic operation if you want a true global guarantee. In a microservice world, any state that lives only inside a process is a liability for cross‑cutting concerns like throttling, caching, or distributed locks.

The fix? Offload the counter to a fast, strongly consistent store that all instances can talk to—enter Redis. By using a single Redis key per client (or API key) and performing the check‑and‑update in one atomic Lua script, we get a linearizable token‑bucket algorithm that works no matter how many instances you spin up.

Why does this beat the naïve per‑instance approach?

• Correctness – No more “limit per instance” loopholes.
• Scalability – Redis handles tens of thousands of ops/sec with sub‑millisecond latency; you can shard or cluster it as traffic grows.
• Observability – One place to inspect limits, reset counters, or emit metrics.

Sure, you add a network hop and an external dependency, but the trade‑off is worth it when correctness outweighs the tiny latency cost. Think of it as swapping a blaster for a lightsaber—you lose the simplicity of a cheap gun, but you gain precision you can rely on when the stakes are high.

Wielding the Power (Code & Examples)

The Struggle: Per‑Instance Counter (the trap)

# flask‑like pseudo‑code – DON’T USE THIS IN PRODUCTION
from flask import request, abort
import time

# each service instance keeps its own dict
_counters = {}          # {key: [count, reset_time]}
WINDOW_SECONDS = 60
LIMIT = 100

def allow_request(key: str) -> bool:
    now = time.time()
    count, reset = _counters.get(key, [0, now + WINDOW_SECONDS])

    if now > reset:               # window expired
        count, reset = 0, now + WINDOW_SECONDS

    if count >= LIMIT:
        return False              # reject

    _counters[key] = [count + 1, reset]
    return True

@app.route("/upload")
def upload():
    if not allow_request(request.remote_addr):
        abort(429)
    # …handle upload…

What’s wrong? If you run three instances behind a load balancer, each tracks its own count. A client can send 150 requests, split 50/50/50, and every instance thinks it’s under the limit. The global limit of 100 is silently violated.

The Victory: Atomic Redis Token Bucket

We’ll implement a simple token bucket: each key holds tokens (how many requests are left) and last_refill (timestamp). On each request we refill based on elapsed time, consume a token if available, and write back—all in one Lua script to guarantee atomicity.

-- rate_limit.lua
local key = KEYS[1]               -- e.g. "rate_limit:192.168.1.42"
local limit = tonumber(ARGV[1])   -- max tokens per window
local window = tonumber(ARGV[2])  -- refill interval in seconds
local now = tonumber(ARGV[3])     -- current unix time (float)

local data = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens = tonumber(data[1]) or limit
local last_refill = tonumber(data[2]) or now

-- refill tokens based on time passed
local delta = math.max(0, now - last_refill)
local added = math.floor(delta / window) * limit
tokens = math.min(limit, tokens + added)

local allowed = 0
if tokens >= 1 then
    tokens = tokens - 1
    allowed = 1
end

-- persist updated state
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
redis.call('EXPIRE', key, window * 2)   -- auto‑clean after 2 windows
return allowed

And the thin wrapper in Python:

import time
import redis

r = redis.Redis(host='redis-host', port=6379, db=0)

def allow_request_redis(key: str, limit: int = 100, window: int = 60) -> bool:
    lua = open('rate_limit.lua').read()
    script = r.register_script(lua)
    now = time.time()
    allowed = script(
        keys=[key],
        args=[limit, window, now],
        client=r
    )
    return bool(allowed)

@app.route("/upload")
def upload():
    client_ip = request.remote_addr
    if not allow_request_redis(client_ip):
        abort(429)
    # …process upload…

Why this works: The Lua script runs inside Redis as a single atomic command. No two requests can interleave between reading tokens and writing the new value, so the count is always correct. The EXPIRE ensures stale keys disappear automatically, keeping memory usage bounded.

Common traps to avoid:

1. Separate GET/SET – Doing GET, then calculating, then SET opens a race window.
2. Using plain INCR without refill logic – That gives a fixed‑window counter, which can allow bursts at window edges.
3. Neglecting timeout – Forgetting to set an expiry leads to unlimited key growth over time.

Why This New Power Matters

Switching to a shared, atomic rate limiter transforms your system from a fragile, “hope‑for‑the‑best” guardrail into a reliable, observable safety net. You can now:

• Scale horizontally without re‑thinking limits—just add more API instances behind the same load balancer.
• Tune limits dynamically by changing the Lua script arguments or pushing new values via a config service; every instance sees the change instantly.
• Integrate with observability—Expose Redis INFO or use a sidecar to export per‑key token counts to Prometheus, giving you insight into who’s flirting with the limit.

The extra network hop adds maybe a millisecond of latency, but compared to the cost of letting a malicious client overwhelm your services (or worse, corrupt data), it’s a bargain. And because the logic lives in a single script, you can port it to other languages or even to a sidecar like Envoy’s Lua filter with minimal changes.

Your Turn

Grab a Redis instance, drop the Lua script in, and wrap your endpoints with the allow_request_redis helper. Try bursting traffic with a tool like hey or writh and watch the limiter hold firm. Then, experiment: swap the token bucket for a leaky bucket, or add a per‑user tier by including the user ID in the key.

What will you rate‑limit next? A login endpoint? A webhook receiver? Drop a comment below with your twist—I’d love to hear how you’re using this newfound power in your own services!

---

May your limits be strict, your latency low, and your services stay forever resilient.