The Empire Strikes Back: Designing a Rate Limiter for Your URL Shortener

The Quest Begins (The "Why")

I still remember the first time I threw together a URL shortener for a side‑project. It felt like I’d just unlocked a secret level in a game—short links, click stats, the whole shebang. But after a few days of sharing the link on Reddit, the traffic spiked and my little service started returning HTTP 429 errors left and right. Users complained, my logs filled with “Too Many Requests”, and I realized I’d forgotten to put a bouncer at the door.

Honestly, I was trying to solve the wrong problem. I kept tweaking the hash generation, thinking a better algorithm would stop the abuse. The real dragon wasn’t the collision‑prone base‑62 encoder; it was the uncontrolled flood of requests hammering my endpoint. If I wanted the shortener to survive real‑world usage, I needed a rate limiter that could keep the bad actors out while letting legit users click away.

So I embarked on a quest: find a limiter that’s simple enough to prototype in an afternoon, yet robust enough to handle thousands of requests per second without melting down my server.

The Revelation (The Insight)

After reading a few blog posts and wrestling with leaky bucket implementations, the insight hit me like a plot twist in The Matrix: the limiter doesn’t need to be perfect—it just needs to be fair and predictable for the majority of users.

Instead of trying to count every request in a sliding window with razor‑sharp precision (which often means heavy Redis calls or complex data structures), I settled on a token bucket algorithm with a fixed refill rate and a burst capacity. Here’s why it clicked:

• Predictable latency – each request either consumes a token instantly or gets a 429. No waiting queues, no complex back‑off math.
• Burst‑friendly – legit clients that click a few links in quick succession (think a power‑user sharing a meme pack) still get served because the bucket starts full.
• Low overhead – the bucket state is just two numbers: tokens and lastRefill. Updating them is a few integer ops; no external store required for a single‑node service.
• Easy to reason about – you can explain the limit in one sentence: “You get N tokens per second, up to a maximum of B.”

Compare that to a fixed‑window counter (reset every minute) which can allow up to 2N requests at the window edge, or a sliding‑window log that requires storing timestamps for every request (hello, memory explosion). The token bucket gives you the best of both worlds: smooth throttling with minimal bookkeeping.

Here’s the ASCII diagram that helped me visualise it:

+-------------------+      refill rate (r tokens/sec)      +-------------------+
|   Token Bucket    |  <----------------------------------  |   Refill Timer    |
| (capacity = B)    |                                      | (every 1/sec)    |
+-------------------+                                      +-------------------+
        ^                                                       |
        | consume 1 token per request                           |
        |                                                       v
+-------------------+                                 +-------------------+
|   Request Handler |  ----------------------------->  |   429 if empty    |
+-------------------+                                 +-------------------+

If the bucket has at least one token, the request passes and we decrement. Otherwise we reject with 429. The refill timer simply adds r tokens (capped at B) each second.

Wielding the Power (Code & Examples)

The Struggle – A Naïve Counter

My first attempt looked like this (Node.js/Express pseudo‑code):

// BEFOREVER
let hits = 0;
const WINDOW_MS = 60_000; // 1 minute
const LIMIT = 100;        // 100 req/min

app.get('/:code', (req, res) => {
  const now = Date.now();
  if (now - lastReset > WINDOW_MS) {
    hits = 0;
    lastReset = now;
  }

  if (hits >= LIMIT) {
    return res.status(429).send('Too many requests');
  }

  hits++;
  // ... redirect logic
});

Why it sucked:

• The hits reset created a cliff at the minute boundary—clients could blast 199 requests in the last second of a window and another 199 in the first second of the next window, effectively ~2× the limit.
• If the process crashed, the counter was lost, causing a sudden surge of allowed traffic on restart.
• Scaling to multiple instances meant sharing this state somewhere, adding complexity.

The Victory – Token Bucket in Memory

Here’s the same endpoint with a token bucket. I kept it deliberately simple so you can drop it into any language.

// tokenBucket.js
class TokenBucket {
  constructor(ratePerSec, burst) {
    this.rate = ratePerSec;      // tokens added each second
    this.capacity = burst;       // max tokens in bucket
    this.tokens = burst;         // start full
    this.lastRefill = Date.now();
  }

  consume() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000; // seconds
    // refill
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.rate);
    this.lastRefill = now;

    if (this.tokens >= 1) {
      this.tokens -= 1;
      return true; // allowed
    }
    return false; // rate limited
  }
}

// limiter.js
const limiter = new TokenBucket(10, 20); // 10 req/sec, burst up to 20

app.get('/:code', (req, res) => {
  if (!limiter.consume()) {
    return res.status(429).send('Too many requests');
  }
  // ... normal redirect logic
});

What changed?

• No window cliffs – tokens flow continuously, so the allowed rate is smooth over any interval.
• Burst handling – a client can spend up to burst tokens instantly, then must wait for the refill.
• Stateless per instance – each node holds its own bucket; if you run multiple nodes behind a load balancer, you either accept a slightly higher aggregate limit (still fine for most APIs) or share the bucket via Redis for strict global limits.
• Tiny footprint – just two numbers and a timestamp; negligible CPU and memory.

Common Traps to Avoid

1. Forgetting to clamp the token count – if you let tokens grow beyond capacity, a long idle period could let a client save up a huge bank and blast through the limit later. Always Math.min(tokens + added, capacity).
2. Using floating‑point math for token math – while fine for small rates, repeated additions can cause drift. I prefer to keep everything in milliseconds and integer tokens: store tokens as a number of milliseconds allowed, refill by rateMsPerToken. This avoids any rounding surprises.
3. Neglecting the refill timestamp update on every call – if you only refill on a timer, you lose the benefit of a truly continuous bucket and re‑introduce window‑like behavior.

Why This New Power Matters

With the token bucket in place, my URL shortener stopped crying wolf at every traffic spike. Legit users could share a dozen links in a row without seeing a 429, while abusive bots got gently turned away after a few clicks. The service stayed responsive, my logs stayed clean, and I could finally focus on the fun parts—analytics dashboards, custom slugs, and that sweet, sweet feeling when a link goes viral.

More importantly, the pattern is portable. Drop the same TokenBucket class into any API you build: authentication endpoints, webhook receivers, even internal micro‑services. It’s the Swiss‑army knife of rate limiting—simple, effective, and easy to reason about.

Now it’s your turn. Try swapping out a naïve fixed‑window counter in one of your projects for a token bucket. Play with the rate and burst values to see how they affect legitimate traffic versus abuse. If you feel adventurous, hook the bucket up to a Redis backend and share limits across a cluster—just remember to keep the core logic the same.

What’s the coolest rate‑limiting trick you’ve ever discovered? Drop it in the comments—I’m always hunting for the next upgrade to my developer’s utility belt! 🚀