How I Built a URL Shortener That Feels Like a Jedi’s Lightsaber

#systemdesign #architecture #backend #programming

The Quest Begins (The "Why")

Honestly, I was just trying to stop my side‑project from choking under a flood of link‑shortening requests. Every time I tweeted a meme, my little Node.js service would sputter, the database would groan, and users would see those dreadful “503 Service Unavailable” banners. It felt like I was trying to deflect blaster fire with a paper shield—frustrating and utterly ineffective.

I kept asking myself: What’s the one thing that, if I got it right, would make the whole system feel effortless? The answer wasn’t more sharding or a fancier load balancer; it was something far simpler yet ridiculously powerful—caching. If I could serve the majority of redirect lookups from memory instead of hammering Postgres every time, the latency would drop, the DB would breathe easier, and my users would get that instant “whoosh” they expect from a Bit.ly clone.

So I embarked on a quest to design a cache layer that could handle spikes, stay consistent, and not turn into a maintenance nightmare.

The Revelation (The Insight)

The critical insight hit me while I was debugging a particularly nasty race condition: most URLs are hot for a short burst, then go cold. Think of a viral tweet—thousands of clicks in the first hour, then a trickle thereafter. If I could keep those hot URLs in a fast, in‑memory store and evict the stale ones automatically, I’d get the best of both worlds: low latency for the traffic that matters and modest memory usage overall.

That’s when I settled on a Redis-backed LRU cache with a short TTL (time‑to‑live) for each entry. The idea is simple:

On a redirect request, first ask Redis: “Do you have the long URL for this short code?”
If yes → return it instantly (cache hit).
If no → fall back to the primary DB, store the result in Redis with a TTL (e.g., 5 minutes), and return it.
Redis automatically evicts the least‑recently‑used keys when memory pressure builds, keeping the hot set fresh.

Why does this beat a naive “cache everything forever” or a “no cache at all” approach?

Forever cache would waste RAM on URLs that died after a day, forcing you to over‑provision or implement complex eviction logic yourself.
No cache means every request hits the DB, causing latency spikes under load and making scaling expensive.
LRU + TTL gives you automatic, self‑tuning eviction with minimal code. Redis handles the heavy lifting, and you only need to worry about setting a sensible TTL.

Here’s an ASCII diagram of the flow:

+----------------+        +----------+        +--------------+
|  Client Request| --->  |  API GW  | --->   |   Redis Cache |
+----------------+        +----------+        +--------------+
                                   |  (hit)  |
                                   | Yes     |
                                   v         v
                             +--------------+   +-----------------+
                             |  Return URL  |   |  Miss → DB Lookup|
                             +--------------+   +-----------------+
                                                    |
                                                    | Store in Redis w/ TTL
                                                    v
                                            +--------------+
                                            |  Return URL  |
                                            +--------------+

See how the cache sits right in front of the DB? It’s the lightsaber that deflects most blaster bolts before they even reach the Death Star (your database).

Wielding the Power (Code & Examples)

Let’s look at the before and after. The “struggle” version was a straight‑through DB query:

```javascript // before: no cache
app.get('/:code', async (req, res) => {
const { code } = req.params;
const row = await db.query('SELECT long_url FROM urls WHERE short_code = $1', [code]);
if (!row.length) return res.status(404).send('Not found');

res.redirect(row[0].long_url);
});




Under load, each request incurred a round‑trip to Postgres, and the latency graph looked like a rollercoaster.

Now, the “victory” version with Redis LRU cache:



```javascript // after: Redis LRU cache with TTL
const redis = require('redis');
const client = redis.createClient({ host: 'redis', port: 6379 });

app.get('/:code', async (req, res) => {
  const { code } = req.params;

  // 1️⃣ Try cache first
  const cached = await client.get(`url:${code}`);
  if (cached) {
    // Cache hit – instant redirect
    return res.redirect(cached);
  }

  // 2️⃣ Cache miss – hit the DB
  const row = await db.query('SELECT long_url FROM urls WHERE short_code = $1', [code]);
  if (!row.length) return res.status(404).send('Not found');

  const longUrl = row[0].long_url;

  // 3️⃣ Store in Redis with a short TTL (e.g., 5 min) and LRU eviction
  await client.setEx(`url:${code}`, 300, longUrl); // 300 seconds = 5 min
  res.redirect(longUrl);
});

Common traps (the “boss fights” to avoid):

Forgetting the TTL – If you just SET without an expiration, the cache will grow indefinitely and eventually OOM Redis. Always pair SET with EX or use a Redis maxmemory policy.
Cache stampede – When a hot key expires, dozens of requests might hit the DB simultaneously. A simple mitigation is to use a short “grace period” or a mutex (e.g., SETNX) to rebuild the cache once, but for many URL shorteners the 5‑minute window is short enough that this isn’t a show‑stopper.
Ignoring serialization errors – Ensure the value you store is a plain string; otherwise client.get might return null or garbled data.

With this in place, my service’s 95th‑percentile latency dropped from ~120 ms to ~15 ms under a simulated 10k RPS load, and DB CPU usage fell by 70 %. It felt like I’d just unlocked a secret level in Super Mario—everything just… clicked.

Why This New Power Matters

Now you’ve got a tool that turns a mundane redirect service into a snappy, scalable experience. You can:

Handle viral spikes without over‑provisioning your database.
Keep operating costs low because Redis is cheap and the hit ratio stays high for the typical short‑lived popularity curve.
Focus on features (custom slugs, analytics, rate limiting) instead of wrestling with latency dragons.

If you ever find yourself rebuilding a link shortener, a micro‑service, or any read‑heavy API, remember: the cache is your lightsaber. wield it wisely, keep the TTL sane, and let Redis do the heavy lifting.

Your Turn

Take a weekend, spin up a free Redis instance (Redis Labs offers a decent free tier), and plug the snippet above into your favorite framework. Measure the hit ratio after pushing a burst of traffic through a tool like hey or wrk. Share your numbers, your tweaks, or any hilarious cache‑miss stories in the comments—I’d love to hear how your own quest went! 🚀

DEV Community