DEV Community

Timevolt
Timevolt

Posted on

Cache Like Neo: Designing a URL Shortener That Dodges Bottlenecks

#systemdesign #architecture #backend #programming

The Quest Begins (The "Why")

Hey friend, picture this: you’ve just launched your shiny new URL shortener. Users are pasting links, hitting that sweet “Shorten!” button, and… the response time starts to crawl like a snail on a lazy Sunday. You check the logs and see thousands of hits on the same popular URLs—think meme videos breaking the internet or a trending tweet that everyone wants to share. Your database is getting hammered with read after read for the same key, and each query adds latency. It felt like I was stuck in a loop, watching the same scene replay over and over, waiting for the system to catch up.

I asked myself: Why are we hitting the database for every request when the mapping from short code to long URL rarely changes? The answer hit me like a plot twist: we need a cache that serves hot redirects instantly, while still persisting the durable store for those rare misses.

The Revelation (The Insight)

The critical insight was simple but powerful: treat the short‑to‑long URL map as a read‑heavy, write‑light workload and put an LRU (Least Recently Used) cache in front of the datastore.

Why LRU? Because the popularity of URLs follows a classic 80/20 rule— a tiny fraction of short links generate the bulk of traffic. An LRU cache automatically keeps those hot entries at the top, evicting the cold ones when we run out of memory. Pair it with a TTL (time‑to‑live) so that even if a URL gets updated (rare, but possible), stale entries don’t linger forever.

Here’s the ASCII diagram of the flow:

+----------------+        +--------------+        +-----------------+
|   Client       | --->   |  LRU Cache   | --->   |   Persistent    |
| (HTTP GET /abc)|        | (in‑memory)  |        |   Store (DB)    |
+----------------+        +--------------+        +-----------------+
        ^                         |                         |
        |                         v                         v
        |               +----------------+        +-----------------+
        |               |  Miss? Write   | <----  |  Update/Delete  |
        |               |  Through to DB |        |   (rare)        |
        |               +----------------+        +-----------------+
        |                         ^                         |
        +-------------------------+-------------------------+
                               Cache Hit (fast path)
Enter fullscreen mode Exit fullscreen mode

Trade‑offs & why this beats the alternatives

  • Plain DB reads – Simple, but every request pays the cost of a disk/network round‑trip. Under load, latency spikes and you waste precious read capacity.
  • Write‑through cache (like Redis) – Gives you sub‑millisecond hits, but you must manage cache invalidation. Our LRU+TTL approach does that automatically: on a cache miss we fetch from DB, store the result, and let the LRU evict old entries. If a URL ever changes, we simply delete the key; the next request will repopulate the cache with the fresh value.
  • Global CDN – Overkill for a short‑link service; you’d still need origin validation and would add complexity for little gain when the dataset fits comfortably in memory.

The LRU cache gives us O(1) lookup and insert, constant‑time eviction, and a predictable memory footprint. It’s the kind of elegant solution that feels like discovering a hidden shortcut in a maze—suddenly everything just flows.

Wielding the Power (Code & Examples)

Let’s look at a before/after in Node.js (but the idea translates to any language).

The Struggle – Direct DB Hit 

// before.js – every request hits PostgreSQL
const { Pool } = require('pg');
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

app.get('/:code', async (req, res) => {
  const { code } = req.params;
  const { rows } = await pool.query(
    'SELECT long_url FROM urls WHERE short_code = $1', [code]
  );

  if (rows.length === 0) return res.status(404).send('Not found');
  res.redirect(rows[0].long_url);
});
Enter fullscreen mode Exit fullscreen mode

What’s painful? Each request does a round‑trip to the DB, even for the same viral link that’s been requested a thousand times in the last minute. Under 10k RPM, the DB CPU spikes and latency creeps up.

The Victory – LRU Cache in Front

We’ll use a tiny in‑memory LRU implementation (you could swap it for Redis if you need multi‑node sharing).

// after.js – LRU cache with TTL
const LRU = require('lru-cache');
const { Pool } = require('pg');
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

// max 50k entries, each lives 5 minutes
const urlCache = new LRU({
  max: 50000,
  ttl: 1000 * 60 * 5, // 5 min in ms
});

app.get('/:code', async (req, res) => {
  const { code } = req.params;

  // 1️⃣ Try cache first – O(1) lookup
  const cached = urlCache.get(code);
  if (cached) {
    // Cache hit – instant redirect
    return res.redirect(cached);
  }

  // 2️⃣ Miss – fetch from DB
  const { rows } = await pool.query(
    'SELECT long_url FROM urls WHERE short_code = $1', [code]
  );

  if (rows.length === 0) return res.status(404).send('Not found');

  const longUrl = rows[0].long_url;
  // 3️⃣ Write‑through: store in cache for next requests
  urlCache.set(code, longUrl);

  res.redirect(longUrl);
});
Enter fullscreen mode Exit fullscreen mode

Common traps to avoid

  1. Forgetting to delete on update – If you ever allow editing a short link, you must urlCache.delete(code); otherwise you’ll serve stale redirects.
  2. Setting TTL too long – A long TTL defeats the purpose of evicting cold entries; you’ll waste memory on URLs that have fallen out of popularity.
  3. Using a naïve map instead of LRU – A plain JavaScript object will grow unbounded; you’ll eventually OOM the process.

That’s it—just a few lines and you’ve turned a database‑bound service into a blazing‑fast redirector. The first time I saw the latency drop from ~120ms to ~2ms on a hot link, I felt like I’d just dodged a barrage of bullets in The Matrix—pure, satisfying flow.

Why This New Power Matters

With this cache in place, your shortener can comfortably handle tens of thousands of requests per second on a modest instance, leaving your database free for writes, analytics, or those rare link updates. You’ve essentially given your service a reflex: hot redirects are served straight from memory, while the durable store remains the source of truth for durability and consistency.

Now you can spend your energy on fun features—custom aliases, click analytics, or spam detection—knowing the core lookup path won’t become a bottleneck. It’s the kind of foundation that lets you build higher, faster, and safer.

Your Turn

Grab a laptop, spin up a quick Node (or Go, Python, whatever you love) server, and plug in an LRU cache like the one above. Try simulating a traffic spike with a tool like wrk or hey and watch the latency stay flat.

Challenge: Add a simple hit‑rate metric to your endpoint and log it every minute. See how the hit‑rate climbs as you pump more traffic—when does it plateau?

Happy caching, and may your redirects always be swift! 🚀

Top comments (0)