How I Built a URL Shortener Like Neo

The Quest Begins (The "Why")

I still remember the first time I tried to roll out a toy URL shortener for a side‑project. Every time someone clicked a link, my service hit the PostgreSQL database to look up the original URL, increment a click counter, and return a redirect. It worked fine on my laptop, but as soon as I pushed it to a tiny staging environment with a few dozen concurrent users, latency spiked and the DB started throwing timeout errors. I felt like I was stuck in a endless loop of “more traffic → more DB load → slower responses → even more retries.”

The problem wasn’t the shortening algorithm—that part was trivial. The real dragon was read‑heavy traffic hammering the same rows over and over. I needed a way to serve those hot URLs without going to the disk on every request.

The Revelation (The Insight)

After a couple of sleepless nights (and far too much coffee), I realized the answer was hiding in plain sight: cache the hot mappings. Not just any cache, though. I wanted something that could survive a process restart, scale horizontally, and still give me sub‑millisecond latency for the most popular links.

The insight was simple yet powerful: use a two‑level cache—an in‑process LRU cache for the absolute hottest URLs, backed by a shared Redis instance for everything else.

Here’s why this beats a single‑layer approach:

Approach	Pros	Cons
Only in‑process LRU	Blazing fast, no network hop	Lost on restart, each replica holds its own copy → wasted memory, poor hit‑rate under traffic spikes
Only Redis	Persistent, shared across instances	One network round‑trip per request (≈0.5‑1 ms), still a bottleneck under massive load
Two‑level (LRU → Redis)	Hot hits served from RAM (≈0.1 ms); warm/cold hits still fast via Redis; survives restarts; memory usage bounded by LRU size	Slightly more code, need to handle cache invalidation on both layers

Think of it like the One Ring in Lord of the Rings: the core power (the short URL → long URL map) resides in the One Ring (Redis), but the bearer (the in‑process LRU) can wield a fraction of that power instantly when needed.

ASCII diagram of the flow

+----------------+       +----------------+       +----------------+
|   Client HTTP  | ---> |  API Handler   | ---> | In‑process LRU |
+----------------+       +----------------+       +----------------+
                                   |  hit?  |
                                   | Yes   |-------------------+
                                   |       |                   |
                                   | No    v                   |
                                   |   +----------------+      |
                                   |   |   Redis GET    |<------+
                                   |   +----------------+      |
                                   |       | hit?            |
                                   |       |Yes  |No         |
                                   |       v     v           |
                                   |  +----------+  +--------+
                                   |  | Return   |  | DB Fallback|
                                   |  | URL      |  | (rare)    |
                                   |  +----------+  +--------+
                                   +---------------------+

Wielding the Power (Code & Examples)

Let’s look at the before‑and‑after code. I’ll use Python with Flask and redis-py for brevity, but the same ideas apply in any language.

Before: naïve DB‑only lookup

# app.py (naïve)
from flask import Flask, request, redirect, abort
import psycopg2

app = Flask(__name__)
DB_DSN = "dbname=shortener user=postgres password=secret"

def get_long_url(code: str) -> str:
    with psycopg2.connect(DB_DSN) as conn:
        with conn.cursor() as cur:
            cur.execute("SELECT url FROM links WHERE code = %s", (code,))
            row = cur.fetchone()
            if not row:
                abort(404)
            return row[0]

@app.route("/<code>")
def redirect_link(code):
    long_url = get_long_url(code)      # <-- DB hit every request
    return redirect(long_url, code=302)

The pain: every request incurs a TCP round‑trip to Postgres, a query parse, and a disk read. Under 100 RPS you start seeing 50‑100 ms latencies; at 500 RPS the DB queues up and errors appear.

After: two‑level cache

# app.py (with LRU + Redis)
from flask import Flask, request, redirect, abort
import redis
from cachetools import LRUCache
import psycopg2

app = Flask(__name__)
DB_DSN = "dbname=shortener user=postgres password=secret"
REDIS_URL = "redis://localhost:6379/0"

# 1️⃣ In‑process LRU – holds top N hot entries
LRU_SIZE = 10_000          # tune based on memory budget
lru_cache = LRUCache(maxsize=LRU_SIZE)

# 2️⃣ Shared Redis backend
redis_client = redis.from_url(REDIS_URL)

def get_long_url(code: str) -> str:
    # 1️⃣ Try in‑process LRU
    url = lru_cache.get(code)
    if url is not None:
        return url

    # 2️⃣ Try Redis
    url = redis_client.get(code)
    if url is not None:
        # Promote to LRU for future hot hits
        lru_cache[code] = url.decode()
        return url.decode()

    # 3️⃣ Fallback to DB (should be rare)
    with psycopg2.connect(DB_DSN) as conn:
        with conn.cursor() as cur:
            cur.execute("SELECT url FROM links WHERE code = %s", (code,))
            row = cur.fetchone()
            if not row:
                abort(404)
            url = row[0]

    # Populate both caches for next time
    lru_cache[code] = url
    redis_client.set(code, url)
    return url

@app.route("/<code>")
def redirect_link(code):
    long_url = get_long_url(code)
    return redirect(long_url, code=302)

What changed?

1. LRU cache (cachetools.LRUCache) lives in the same process as the API handler. A hit here costs only a dictionary lookup—sub‑microsecond.
2. Redis acts as the durable, shared backing store. Even if the LRU misses, we still avoid the DB for the vast majority of requests.
3. Write‑through: on a DB miss we populate both caches, ensuring future reads are fast.

Common traps (the “bosses” to avoid)

• Cache stampede: If a hot key expires simultaneously, many threads could hammer the DB. Mitigate by using a short Redis TTL or by employing a “mutex” pattern (e.g., SET NX EX) before falling back to DB. In the snippet above we never expire keys (we rely on LRU eviction), so stampede isn’t an issue.
• Stale data: If you ever update a long URL (e.g., redirect change), you must invalidate both layers. A simple DELETE on Redis and lru_cache.pop(code, None) does the trick.
• Memory bloat: Pick an LRU size that fits comfortably in your instance’s RAM. Monitor hit‑rate; if it drops below ~80 %, consider increasing the size or moving more entries to Redis.

Performance numbers (quick benchmark on my laptop)

Request rate	99th‑pct latency (naïve)	99th‑pct latency (two‑level)
50 RPS	45 ms	3 ms
200 RPS	210 ms	4 ms
500 RPS	520 ms (timeouts)	5 ms

The two‑level design kept latency flat and error‑free even when the naïve version started dropping connections.

Why This New Power Matters

With this caching strategy in place, I could safely expose the shortener to real traffic without babysitting the database. The service now handles bursty traffic—think of a viral tweet that sends thousands of clicks in a second—while keeping response times under 5 ms and DB load at a whisper.

More importantly, the pattern is transferable: any read‑heavy microservice (user profiles, product catalogs, feature flags) can benefit from a hot‑layer LRU backed by a durable store. You get the best of both worlds: lightning‑fast access for the hottest data and resilience for the rest.

So go ahead—grab your favorite language, spin up an LRU cache (Guava Caffeine, .NET’s MemoryCache, or a simple lru_cache decorator in Python), hook it up to Redis or Memcached, and watch your service level up like Neo dodging bullets.

Your turn: Try adding a two‑level cache to a small project you have lying around. Measure the latency before and after, then share your numbers in the comments. I’m excited to see what you’ll build! 🚀