Designing a URL Shortener From First Principles (and the Hashing Trap I Fell Into)

I'm working through system design as an AI engineer, and I'm writing up each problem as I go. This one is the classic: design a URL shortener like bit.ly. It looks simple, but it has one design decision that catches people out, and I walked right into it.

The two operations

1. Shorten: submit a long URL, get back a short one
2. Redirect: visit the short URL, get sent to the original

POST /urls { "longUrl": "...", "expiresAt": "..." } → 201 { "shortUrl": "sho.rt/21" }
GET /{shortCode} → 302 redirect to longUrl

Step 1: Estimate before designing

I anchored on a user base and derived everything else from it, rather than guessing RPS directly.

Reads (clicks):

• 100M total users, 50M active per day
• ~5 clicks per active user per day → 250M clicks/day
• 250M ÷ 86,400 seconds ≈ 2,900 RPS

Writes (creating links):
Creating a link is much rarer than clicking one. My first estimate had more creates than clicks, which contradicted the read-heavy intuition, so I redid it:

• ~5M users create links on a given day, ~5 links each → 25M creates/day
• 25M ÷ 86,400 ≈ 290 RPS

That's roughly a 10:1 read/write ratio, and it shapes everything below: the design effort belongs on the read path.

Storage:
Each record holds the long URL (~100 bytes), short code (~10), creator ID (~8–16), created timestamp (~8), expiry (~8) and a click counter (~8). That rounds to ~150 bytes.

25M records/day × 365 days ≈ 9.1B records/year
9.1B × 150 bytes ≈ 1.37 TB/year

1.37 TB a year fits on a single well-provisioned database. Storage isn't the bottleneck here, so there's no reason to shard just because the problem "sounds like scale." The numbers should drive that call, not the vibe.

Latency target: 100–200 ms. A redirect is a lookup with no heavy computation in the path, so it should feel instant.

Step 2: The architecture

flowchart LR
    U[User / Browser] --> LB[Load Balancer]
    LB --> A1[App Server 1]
    LB --> A2[App Server 2]
    LB --> A3[App Server N]
    A1 & A2 & A3 --> C[("Redis cache: shortCode to longUrl")]
    A1 & A2 & A3 --> DB[("Database: urls table + sequence")]

Stateless app servers sit behind a load balancer. A cache holds hot redirects, and a single relational database stores the records and generates IDs.

Step 3: The hard part is generating the short code

This is the actual design problem. There were three options:

flowchart TD
    Q{"How do we generate the short code?"}
    Q --> H[Hash the long URL]
    Q --> R[Random string]
    Q --> K[Counter + Base62]
    H --> H1["Same URL always gives the same code: breaks per-user expiry and analytics"]
    R --> R1["Can collide: needs retry when the DB rejects a duplicate"]
    K --> K1["Unique by construction: needs one shared counter"]

The hashing trap

My first instinct was to hash the long URL. It feels natural, since hashing turns a long input into a short output.

Then consider this case. User A shortens example.com/page in January for a 30-day campaign and wants their own click analytics. User B shortens the same URL in July with a different expiry.

Hashing is deterministic: the same input always gives the same output. Both users would get the same short code, and therefore one shared record. Whose expiry wins? Whose clicks are whose?

The real requirement is that each shorten request is its own record, because ownership, expiry and analytics belong to the request, not to the URL. So the short code has to be generated independently of the URL's content.

Random vs counter

Random strings mostly work, but they can collide. The database's unique constraint catches a collision, and you retry. As the table grows into billions of rows, collisions become less rare (the birthday problem).

A counter encoded in Base62 can't collide at all. Take an ever-increasing number and encode it with [0-9a-zA-Z]:

125 = 2 × 62 + 1  →  "21"

No two counter values are the same, so no two codes are the same. There's no retry logic needed.

How long does it last? With 7 characters, 62⁷ ≈ 3.5 trillion codes. At 9.1B links a year, that's over 380 years of headroom.

The distributed counter problem

There are many app servers. If each one keeps its own counter in memory, two servers can hand out the same number at the same moment.

There are two standard fixes:

1. Let the database own the counter (an auto-increment column or a Postgres SEQUENCE). The database guarantees uniqueness even under concurrent load.
2. Hand out ranges. Each server grabs a block of, say, 1,000 IDs at a time and assigns them locally. This reduces contention on the shared counter, at the cost of some wasted IDs if a server crashes mid-block.

I went with option 1. At ~290 creates/sec, a single database sequence handles the load comfortably, so ranges would solve a problem this system doesn't have. Match the complexity to the numbers you actually calculated.

sequenceDiagram
    participant C as Client
    participant S as App Server
    participant DB as Database
    C->>S: POST /urls (longUrl, expiresAt)
    S->>DB: nextval(url_seq)
    DB-->>S: 125
    S->>S: base62(125) = "21"
    S->>DB: INSERT shortCode, longUrl, createdBy, createdAt, expiresAt
    DB-->>S: OK
    S-->>C: 201 shortUrl = sho.rt/21

Step 4: Make the read path fast

With a 10:1 read/write ratio, redirects are where performance matters.

• Index: the short code is the primary key, so lookups are indexed automatically.
• Cache: hot shortCode → longUrl mappings live in Redis. The cache key is just the short code, because a redirect is public and identical for everyone.

sequenceDiagram
    participant C as Client
    participant S as App Server
    participant R as Redis
    participant DB as Database
    C->>S: GET /21
    S->>R: GET 21
    alt cache hit
        R-->>S: longUrl
    else cache miss
        R-->>S: null
        S->>DB: SELECT longUrl WHERE shortCode = 21
        DB-->>S: longUrl
        S->>R: SET 21 longUrl with TTL
    end
    S-->>C: 302 redirect to longUrl

Why 302 and not 301? Browsers cache a 301 (permanent) redirect, so repeat clicks never reach the server and the click counts would be wrong. A 302 keeps every click visible.

What I'd revisit next

• Guessable codes: Sequential codes mean anyone can walk through 21, 22, 23... and find other people's links. If that matters, shuffle or obfuscate the ID before encoding it.
• Expiry cleanup: Expired links need a background job to purge them, and the cache TTL shouldn't outlive the link's own expiry.
• Cache eviction: Decide on a policy (LRU is the usual choice) and how big the cache needs to be.

Takeaways

1. Derive the numbers, don't guess them. Total users, then active users, then per-user rate, then RPS. The same method works for storage.
2. Sanity-check your estimates against each other. My first write estimate exceeded my read estimate in a read-heavy system, and that contradiction was the signal to redo it.
3. Not every system needs to shard. 1.37 TB a year fits on one box.
4. Deterministic isn't always what you want. Hashing is ideal when identical inputs should map together. Here they shouldn't.
5. Match solution complexity to real scale. Use a database sequence at 290 writes/sec, and switch to ID ranges when the numbers say so.

This is part of a series where I learn system design in public. Next up: designing a rate limiter.