SD #000 - Designing a URL Shortener

Designing a URL shortener is one of the most popular system design interview questions. It looks simple — but it tests your understanding of scalability, databases, caching, distributed systems, and trade-offs.

Let’s break it down in a way that’s easy to understand and easy to remember.

Step 1: Clarify Requirements

Before jumping into architecture, always clarify requirements.

Functional Requirements

1. Users should be able to create a short URL from a long URL.
2. When users visit the short URL, they should be redirected to the original URL.
3. Optional:
   • Custom alias (e.g., short.ly/myname)
   • Expiry time for links
   • Analytics (click count, geo, device info)

Non-Functional Requirements

1. High availability (service should almost never go down)
2. Low latency redirection (redirect should feel instant)
3. Massive scale (millions of URLs, billions of redirects)
4. Durable storage (no data loss even if a server crashes)

👉 URL shorteners are read-heavy systems. Redirects happen far more often than URL creations.

Step 2: Capacity Estimation

Let’s assume:

• 10 million new URLs per month
• 1 billion redirects per month
• 100:1 read-to-write ratio

What does this tell us?

• This is a read-heavy system
• Caching will be extremely important
• Database must scale horizontally
• Writes are manageable, but reads can explode

Step 3: High-Level Architecture

Here’s the overall architecture:

Let’s understand what’s happening.

Short URL Creation Flow

1. Client sends POST /shorten
2. Request goes through Load Balancer
3. URL Generation Service creates a unique short code
4. Mapping is stored in Database (Key-Value Store)
5. Short URL is returned to the user

👉 Note :

• This path is write-heavy but not extremely frequent.
• Uniqueness of short code is critical.

URL Redirection Flow

1. User hits GET /abc123
2. Load Balancer forwards request
3. Redirection Service checks Cache (Redis)
4. If cache miss → fetch from Database
5. System responds with HTTP 301/302 redirect

👉 Note :

• This is the hot path.
• Must be extremely fast.
• Cache hit rate should be high.

Why This Architecture Works

• Load Balancer → enables horizontal scaling
• Separate generation and redirection logic → clean design
• Cache layer → reduces database load
• Key-value database → fast lookup by short_code

👉 Since this is a read-heavy system, the redirection path must be optimized for low latency using aggressive caching.

Step 4: API Design

Create Short URL

POST /shorten
Body:
{
  "long_url": "https://example.com/very/long/url",
  "custom_alias": "optional",
  "expiry": "optional timestamp"
}

Response:

{
  "short_url": "https://sho.rt/abc123"
}

Redirect

GET /{short_code}

Returns:

301 or 302 redirect

How to Generate Short Codes

Option 1: Base62 Encoding

• Use auto-increment ID
• Convert ID to Base62 (a-zA-Z0-9)
• Guarantees uniqueness
• Short and efficient

Why it’s good:

• No collisions
• Deterministic
• Easy to scale

Option 2: Hashing

• Hash the long URL
• Take first 6–8 chars

Problem:

• Collisions possible
• Need collision resolution

Option 3: Distributed ID Generator

• Timestamp
• Machine ID
• Sequence number

Use when:

• Multiple servers generating IDs
• High scale

Database Scaling Strategy

Sharding Strategy

Shard by:

hash(short_code) % N

Why?

• Even distribution
• Prevents hotspotting
• Easy lookup

Replication Strategy

Since reads are much higher than writes, we can use:

• Primary node → Handles writes (creating new short URLs)
• Read replicas → Handle redirects

Why?

• Redirects dominate traffic
• We can scale reads independently
• Improves availability

If the primary fails:

• Promote a replica
• Ensure minimal downtime

Step 5: Caching Strategy

Because this is a read-heavy system, caching is essential.

What do we cache?

short_code → long_url

Redirect Flow with Cache

• Check Redis (or in-memory cache)
• If hit → return redirect immediately
• If miss → query DB → update cache → return redirect

Why this matters:

• Reduces database load dramatically
• Improves latency
• Helps handle traffic spikes

Optional optimizations:

• Use TTL for cache entries
• Use LRU eviction policy
• Pre-warm cache for popular URLs

Step 6: Handling Analytics at Scale

If we update click count in the database on every redirect, the system will slow down.

Better approach:

Redirect immediately

• Send click event to a message queue (Kafka, RabbitMQ, etc.)
• Background workers process and update analytics asynchronously

Why?

• Keeps redirect path fast
• Decouples analytics from user experience
• Scales independently

Step 7: Edge Cases

What if two users request the same custom alias?
→ Enforce uniqueness constraint in database.

What if a short code collision happens?
→ Retry with a new generated code.

What about malicious or spam URLs?
→ Validate URLs
→ Rate limit API
→ Block suspicious domains
→ Add CAPTCHA for abuse prevention

What about expired links?
→ Store expiry timestamp
→ Return 404 if expired

301 vs 302 — Which one to use?
301 = Permanent redirect (browser caches aggressively)
302 = Temporary redirect (more control)

Most systems prefer 302 because it allows link destination changes later.

Step 8: Trade-offs

Every design decision comes with trade-offs.