DEV Community

Derek mwale
Derek mwale

Posted on

Building a URL Shortener Like a Systems Engineer

A URL shortener looks like one of the simplest applications you can build.

You paste a long URL.

You get a short link.

Someone clicks it.

They are redirected.

Done.

At least that's how it looks from the outside.

But behind a service like Bitly, TinyURL, or any large-scale link platform is a fascinating systems engineering problem.

A production URL shortener needs to answer questions like:

  • How do we generate unique short URLs?
  • How do we handle billions of redirects?
  • How do we make redirects extremely fast?
  • How do we prevent abuse?
  • How do we track analytics?
  • How do we scale without everything breaking?

The interesting part is not writing the first version.

The interesting part is designing something that survives growth.

Let's build one like a systems engineer.

Understanding the Core Problem

At the surface, the system does two things:

1. Create a Short URL

Input:

https://www.example.com/products/software-development-course
Enter fullscreen mode Exit fullscreen mode

Output:

https://short.ly/a7Kx92
Enter fullscreen mode Exit fullscreen mode

2. Redirect Users

Input:

https://short.ly/a7Kx92
Enter fullscreen mode Exit fullscreen mode

Output:

https://www.example.com/products/software-development-course
Enter fullscreen mode Exit fullscreen mode

The second operation is much more important.

Why?

Because creating links happens occasionally.

Redirects happen constantly.

A popular link might receive:

  • Thousands of clicks per second
  • Millions of clicks per day

The system should be optimized for reading, not writing.

High-Level Architecture

A simple architecture might look like this:

                User
                 |
                 |
                 ▼

          API Gateway

                 |
        -----------------
        |               |
        ▼               ▼

 URL Service       Redirect Service

        |               |
        ▼               ▼

   Database          Cache

                         |
                         ▼

                    Analytics
Enter fullscreen mode Exit fullscreen mode

Each component has a clear responsibility.

The URL service creates links.

The redirect service handles traffic.

The cache keeps popular links fast.

Analytics collects information without slowing redirects.

Designing the Database

At the core, we need to store the relationship:

short_code → original_url
Enter fullscreen mode Exit fullscreen mode

A simple table:

CREATE TABLE urls (
    id BIGINT PRIMARY KEY,
    short_code VARCHAR(10) UNIQUE,
    original_url TEXT,
    created_at TIMESTAMP,
    expires_at TIMESTAMP
);
Enter fullscreen mode Exit fullscreen mode

Example:

short_code original_url
a7Kx92 https://example.com/article
b8Lm21 https://store.com/product

The database answers one important question:

"Where should this short code send the user?"

The Challenge: Generating Short Codes

The biggest design decision is creating the short identifier.

We need something:

  • Short
  • Unique
  • Fast to generate
  • Collision resistant

A common approach is Base62 encoding.

Base62 uses:

a-z
A-Z
0-9
Enter fullscreen mode Exit fullscreen mode

That's 62 characters.

A code like:

a7Kx92
Enter fullscreen mode Exit fullscreen mode

contains:

62^6
Enter fullscreen mode Exit fullscreen mode

possible combinations.

That gives billions of unique URLs.

Generating IDs

One approach:

Create a unique numeric ID.

Example:

Database ID:

123456
Enter fullscreen mode Exit fullscreen mode

Convert it:

123456 → Base62

→

a7Kx92
Enter fullscreen mode Exit fullscreen mode

The flow:

New URL

    |

Generate ID

    |

Convert ID

    |

Store mapping

    |

Return short URL
Enter fullscreen mode Exit fullscreen mode

Simple.

Reliable.

Fast.

Redirects Are the Critical Path

When someone clicks:

short.ly/a7Kx92
Enter fullscreen mode Exit fullscreen mode

we need the fastest possible response.

The flow:

Request

   |

Extract code

   |

Check Cache

   |

Database Lookup

   |

Redirect User
Enter fullscreen mode Exit fullscreen mode

The first place we look should be memory.

Adding Redis Cache

Most URLs are not equally popular.

A few links receive most traffic.

This is called a:

Long Tail Distribution

Example:

Popular Link

████████████████████

Normal Links

██

Rare Links

█
Enter fullscreen mode Exit fullscreen mode

Instead of hitting the database every time:

User

 |

Redis

 |

Database
Enter fullscreen mode Exit fullscreen mode

The first request loads the URL.

Future requests are served instantly.

Example:

a7Kx92 → https://example.com/article
Enter fullscreen mode Exit fullscreen mode

stored in Redis.

Now redirects happen in milliseconds.

Handling Billions of URLs

A small database works at first.

But eventually:

10 million URLs

100 million URLs

10 billion URLs
Enter fullscreen mode Exit fullscreen mode

becomes a different problem.

We need strategies.

Database Sharding

Instead of one huge database:

Database

   |

----------------

Shard 1

Shard 2

Shard 3

Shard 4
Enter fullscreen mode Exit fullscreen mode

URLs are distributed.

Example:

hash(short_code) % number_of_shards
Enter fullscreen mode Exit fullscreen mode

determines where data lives.

Now traffic is spread across multiple machines.

The Importance of Read Optimization

A URL shortener is a read-heavy system.

Think about the ratio:

Creating URLs:

1 request
Enter fullscreen mode Exit fullscreen mode

Clicking URLs:

1000+ requests
Enter fullscreen mode Exit fullscreen mode

This changes our design priorities.

We optimize:

  • Caching
  • Database indexes
  • Low latency
  • Fast lookups

Not complicated write systems.

Adding Analytics

Users want to know:

  • How many people clicked?
  • Where did they come from?
  • What devices did they use?
  • Which countries clicked?

The mistake would be:

User clicks

   |

Save analytics

   |

Redirect
Enter fullscreen mode Exit fullscreen mode

Now analytics slows everything down.

Instead:

User clicks

   |

Redirect immediately

   |

Send event asynchronously

   |

Process analytics later
Enter fullscreen mode Exit fullscreen mode

Using:

  • Message queues
  • Event streams
  • Background workers

keeps the redirect fast.

Preventing Abuse

A URL shortener can easily become an abuse platform.

Attackers may use it for:

  • Phishing
  • Spam
  • Malware distribution

A production system needs:

Rate Limiting

Example:

User A

100 requests/minute

Allowed
Enter fullscreen mode Exit fullscreen mode 
User B

10,000 requests/minute

Blocked
Enter fullscreen mode Exit fullscreen mode

URL Scanning

Before creating a link:

Check:

  • Known malicious domains
  • Suspicious patterns
  • Malware databases

Expiration

Some links should not live forever.

Example:

Temporary campaign link

Expires after 30 days
Enter fullscreen mode Exit fullscreen mode

Handling Failures

Distributed systems fail.

Servers crash.

Networks disconnect.

Databases become unavailable.

Design for failure.

Examples:

Database Failure

Use:

  • Replication
  • Backups
  • Failover systems

Cache Failure

The system should still work:

Cache

   X

Database

   ✓
Enter fullscreen mode Exit fullscreen mode

The cache improves speed.

It should never be the only source of truth.

API Design

A simple API:

Create Short URL 

POST /api/urls
Enter fullscreen mode Exit fullscreen mode

Request:

{
"url": "https://example.com/article"
}
Enter fullscreen mode Exit fullscreen mode

Response:

{
"short_url": "https://short.ly/a7Kx92"
}
Enter fullscreen mode Exit fullscreen mode

Redirect 

GET /a7Kx92
Enter fullscreen mode Exit fullscreen mode

Response:

HTTP 301 Redirect
Enter fullscreen mode Exit fullscreen mode

to:

https://example.com/article
Enter fullscreen mode Exit fullscreen mode

The Technologies I Would Choose

A practical stack:

Backend

  • Go
  • Rust
  • Java
  • Node.js

Database

  • PostgreSQL
  • MySQL
  • DynamoDB

Cache

  • Redis

Queue

  • RabbitMQ
  • Kafka

Infrastructure

  • Docker
  • Kubernetes
  • Cloud Load Balancer

The technology matters less than the architecture.

What This Project Teaches

A URL shortener is a small project with big lessons.

You learn:

  • Database design
  • Indexing
  • Caching
  • Distributed systems
  • Load balancing
  • Asynchronous processing
  • Rate limiting
  • System reliability

This is why system design interviews love URL shorteners.

The application is simple.

The engineering is not.

Final Thoughts

The difference between a beginner implementation and a systems-engineered implementation is not the number of files or the complexity of the code.

It is the ability to think about:

  • Scale
  • Failure
  • Performance
  • Trade-offs

A simple URL shortener can teach you more about backend architecture than many large projects.

Because great engineers don't just build things that work.

They build things that keep working when millions of people depend on them.

Top comments (0)