Anyone can write code that works. System design is what makes it work for millions of users at once.
If you're preparing for a software engineering interview, you've probably heard the term System Design many times.
But system design isn't just about drawing boxes and arrows on a whiteboard.
It's about answering questions like:
- What happens when 10 users become 10 million?
- What happens when one server is overloaded?
- How do multiple servers share the same data?
- How do we make frequently requested data faster?
- What happens when one service goes down?
- How do we process millions of requests reliably?
The easiest way to understand system design is to start with a simple real-world example.
Let's build a bank.
What Is System Design?
Before jumping into technical components, let's break down the words:
System + Design
A system is a collection of components working together toward a common goal.
We can think of it as:
System = Components + Common Goal
For example, an online banking system might contain:
Users
↓
Client Application
↓
API
↓
Application Server
↓
Database
Each component has a responsibility, but together they solve the overall business problem.
The important part is that our system needs to continue working as:
- Users increase
- Traffic increases
- Data increases
- Features increase
- Failures occur
That's where system design becomes important.
Let's Build a Bank 🏦
Imagine we are building a bank called Alien Bank.
Initially, our bank has:
- One cash counter
- One cashier
- Multiple customers
- One simple workflow
A customer comes to the counter and performs a transaction.
The flow is:
Customer
↓
Cash Counter
↓
Deposit / Withdraw
↓
Receipt
↓
Transaction Complete
Simple, right?
This system might work perfectly when we have a small number of customers.
But now let's introduce some problems.
Problem 1: The Process Is Too Slow
Suppose our cashier takes 10 minutes per customer.
That means:
60 minutes / 10 minutes = 6 customers/hour
Now imagine the number of customers starts increasing.
During peak hours, hundreds of customers might arrive.
Our system can't handle the traffic.
So what can we do?
First, let's optimize the existing cashier.
The cashier currently spends time:
- Understanding whether the customer wants to deposit or withdraw
- Counting cash manually
- Preparing the receipt
We can improve these individual tasks.
For example:
- Train the cashier to work faster
- Improve the cash-counting process
- Make receipt generation faster
Suppose the cashier improves from:
10 minutes/customer
↓
5 minutes/customer
We just improved the performance by roughly 50%.
Software Equivalent
This is similar to improving our application code.
For example:
Bad Algorithm
↓
Optimization
↓
Better Algorithm
↓
Faster Execution
This is where concepts like DSA and Low-Level Design (LLD) become important.
You might optimize:
- Algorithms
- Loops
- Data structures
- Database queries
- Code execution paths
Before adding more infrastructure, optimize what you already have.
Problem 2: More Customers Are Arriving
Our cashier is already working at maximum capacity.
We can't make the cashier infinitely faster.
So instead of improving the cashier, we improve the resources around them.
For example:
- Bigger counter
- Cash-counting machine
- Customer form filled before reaching the counter
Now the process becomes:
Customer
↓
Pre-filled Form
↓
Cashier
↓
Cash Counting Machine
↓
Receipt
Suppose the processing time decreases again:
5 minutes/customer
↓
3 minutes/customer
Now our single counter can handle significantly more customers.
But eventually, we will reach another limit.
Vertical Scaling
What we just did is similar to Vertical Scaling.
Vertical scaling means:
Upgrade the existing server instead of adding more servers.
For example:
Server
├── 4 CPU
├── 8 GB RAM
└── 100 GB Storage
Upgrade it to:
Server
├── 16 CPU
├── 32 GB RAM
└── 1 TB Storage
We are making the same server more powerful.
Vertical Scaling
┌───────────────┐
│ Server │
│ │
│ CPU ↑ │
│ RAM ↑ │
│ Storage ↑ │
└───────────────┘
Advantage
It's simple.
You don't need to manage multiple application servers.
Problem
There is a physical and practical limit.
You cannot keep upgrading the same machine forever.
Eventually, we need another solution.
Problem 3: One Counter Isn't Enough
Our bank now has many customers.
Even though our cashier is fast, customers are still waiting.
Why?
Because we have only one counter.
Suppose each customer takes 3 minutes.
If 10 customers are waiting:
Customer 1 → 3 min
Customer 2 → 6 min
Customer 3 → 9 min
...
Customer 10 → 30 min
The problem isn't the speed of the cashier anymore.
The problem is capacity.
So let's add another counter.
┌─────────────┐
Customers ───►│ Counter 1 │
└─────────────┘
┌─────────────┐
Customers ───►│ Counter 2 │
└─────────────┘
Now two customers can be processed simultaneously.
This is the idea behind Horizontal Scaling.
Horizontal Scaling
Horizontal scaling means:
Add more servers instead of making one server more powerful.
For example:
Before:
Client
↓
Server 1
After:
┌── Server 1
│
Client ──────┼── Server 2
│
└── Server 3
Instead of having one extremely powerful server, we have multiple servers working together.
This provides greater capacity and can improve availability.
But now we have a new problem.
Problem 4: Data Synchronization
Imagine Counter 1 maintains its own customer data.
Counter 2 maintains completely separate data.
Counter 1 → Database 1
Counter 2 → Database 2
A customer deposits ₹10,000 at Counter 1.
Database 1 says:
Balance = ₹10,000
But Database 2 doesn't know about that transaction.
It might still say:
Balance = ₹0
Now imagine the customer goes to Counter 2 and tries to withdraw ₹10,000.
We have a data consistency problem.
The two counters don't know the same state.
Centralized Database
To solve this, both counters can communicate with a common database.
┌──────────────┐
│ Database │
└──────┬───────┘
│
┌─────────┴─────────┐
│ │
┌─────▼─────┐ ┌─────▼─────┐
│ Counter 1 │ │ Counter 2 │
└───────────┘ └───────────┘
Now both counters read and write to the same source of data.
If Counter 1 updates the customer's balance, Counter 2 can retrieve the updated information.
In software systems, this represents the importance of shared data storage and database architecture.
Problem 5: One Server Gets All the Traffic
Now we have two servers.
But imagine most customers always walk toward Counter 1 because it's closer to the entrance.
Customers
↓
Counter 1 → 100 customers
Counter 2 → 10 customers
Counter 1 is overloaded.
Counter 2 is mostly idle.
We have multiple servers, but we're not using them efficiently.
We need something in the middle.
Load Balancer
Let's introduce a Load Balancer.
┌─────────────┐
│Load Balancer│
└──────┬──────┘
│
┌─────────┴─────────┐
↓ ↓
┌───────────┐ ┌───────────┐
│ Server 1 │ │ Server 2 │
└───────────┘ └───────────┘
Customers no longer decide which counter to use.
The load balancer does.
It can check:
- Which server is healthy?
- Which server has capacity?
- How should requests be distributed?
For example:
Request 1 → Server 1
Request 2 → Server 2
Request 3 → Server 1
Request 4 → Server 2
The exact strategy can vary, but the goal is the same:
Distribute traffic across available servers.
What Does This Mean in Software?
Our bank analogy maps nicely to a software architecture.
| Bank | Software |
|---|---|
| Customers | Requests / Traffic |
| Cashier | Application Code |
| Cash Counter | Server |
| Multiple Counters | Multiple Servers |
| Central Database | Shared Database |
| Middleman | Load Balancer |
| Queue of Customers | Request Queue |
So our architecture becomes:
Users
│
▼
┌──────────────┐
│Load Balancer │
└──────┬───────┘
│
┌──────┴───────┐
▼ ▼
┌───────────┐ ┌───────────┐
│ App Server│ │ App Server│
│ 1 │ │ 2 │
└─────┬─────┘ └─────┬─────┘
│ │
└──────┬───────┘
▼
┌──────────────┐
│ Database │
└──────────────┘
This is already much closer to a real production architecture.
The Core Components of System Design
Most modern applications can be understood by breaking them into several important components.
The major components include:
- Client
- Application Server
- API
- Database
- Cache
- Load Balancer
- Message Queue
Let's understand them one by one.
1. Client
The client is what the user interacts with.
Examples:
- Web application
- Mobile application
- Desktop application
- ATM
For example:
Mobile App
│
▼
API
The client sends requests to our backend.
2. Application Server
The application server contains our business logic.
For example:
Client
↓
Application Server
↓
Database
Suppose a user wants to transfer money.
The application server may:
- Authenticate the user
- Validate the request
- Check the account
- Update the balance
- Create a transaction
- Return the response
The application server hides the complexity from the client.
3. API
Instead of allowing clients to directly access our database, we expose APIs.
For example:
GET /users
or:
POST /orders
The client communicates with the API.
The API communicates with the application logic.
The application communicates with the database.
Client
↓
API
↓
Application
↓
Database
This separation is extremely important.
Why Not Let Clients Access the Database Directly?
Imagine giving every user direct database access.
A user would need to understand:
- Tables
- Columns
- Relationships
- Queries
- Database structure
That's obviously not practical.
Instead, the application layer hides all this complexity.
User
↓
Simple API
↓
Application Logic
↓
Complex Database Operations
This abstraction is one of the fundamental ideas behind application architecture.
4. Database
At the core of almost every system is data.
Data can include:
- Text
- Images
- Videos
- Audio
- User information
- Transactions
- Activities
Databases provide persistent storage.
Conceptually:
Application
↓
Database
↓
Persistent Data
Two major categories are:
Databases
├── SQL
└── NoSQL
The correct choice depends on the application's requirements.
SQL and NoSQL databases have different strengths and trade-offs, which are important topics in system design.
5. Cache
Let's imagine our application has millions of users.
Some information is requested repeatedly.
For example:
GET /popular-products
Suppose the application queries the database every time.
That could become expensive.
Instead, we can store frequently accessed data in a cache.
Client
↓
Application
↓
Cache
│
├── Data found → Return quickly
│
└── Data missing → Database
Conceptually:
Application
│
▼
Cache
│
│ Cache Miss
▼
Database
The cache provides faster access to frequently used data and reduces pressure on the database.
Cache eviction and invalidation strategies become important once we introduce caching.
6. Load Balancer
When our application has multiple servers, we need a way to distribute incoming traffic.
That's the job of the load balancer.
Client Requests
│
▼
┌──────────────┐
│Load Balancer │
└──────┬───────┘
│
┌─────────┼─────────┐
▼ ▼ ▼
Server 1 Server 2 Server 3
A load balancer can also perform health checks.
For example:
Server 1 → Healthy ✅
Server 2 → Healthy ✅
Server 3 → Down ❌
The load balancer can stop sending new requests to Server 3.
This improves availability and reliability.
7. Message Queue
Now imagine an e-commerce application.
A user places an order.
But placing an order might require several operations:
Create Order
↓
Update Inventory
↓
Process Payment
↓
Notify Vendor
↓
Arrange Delivery
↓
Send Notification
Should the user have to wait for every operation to finish?
Not necessarily.
Some operations can be processed asynchronously using a message queue.
Conceptually:
┌───────────────┐
Producer ───►│ Message Queue │
└───────┬───────┘
│
┌───────┴────────┐
▼ ▼
Consumer 1 Consumer 2
The producer creates a message.
The queue stores it.
Consumers process the message.
This allows different parts of the system to work independently and can help absorb traffic spikes.
Putting Everything Together
Now let's combine everything we've learned.
A more complete architecture could look like:
┌─────────────┐
│ Clients │
│ Web / Mobile│
└──────┬──────┘
│
▼
┌─────────────────┐
│ Load Balancer │
└────────┬────────┘
│
┌───────────┴───────────┐
▼ ▼
┌─────────────┐ ┌─────────────┐
│ App Server 1│ │ App Server 2│
└──────┬──────┘ └──────┬──────┘
│ │
└───────────┬───────────┘
│
┌──────────┴──────────┐
│ │
▼ ▼
┌─────────┐ ┌────────────┐
│ Cache │ │ Database │
└─────────┘ └────────────┘
┌──────────────┐
│ Message Queue│
└──────┬───────┘
│
┌──────────┴──────────┐
▼ ▼
Worker 1 Worker 2
Each component solves a different problem.
From Our Five Problems to System Design Concepts
The bank example gives us a very useful mental model.
| Problem | Solution | System Design Concept |
|---|---|---|
| Process is slow | Optimize cashier | DSA / LLD / Code Optimization |
| Single server reaches limits | Upgrade server | Vertical Scaling |
| Customers are waiting | Add counters | Horizontal Scaling |
| Servers have inconsistent data | Shared database | Database Architecture |
| One counter is overloaded | Middleman | Load Balancer |
| Database queries are expensive | Store frequent data separately | Cache |
| Many tasks need processing | Queue work | Message Queue |
This is the important takeaway:
System design is fundamentally about identifying bottlenecks and choosing the right architectural component to solve them.
System Design Is Not Just About Adding Servers
One common misconception is:
"System design means adding more servers."
That's only one part of it.
A good system design asks:
Where is the bottleneck?
↓
Why is it happening?
↓
What is the appropriate solution?
↓
What trade-offs does that solution introduce?
For example:
Slow code?
Optimize the code.
Server overloaded?
Consider vertical or horizontal scaling.
Too many requests?
Use load balancing.
Database overloaded?
Consider caching, indexing, replication, partitioning, or other database strategies depending on the workload.
Long-running operations?
Consider asynchronous processing and message queues.
Server failure?
Design for fault tolerance and recovery.
Reliability Matters
Scaling isn't enough.
Imagine you have 100 servers but one critical component fails.
What happens?
A production system needs to consider:
- Failures
- Recovery
- Monitoring
- Redundancy
- Health checks
- Fault tolerance
For example:
Server 1 ──┐
│
Server 2 ──┼── Load Balancer
│
Server 3 ──┘
If Server 2 fails:
Server 1 ──┐
│
Server 2 ──X
│
Server 3 ──┘
The system should ideally continue serving traffic through healthy servers.
This is why availability and fault tolerance are critical parts of system design.
Data Is at the Center
If you look at applications such as:
- YouTube
- Netflix
- Amazon
they may look completely different.
But at a high level, they all deal with data.
That data may be:
Images
Videos
Audio
Text
User Data
Transactions
Activities
So one way to think about system design is:
DATA
│
┌───────┼────────┐
│ │ │
Storage Compute Delivery
│ │ │
Database Servers APIs
│
Cache
The architecture exists to store, process, and deliver data efficiently and reliably.
A Simple Mental Model for System Design Interviews
When you're given a system design problem in an interview, don't immediately start drawing 20 boxes.
Start simple.
Step 1: Understand the requirements
Ask:
What are we building?
Who will use it?
What are the core features?
Step 2: Estimate scale
Think about:
Users
Requests/sec
Data volume
Read/write ratio
Traffic patterns
Step 3: Start with a simple architecture
For example:
Client
↓
Server
↓
Database
Step 4: Find bottlenecks
Ask:
What happens when traffic increases?
What happens when data grows?
What happens when the server fails?
Step 5: Scale the bottleneck
Introduce components only when they solve an actual problem:
Multiple Servers → Load Balancer
Frequently Read Data → Cache
Long Operations → Message Queue
Large Data → Database Scaling
Server Failure → Redundancy / Failover
This approach is much better than randomly adding technologies.
The Biggest Lesson
The most important lesson from this example is simple:
Don't design for complexity from day one. Design for the problems your system actually needs to solve.
Start with:
Client
↓
Application
↓
Database
Then evolve:
Client
↓
Load Balancer
↓
Multiple Application Servers
↓
Cache + Database
↓
Message Queue + Workers
Each additional component should exist for a reason.
Final Architecture Cheat Sheet
Keep this mental model handy:
USERS
│
▼
┌─────────────┐
│Load Balancer│
└──────┬──────┘
│
┌─────────┴─────────┐
▼ ▼
┌───────────┐ ┌───────────┐
│ App Server│ │ App Server│
│ 1 │ │ 2 │
└─────┬─────┘ └─────┬─────┘
│ │
└─────────┬─────────┘
│
┌──────┴──────┐
│ │
▼ ▼
CACHE DATABASE
│
│
▼
MESSAGE QUEUE
│
┌─────┴─────┐
▼ ▼
Worker 1 Worker 2
Remember the evolution:
Slow Code
↓
Code Optimization
Single Powerful Server
↓
Vertical Scaling
Single Server Capacity Limit
↓
Horizontal Scaling
Multiple Servers
↓
Shared Database
Uneven Traffic
↓
Load Balancer
Heavy Database Reads
↓
Cache
Long / Async Operations
↓
Message Queue
Conclusion
System design is not about memorizing architectures.
It's about problem-solving.
A system starts simple.
Then users grow.
Traffic increases.
Data increases.
Servers become overloaded.
Failures happen.
And every new problem forces us to make an architectural decision.
The bank example demonstrates the evolution clearly:
One Counter
↓
Better Counter
↓
Multiple Counters
↓
Centralized Data
↓
Load Balancer
↓
Cache
↓
Message Queue
↓
Scalable System
That's the essence of system design.
Build something that works first. Then design it so it can continue working when millions of users depend on it. 🚀
What's Next?
Once you understand this foundation, the next concepts worth learning are:
- DNS
- HTTP & HTTPS
- REST APIs
- SQL vs NoSQL
- Database Indexing
- Database Replication
- Database Partitioning / Sharding
- Caching Strategies
- Load Balancing Algorithms
- CAP Theorem
- Message Queues
- Fault Tolerance
- Monitoring & Observability
- Distributed Systems
These concepts are the building blocks you'll repeatedly use when designing real-world systems.
Top comments (0)