If you are preparing for software engineering interviews, you have probably heard the term System Design many times.
But what does it actually mean?
System design is not just about drawing boxes and arrows. It is about deciding how different parts of a software system should work together as the number of users, requests and data grows.
For example, building a simple Instagram clone for 100 users is easy. But what happens when millions of users are uploading photos, liking posts and refreshing their feeds at the same time?
That is where system design becomes important.
System design is the process of designing the architecture of a system so it can handle users, traffic, data, failures and future growth.
One of the first concepts you should understand is scaling.
What Is Scaling?
Suppose your application is running on one server.
Users
|
v
Server
|
v
Database
Everything works fine when you have a small number of users.
But as users increase, your server may start running out of CPU, memory or other resources.
So you need to scale the system.
There are two common ways to do this:
- Vertical Scaling
- Horizontal Scaling
What Is Vertical Scaling?
Vertical scaling means making your existing server more powerful.
For example, your application currently runs on:
CPU: 4 cores
RAM: 8 GB
You upgrade it to:
CPU: 16 cores
RAM: 64 GB
You did not add another server. You simply made the existing server stronger.
Vertical scaling = Scale up by increasing the resources of one machine.
A simple real life example is upgrading your laptop by adding more RAM or getting a faster processor.
When should you use vertical scaling?
It can be a good choice when:
- Your application is small
- Traffic is predictable
- You need a quick and simple solution
- Your workload is difficult to distribute
For example, if an internal company dashboard is used by only 50 people, adding 10 servers would probably be unnecessary.
The Problem With Vertical Scaling
The biggest problem is that there is a limit to how powerful one machine can become.
There is also a single point of failure.
Users
|
v
One Server
If that server goes down, your entire application can go down.
This is one reason we often consider horizontal scaling.
What Is Horizontal Scaling?
Horizontal scaling means adding more servers instead of making one server bigger.
For example:
Server 1
/
Users → Load Balancer → Server 2
\
Server 3
Now the traffic is distributed across multiple servers.
Horizontal scaling = Scale out by adding more machines.
Think about a restaurant.
If one cashier is not enough during lunch time, you don't necessarily make that cashier work five times faster. You open more counters.
That is basically horizontal scaling.
What Does a Load Balancer Do?
When you have multiple servers, something needs to decide which server should handle each request.
That's the job of a load balancer.
For example:
User 1 → Server 1
User 2 → Server 2
User 3 → Server 3
Some common load balancing strategies include:
- Round Robin
- Least Connections
- IP Hash
The goal is to distribute traffic so that one server does not become overloaded.
Vertical Scaling vs Horizontal Scaling
| Vertical Scaling | Horizontal Scaling |
|---|---|
| Makes one server more powerful | Adds more servers |
| Also called scaling up | Also called scaling out |
| Simpler to implement | More complex |
| Has hardware limitations | Can scale much further |
| Can have a single point of failure | Better availability |
| Good for smaller workloads | Good for large and growing workloads |
The easiest way to remember it:
Vertical = Make the machine bigger
Horizontal = Add more machines
Which One Should You Use?
The answer is not always "horizontal scaling".
It depends on the problem.
Example 1: Small Application
Imagine you have an internal dashboard with 100 users.
Your server is using only 30% CPU.
Do you need five servers?
Probably not.
Vertical scaling may be simpler and cheaper.
Example 2: Large E-commerce Platform
Now imagine an e-commerce website normally receives:
100,000 requests/minute
During a big sale, traffic suddenly increases to:
2,000,000 requests/minute
One server may not be enough.
Horizontal scaling makes more sense:
Server 1
/
Users → Load Balancer → Server 2
\
Server 3
|
Server N
You can add more servers as traffic increases.
One Important Problem With Horizontal Scaling
Horizontal scaling sounds great, but it introduces new challenges.
Suppose a user's session is stored only on Server 1.
User → Server 1
Session stored on Server 1
The next request goes to Server 2.
User → Server 2
Server 2 does not have the session information.
This is why horizontally scaled applications often use:
- Stateless servers
- Shared session storage
- Distributed caches such as Redis
For example:
Server 1
/
Users → Load Balancer
\
Server 2
|
v
Redis
Now multiple servers can access shared information.
A Common Interview Question
Interviewer:
Your application is running on one server and CPU usage is constantly around 95%. What would you do?
A beginner might immediately say:
"Add another server."
But a better answer is:
"First, I would identify the bottleneck. If the application is CPU bound and a larger machine can handle the workload, vertical scaling could be a quick solution. If traffic is growing and we need better availability and long term scalability, I would consider horizontal scaling."
This shows that you are thinking about requirements and tradeoffs, rather than blindly choosing a solution.
Another Tricky Interview Question
Interviewer:
You have 10 servers, but one server is receiving most of the traffic. You are already using a load balancer. Why could this happen?
Possible reasons include:
- The load balancing strategy is not distributing requests evenly
- Sticky sessions are being used
- Some requests are much heavier than others
- Traffic itself is uneven
The important lesson is:
Having multiple servers does not automatically mean your traffic is properly distributed.
The System Design Mindset
When learning system design, don't memorize:
"Horizontal scaling is always better."
Instead, ask:
- What is the bottleneck?
- How much traffic do we have?
- How fast is traffic growing?
- Do we need high availability?
- Can the workload be distributed?
- What will the solution cost?
- What happens if a server fails?
System design is about understanding tradeoffs and choosing the right solution for the problem.
Quick Recap
System Design: Designing the architecture of a system so it can handle users, traffic, data, failures and growth.
Vertical Scaling: Make one machine more powerful.
Small Server → Bigger Server
Horizontal Scaling: Add more machines.
1 Server → 5 Servers → 50 Servers
Use vertical scaling when simplicity and a smaller workload are more important.
Use horizontal scaling when traffic is large, growing or requires better availability.
And remember:
Vertical scaling makes one machine stronger. Horizontal scaling makes the system wider.
The goal of system design is not to use the most complicated architecture.
It is to choose the right architecture for the problem you are solving.
Top comments (0)