Imagine you have just launched a web application.
At first, everything is simple.
You have a backend server, a database, and a few users.
Users
↓
Backend Server
↓
Database
Your application works perfectly.
Then something changes.
100 users become 1,000.
1,000 become 10,000.
10,000 become 100,000.
Suddenly, your server is using more CPU and memory, response times are increasing, and requests are starting to fail.
So you face a fundamental system-design question:
How do you make your system handle more traffic?
This is where scaling comes in.
There are two fundamental approaches:
- Vertical Scaling — Scale Up
- Horizontal Scaling — Scale Out
Understanding the difference is one of the first important concepts in system design.
What Is Scaling?
In software systems, scaling is the ability to increase a system's capacity as its workload grows.
That workload might mean:
- More users
- More requests
- More data
- More concurrent connections
- More background jobs
- Higher traffic during peak periods
Suppose your server can currently handle 1,000 requests per second.
If your application grows and starts receiving 5,000 requests per second, you need additional capacity.
You have two broad choices:
Make the existing server more powerful.
Or:
Add more servers.
That's the fundamental difference between vertical and horizontal scaling.
1. Vertical Scaling — Scale Up
Vertical scaling means increasing the resources of an existing machine.
Instead of adding more servers, you make the current server more powerful.
For example:
Before
┌─────────────────────┐
│ Server │
│ │
│ 2 CPU │
│ 4 GB RAM │
└─────────────────────┘
You upgrade it:
After
┌─────────────────────┐
│ Server │
│ │
│ 16 CPU │
│ 64 GB RAM │
└─────────────────────┘
The application architecture hasn't fundamentally changed.
You simply gave the existing machine more resources.
This is why vertical scaling is often the easiest way to increase capacity in the early stages of an application.
Why Vertical Scaling Is Attractive
Imagine you're running a small application.
Your server is reaching 80% CPU utilization.
You could immediately introduce:
- Multiple servers
- A load balancer
- Distributed sessions
- Service discovery
- Additional monitoring
- More deployment complexity
But that may be unnecessary.
Instead, upgrading the machine might solve the problem.
Advantages of Vertical Scaling
- Simple to implement
- Minimal architectural changes
- Easier to operate
- Fewer machines to manage
- Useful for smaller workloads
- Can be a quick way to increase capacity
For a small or moderately sized application, simplicity can be a major advantage.
The Problem With Vertical Scaling
The problem is that a single machine has limits.
You can keep increasing CPU, RAM, storage, and network capacity, but eventually you reach a hardware, platform, or cost boundary.
More importantly, your system may still depend heavily on one machine.
Consider:
Users
↓
┌─────────────┐
│ Server │
└─────────────┘
↓
Database
What happens if that server fails?
Your entire application may become unavailable.
This introduces another important system-design concept:
A single point of failure.
Vertical scaling can increase capacity, but simply making one machine bigger does not automatically provide redundancy.
This distinction is important:
Scaling and availability are related, but they are not the same thing.
2. Horizontal Scaling — Scale Out
Horizontal scaling takes a different approach.
Instead of making one server bigger, you add more servers or instances.
For example:
Before
Users
↓
Server
becomes:
Users
↓
┌──────────────┐
│ Load Balancer│
└──────────────┘
/ | \
↓ ↓ ↓
Server Server Server
1 2 3
Now requests can be distributed across multiple servers.
If traffic increases, you can add additional instances:
Server 1
Server 2
Server 3
Server 4
Server 5
...
This is why horizontal scaling is also called scaling out.
Why Horizontal Scaling Changes the Architecture
At first glance, horizontal scaling looks simple:
"Just add more servers."
But adding machines introduces a completely different class of problems.
Now your application has to deal with:
- Network communication
- Load balancing
- Shared state
- Session management
- Server failures
- Data consistency
- Monitoring
- Deployment across multiple instances
You have moved from thinking about one machine to thinking about a distributed system.
That's where system design becomes much more interesting.
The Role of a Load Balancer
If you have multiple servers, something needs to decide where incoming requests should go.
That's one of the jobs of a load balancer.
Users
|
↓
┌──────────────┐
│ Load Balancer│
└──────────────┘
/ | \
↓ ↓ ↓
┌──────┐ ┌──────┐ ┌──────┐
│ App │ │ App │ │ App │
│ A │ │ B │ │ C │
└──────┘ └──────┘ └──────┘
For example:
Request 1 → App A
Request 2 → App B
Request 3 → App C
Request 4 → App A
The exact distribution strategy depends on the load-balancing algorithm and application requirements.
We'll explore load balancing separately on Day 3.
Stateful vs Stateless Servers
Here's where horizontal scaling becomes more interesting.
Suppose a user logs into your application.
The first request reaches Server A:
User
↓
Server A
↓
User Session
Now the user's next request reaches Server B:
User
↓
Server B
What if Server B doesn't know about the session stored on Server A?
You have a problem.
This is one reason horizontally scaled applications commonly aim for stateless application servers.
Instead of keeping important shared state only inside one application instance, shared state can be stored in systems accessible by all instances.
A simplified architecture could look like:
Load Balancer
/ | \
↓ ↓ ↓
App A App B App C
\ | /
\ | /
Shared State
|
Database
Depending on the requirements, dedicated systems such as caches or session stores can also be used.
The important idea is:
Any application server should be able to handle a request without depending on local state that exists only on another server.
This makes horizontal scaling significantly easier.
Vertical vs Horizontal Scaling
Let's compare them directly.
| Aspect | Vertical Scaling | Horizontal Scaling |
|---|---|---|
| Also called | Scaling Up | Scaling Out |
| Basic idea | Make one machine stronger | Add more machines |
| Architecture | Simpler | More distributed |
| Complexity | Lower | Higher |
| Hardware limit | Yes | Can add more instances |
| Redundancy | Limited by itself | Easier to achieve |
| Load balancer | Usually not required | Commonly used |
| State management | Simpler | More important |
| Failure handling | More dependent on one machine | Can distribute failure across instances |
| Best for | Smaller/simple workloads | Large or growing workloads |
But there's an important point here.
Horizontal Scaling Is Not Automatically Better
If you have a small application with predictable traffic, introducing five servers may create more operational complexity than value.
Good system design is not:
"Always use horizontal scaling."
It's:
Choose the architecture that fits the workload and requirements.
A Simple Real-World Example
Imagine you're building an e-commerce application.
Initially, you have:
Users
↓
Application Server
↓
Database
Your application has a few hundred users and moderate traffic.
A single reasonably powerful server may be completely sufficient.
As the application becomes popular, traffic increases:
Users
↓
Application Server
↓
Database
The application server is now reaching its CPU and memory limits.
At this stage, you might vertically scale:
Small Server
↓
Bigger Server
Later, suppose traffic grows dramatically during a major sale.
One server is no longer enough.
You might move toward:
Users
↓
Load Balancer
/ | \
↓ ↓ ↓
App A App B App C
\ | /
\ | /
Database
Now the system can handle more application traffic.
But then another problem may appear.
The database becomes the bottleneck.
That leads to an important lesson:
Scaling is not a one-time architectural decision. It is an iterative process.
The Biggest Mistake: Scaling the Wrong Component
This is one of the most important lessons in system design.
Suppose you have:
Users
↓
Load Balancer
↓
20 Application Servers
↓
Database
Your application servers are fine.
But your database can only handle the workload generated by a few servers.
Now you add more application servers.
Did you solve the problem?
No.
You may have made it worse.
20 Application Servers
|
↓
┌──────────────┐
│ Database │
│ BOTTLENECK │
└──────────────┘
The database is still the limiting component.
This is why scaling should start with identifying the bottleneck.
Potential bottlenecks include:
- CPU
- Memory
- Database
- Network
- Disk I/O
- External APIs
- Application code
- Lock contention
Before scaling anything, ask:
What is actually limiting the system?
That question is often more important than deciding between vertical and horizontal scaling.
Can We Use Vertical and Horizontal Scaling Together?
Absolutely.
They are not mutually exclusive.
A production system might use both.
For example:
Users
|
↓
Load Balancer
/ | \
↓ ↓ ↓
App A App B App C
8 CPU 8 CPU 8 CPU
\ | /
\ | /
Database
Each server can be vertically scaled when necessary, while additional servers can be added horizontally as traffic grows.
This is often a more realistic way to think about scaling:
Vertical and horizontal scaling are tools, not competing ideologies.
What About Cost?
Scaling decisions aren't only technical decisions.
They are also cost decisions.
Suppose your application is reaching the limits of a small server.
You could upgrade it:
Small Server
↓
Bigger Server
Or you could add several smaller instances:
Small Server
Small Server
Small Server
The cheaper option depends on:
- Workload
- Infrastructure provider
- Performance requirements
- Redundancy requirements
- Operational overhead
- Expected growth
So the decision should consider:
- Infrastructure cost
- Engineering complexity
- Performance
- Availability
- Expected growth
- Operational overhead
There is rarely a universal answer.
When Should You Choose Vertical Scaling?
Vertical scaling can make sense when:
Your application is small
You don't need distributed infrastructure just because large companies use it.
Simplicity matters
A single powerful machine can be easier to operate and debug.
Traffic is predictable
If workload growth is manageable, scaling up may be sufficient.
The workload benefits from a larger machine
Some workloads are easier to handle with more resources on a single node.
When Should You Choose Horizontal Scaling?
Horizontal scaling becomes attractive when:
Traffic is growing significantly
A single machine may no longer provide enough capacity.
High availability matters
Multiple instances can reduce dependence on a single machine.
You expect unpredictable traffic
Additional instances can provide more capacity during traffic spikes.
The application can be distributed effectively
Stateless services are generally easier to scale horizontally.
But remember:
Horizontal scaling gives you more capacity, but it also gives you more distributed-system problems to solve.
Common Scaling Mistakes
1. Scaling Before Measuring
Don't add infrastructure simply because you think your system might become slow.
Measure first.
Look at:
- CPU utilization
- Memory
- Request latency
- Throughput
- Database performance
- Error rates
Then identify the bottleneck.
2. Assuming More Servers Solve Everything
Adding application servers won't fix:
- A slow database
- An inefficient query
- An overloaded external API
- Poor application code
- Network limitations
Always ask which component is actually limiting the system.
3. Ignoring State
Horizontal scaling becomes much harder when application instances depend heavily on local state.
If Server A knows something that Server B doesn't, distributing requests becomes more complicated.
4. Overengineering Too Early
If your application has 100 users, you probably don't need an architecture designed for 100 million users.
Design for your actual requirements while keeping future growth in mind.
5. Ignoring Failure Scenarios
Ask:
What happens if one server crashes?
A scalable system should not only handle more traffic.
It should also handle failures gracefully.
A Simple Mental Model
When thinking about scaling, remember:
More Users
↓
More Traffic
↓
More Work
↓
Find the Bottleneck
↓
Choose a Scaling Strategy
↓
Measure Again
↓
Repeat
This is much more useful than simply memorizing:
Vertical = bigger machine
Horizontal = more machines
Those definitions are important.
But the engineering reasoning behind the choice is what matters.
The Real Lesson
System design isn't about choosing the most sophisticated architecture.
It's about understanding the problem first.
A small application might need:
Server → Database
As it grows, it might evolve into:
Users
↓
Load Balancer
↓
Multiple App Servers
↓
Database
Later, the database may become the bottleneck.
Then you may need caching, replication, partitioning, or other techniques.
The architecture evolves because the requirements evolve.
That's the real purpose of system design.
Conclusion
Vertical and horizontal scaling are two fundamental strategies for handling growth in software systems.
Vertical scaling gives an existing machine more resources.
Horizontal scaling adds more machines or instances.
Vertical scaling is often simpler and can be an excellent choice for smaller or predictable workloads. Horizontal scaling can provide greater capacity and make redundancy easier, but it introduces additional distributed-system complexity.
The important lesson isn't to memorize which one is "better."
Instead, learn to ask:
What is growing?
What is the bottleneck?
What capacity do we need?
How much complexity can we afford?
What happens when a component fails?
A good system doesn't start with:
"Let's use 20 servers."
It starts with:
"Let's understand the requirements and find the bottleneck."
And as the system grows, the architecture should evolve with it.
That's the mindset that makes system design useful—not simply knowing more components, but knowing when and why to use them.
Key Takeaways
- Vertical scaling means adding resources to an existing machine.
- Horizontal scaling means adding more machines or instances.
- Vertical scaling is simpler but has resource and practical limits.
- Horizontal scaling can provide greater capacity and redundancy, but introduces distributed-system complexity.
- Load balancers commonly distribute traffic across horizontally scaled application servers.
- Stateless application servers make horizontal scaling easier.
- Adding more servers doesn't help if another component is the bottleneck.
- Scaling decisions should consider performance, availability, cost, complexity, and expected growth.
- Scaling is an iterative process: measure → identify bottleneck → scale → measure again.
- The best architecture is not the biggest architecture; it's the one that fits the requirements.
Top comments (0)