Imagine a business with a few 100 users. We have a server; we have a database that is good enough to serve 100 users.
Suddenly, our business got a high demand, and the number of active users increased from 100 users to 100K users.
Now we have a problem. Our system started breaking, requests failed, customers became unhappy, and soon our business, which is our only hope, is about to collapse.
What actually happened here? Why did so many failures happen in our system? Why did it happen specifically when the number of users increased from 100 to 100K?
Let's dig a little deeper into it.
Root Cause
We used a simple architecture that was good enough when we had 100 users. Our system handled all users without any issues, and the customers were happy. But when the number of active users increased, the number of requests coming to our system increased exponentially, which made it very, very difficult for our servers to provide any response. In short, our system doesn’t SCALE when the number of users increases.
The Solution? We must scale our system to support 100K users using Vertical or Horizontal Scaling.
Vertical Scaling
During our architecture review meeting, we developed a plan and raised the following questions.
- Why can’t we provide more power to our existing servers?
- Why can’t we add more computing resources — upgrade our hardware, add more CPUs, and stuff it with RAM?
And that is what is called Vertical Scaling.
In Vertical Scaling, we try to provide more resources to our existing machine. Its advantages include:
- No need to make additional code changes, since we aren’t changing anything on the core architecture.
- It’s quite simple to do, thus saving our time.
But the discussion doesn’t end there. Our senior engineers still have many questions.
- What if the server still goes down even after adding more power? It’s a single point of failure.
- Even if we keep adding more resources and power, how much can we add? There is a limit to it.
- Isn’t it too expensive?
All these questions kept everyone in the room silent — until one person came up with another suggestion.
Horizontal Scaling
Instead of one single powerful, heavy server, why can’t we use a few tiny servers? Put some kind of load balancer (which we can discuss later) in front of these tiny servers and route the requests to different servers.
And this is called Horizontal Scaling.
In Horizontal Scaling, we will try to have multiple small servers that can serve the angry customers. Its advantages include:
- No single point of failure.
- If one server fails, another server can fulfil the request.
- It can scale quite well by adding more servers as needed.
However, the room wasn’t entirely sold yet. The senior engineers were quick to point out the new headaches this distributed approach would bring:
- Don’t we have to deploy, manage, and update our code across all these separate servers simultaneously?
- Isn’t it a complex setup?
- Won’t having multiple servers trying to read and write at the exact same time make us highly prone to data syncing issues?
Final Decision
The truth is, modern hardware is astonishingly powerful. We could easily survive the 100K user surge just by scaling vertically — buying a top-tier machine and calling it a day. In fact, some massively successful platforms handle enormous traffic loads primarily through highly optimized vertical scaling.
However, horizontal scaling offers unmatched high availability. It eliminates that terrifying single point of failure and prepares the architecture for the day the user base hits one million, even though it introduces the immediate burden of configuring load balancers, deployment pipelines, and complex database synchronization.
Which path should our engineers take?
In system design, there is rarely a single “correct” answer. The decision ultimately depends on a business’s budget, engineering bandwidth, and long-term vision. Some teams choose to upgrade the server to save precious engineering hours, while others embrace the complexity of a distributed system to guarantee uptime.
Ultimately, this is how systems evolve. We start with a basic, simple setup, and when the necessity arises, we scale our architecture to ensure our system remains reliable and our customers stay happy.
But if we do go the horizontal route, how exactly do we manage all that traffic without causing a massive pile-up? That’s where our “Load Balancer” comes in.
Stay tuned for the next post, where we’ll dive deep into Load Balancers!
System design is vast — if you have a different perspective or caught a mistake, correct me in the comments!
Top comments (0)