DEV Community

Cover image for Stop Scaling Out: Your Server Might Just Need More RAM
Sanu Khan
Sanu Khan

Posted on

Stop Scaling Out: Your Server Might Just Need More RAM

#architecture #backend #distributedsystems #systemdesign

Horizontal scaling sounds architectural. Vertical scaling sounds temporary.

So when an application starts struggling, the instinct is often:

Add instances. Add a load balancer. Scale out.

But sometimes you don't have a distributed-systems problem.

You have a small-server problem.

The Scenario

Imagine your API runs on:

2 vCPU
4 GB RAM
1 application instance
Enter fullscreen mode Exit fullscreen mode

Traffic grows.

CPU starts touching 80%.

Latency increases.

The architecture discussion immediately becomes:

        Load Balancer
             |
      ┌──────┼──────┐
      ▼      ▼      ▼
    App 1  App 2  App 3
Enter fullscreen mode Exit fullscreen mode

But compare that with:

8 vCPU
16 GB RAM
1 application instance
Enter fullscreen mode Exit fullscreen mode

For some workloads, that upgrade might solve the immediate problem with far less operational complexity.

Vertical scaling increases the resources available to an existing machine. Horizontal scaling adds machines and distributes work between them.

Horizontal Scaling Has a Hidden Invoice

Adding another instance isn't simply:

1 server → 2 servers
Enter fullscreen mode Exit fullscreen mode

Your application now has to behave correctly when two independent processes handle requests.

Suddenly you need to think about:

  • session state
  • shared caches
  • distributed locks
  • background jobs
  • duplicate processing
  • connection pools
  • load balancing
  • deployment coordination
  • observability across instances

And eventually:

"We added more application servers."

              ↓

"Why is the database dying?"
Enter fullscreen mode Exit fullscreen mode

Horizontal scaling increases capacity and can improve fault tolerance, but it also introduces networking, coordination and consistency complexity.

But Vertical Scaling Has a Ceiling

This doesn't mean:

Always buy a bigger machine.

Eventually you hit limits.

4 GB  → 8 GB  → 32 GB  → 128 GB
2 CPU → 4 CPU → 16 CPU → 64 CPU
Enter fullscreen mode Exit fullscreen mode

At some point the next machine becomes disproportionately expensive—or simply isn't large enough.

More importantly, one enormous server is still one server.

If it disappears:

BIG SERVER
    ❌
     |
 Entire service unavailable
Enter fullscreen mode Exit fullscreen mode

Vertical scaling can increase capacity without major architectural changes, but hardware limits and single-node failure remain important constraints.

The Better Question

Don't ask:

Horizontal or vertical?

Ask:

What constraint am I actually hitting?

CPU saturated?
      ↓
Can scaling up solve it economically?
      ↓
YES ──────────────► Scale Up
      │
      NO
      ↓
Can workload be distributed safely?
      ↓
YES ──────────────► Scale Out
Enter fullscreen mode Exit fullscreen mode

And don't confuse capacity with availability.

A bigger machine may solve capacity.

Multiple machines may improve redundancy.

Those are related—but different—engineering problems.

The Architecture I Prefer

For many systems:

Start simple
     ↓
Measure
     ↓
Scale vertically
     ↓
Measure again
     ↓
Remove state from compute
     ↓
Scale horizontally when justified
Enter fullscreen mode Exit fullscreen mode

Not every application needs distributed architecture on day one.

And horizontal scaling isn't automatically more mature engineering.

Sometimes the better architectural decision is knowing when not to distribute the system yet.

Because scalability isn't about having more servers.

It's about knowing where your next bottleneck will appear.

What do you usually reach for first when capacity becomes a problem: a bigger machine or another machine?

Top comments (0)