DEV Community

Cover image for Scalability Principles
NISCHIT D S
NISCHIT D S

Posted on

Scalability Principles

#systemdesign #beginners #learning

| What is Scalability?

Scalability is the ability of a system to handle increased load by adding resources. A scalable system can grow to accommodate more users, more data, or more transactions without a significant degradation in performance.

Think of it like a restaurant:

  • A small café can serve 20 customers comfortably
  • As it gets popular, you need to scale
  • You can make the kitchen bigger (vertical scaling)
  • Or open more locations (horizontal scaling)

| Why Does Scalability Matter?

  1. Viral growth : Your startup might go viral overnight
  2. Unpredictable traffic : Black Friday, viral content, news events
  3. Cost efficiency : Pay for what you need, when you need it
  4. User experience : Slow apps lose users (53% abandon if >3s load)

| Vertical vs Horizontal Scaling

There are two fundamental approaches to scaling:

Vertical Scaling (Scale Up)

Add more power to your existing machine - more CPU, RAM, or storage.

Before: 4 CPU, 8GB RAM, 100GB Storage
After:  32 CPU, 128GB RAM, 2TB Storage
Enter fullscreen mode Exit fullscreen mode 
**Pros Of Vertical Scaling**

- Simple to implement
- No code changes needed
- Lower complexity
- ACID compliance easier
Enter fullscreen mode Exit fullscreen mode 
**Cons Of Vertical Scaling**

- Hardware limits exist
- Single point of failure
- Expensive at high end
- Downtime during upgrades
Enter fullscreen mode Exit fullscreen mode

| Horizontal Scaling (Scale Out)

Add more machines to distribute the load.

Before: 1 server handling all traffic
After:  10 servers sharing the load
Enter fullscreen mode Exit fullscreen mode 
**Pros Of Horizontal Scaling**

- Theoretically unlimited
- Better fault tolerance
- Cost-effective at scale
- No single point of failure
Enter fullscreen mode Exit fullscreen mode 
**Cons Of Horizontal Scaling**

- More complex architecture
- Data consistency challenges
- Requires load balancing
- Network latency between nodes
Enter fullscreen mode Exit fullscreen mode 
**Real-World Example: Instagram**

- Started with a single server
- Scaled vertically until they hit limits
- Moved to horizontal scaling with sharded databases
- Now runs on thousands of servers across multiple data centers
Enter fullscreen mode Exit fullscreen mode

| Key Metrics for Scalability
To measure scalability, you need to understand these key metrics:

1. **Throughput** : Requests per second (RPS) your system can handle.

Example: "Our API handles 10,000 RPS"
Enter fullscreen mode Exit fullscreen mode

2. **Latency** : Time taken to process a single request (measured in milliseconds).

Important: P50, P95, P99 percentiles matter more than averages!

P50: 50ms  (50% of requests faster than this)
P95: 100ms (95% of requests faster than this)
P99: 500ms (99% of requests faster than this)
Enter fullscreen mode Exit fullscreen mode

The P99 catches the "unlucky" slow requests that affect user experience.

3. **The Scalability Equation**

Throughput = Concurrency / Latency
Enter fullscreen mode Exit fullscreen mode

If your average latency is 100ms and you have 100 concurrent workers:

  • Throughput = 100 / 0.1 = 1,000 RPS

4. **Amdahl's Law** : The speedup of a program using multiple processors is limited by the sequential fraction:

Speedup = 1 / (S + (1-S)/N)
Enter fullscreen mode Exit fullscreen mode

Where S = sequential fraction, N = number of processors.

Key Insight: If 10% of your code is sequential, adding infinite processors only gives 10x speedup maximum!

| Common Scalability Patterns

1. **Load Balancing** : Distribute requests across multiple servers to prevent any single server from becoming a bottleneck.

        ┌─────────────────┐
        │  Load Balancer  │
        └────────┬────────┘
       ┌─────────┼─────────┐
       ▼         ▼         ▼
   ┌───────┐ ┌───────┐ ┌───────┐
   │Server1│ │Server2│ │Server3│
   └───────┘ └───────┘ └───────┘
Enter fullscreen mode Exit fullscreen mode

2. **Caching **: Store frequently accessed data in memory for faster retrieval.

Rule of thumb: Cache 20% of data = handle 80% of requests (Pareto principle)

3. **Database Sharding** : Split your database across multiple machines based on a shard key.

Shard Key: user_id

User 1-1M    → Shard A
User 1M-2M   → Shard B  
User 2M-3M   → Shard C
Enter fullscreen mode Exit fullscreen mode

4. **Microservices** : Break your monolith into smaller, independently scalable services.

Monolith: One thing does everything
    ↓
Microservices: Each service scales independently

Auth Service  → 2 instances (light load)
Image Service → 20 instances (heavy load)
API Gateway   → 5 instances (medium load)
Enter fullscreen mode Exit fullscreen mode

5. **Asynchronous Processing **: Use message queues to handle time-consuming tasks.

User Request → API → Queue → Worker → Done!
     ↓
  Quick Response (Task queued!)
Enter fullscreen mode Exit fullscreen mode

Perfect for: Email sending, image processing, report generation

6. **Read Replicas** : Create read-only copies of your database.

┌──────────────┐
│  Primary DB  │ ← Writes
└──────┬───────┘
       │ Replicates
   ┌───┴───┐
   ▼       ▼
┌─────┐ ┌─────┐
│Rep 1│ │Rep 2│ ← Reads
└─────┘ └─────┘
Enter fullscreen mode Exit fullscreen mode

Works great for read-heavy workloads (80%+ reads).

| Stateless vs Stateful Architecture

1. **Stateless Services **: Each request contains all information needed to process it. The server doesn't remember previous requests.

Request: { token: "abc123", action: "getData" }
         ↓
Any Server → Process → Response
Enter fullscreen mode Exit fullscreen mode 
**Benefits:**

- Easy to scale horizontally
- Any server can handle any request
- Simple load balancing
- No session affinity required
Enter fullscreen mode Exit fullscreen mode

2. **Stateful Services** : The server maintains client state between requests.

**Challenges:**

- Need sticky sessions or shared state
- More complex failover
- Session storage required
Enter fullscreen mode Exit fullscreen mode

Best Practice: Externalize State

Instead of:
Server Memory → Session Data

Use:
┌─────────┐    ┌───────────┐    ┌─────────┐
│Client A │───▶│  Server 1 │───▶│  Redis  │
└─────────┘    └───────────┘    │ (State) │
┌─────────┐    ┌───────────┐    │         │
│Client B │───▶│  Server 2 │───▶│         │
└─────────┘    └───────────┘    └─────────┘
Enter fullscreen mode Exit fullscreen mode

This gives you the benefits of both: stateful behavior with stateless architecture.

| Real-World Example: Scaling Twitter

Let's see how a real company approached scalability:

The Challenge

- 500 million tweets per day
- Millions of users refreshing feeds
- Celebrity tweets can spike traffic 100x
Enter fullscreen mode Exit fullscreen mode

Solutions Twitter Used

1. Fan-out on Write : When you tweet, it's pre-computed into your followers' timelines.

You tweet → Immediately written to all followers' timeline cache

Trade-off: More storage, but faster reads.

Exception: Celebrities use fan-out on read (accounts with millions of followers).
Enter fullscreen mode Exit fullscreen mode

2. Redis for Timeline Cache : Each user's home timeline is cached. Recent tweets always available instantly.

*3. Separate Read/Write Paths : *

Tweets → Write Cluster (optimized for writes)
Reads  → Read Cluster (optimized for reads)
Enter fullscreen mode Exit fullscreen mode

Eventually consistent (you might not see your tweet for a few seconds).

4. Geographic Distribution : Data centers worldwide. Users routed to nearest location.

| Key Takeaways

  • Scalability is about handling growth without sacrificing performance
  • Horizontal scaling **(adding machines) is generally preferred over **vertical scaling (bigger machines)
  • Stateless architectures are easier to scale than stateful ones
  • Caching, load balancing, and database sharding are essential patterns
  • Measure throughput, latency, and availability to track scalability
  • Real systems use multiple strategies tailored to specific use cases

Top comments (0)