At 8:30 PM on a Friday, millions of people open a food app at once. How does it stay fast — and how does the database survive? Not with one trick, but with five layers, each removing load before it reaches the expensive part. Here they are, using Swiggy.
The 5 techniques
| # | Technique | One-line |
|---|---|---|
| 1 | Caching | Keep hot data close, skip the slow DB |
| 2 | Load Balancing | Spread traffic across many servers |
| 3 | CDN | Serve content from near the user |
| 4 | Partitioning / Sharding | Split one huge DB into pieces |
| 5 | Autoscaling | Add/remove servers automatically |
1. Caching
Keep frequently-used data somewhere fast so you don't hit the slow database every time.
Where caches live:
| Level | Where | Swiggy example |
|---|---|---|
| Client | On your phone | Your cart, cached locally |
| CDN | Edge servers worldwide | Restaurant images |
| Distributed | A shared cache cluster | Menu, sessions (Redis) |
How writes interact with the cache:
| Strategy | How it works | Trade-off |
|---|---|---|
| Cache-aside | App fills the cache on a miss | Simple; first read is slow |
| Write-through | Write to cache + DB together | Always fresh; writes slower |
| Write-back | Write to cache now, DB later | Fast writes; risk of loss if cache dies |
Swiggy: menu in a distributed cache (Redis), images on a CDN, cart on the client. Different data, different cache home.
Azure: Azure Cache for Redis.
The golden rule: cache things where "a few seconds stale is fine" (menu, restaurant list). Never cache things where stale = wrong (wallet balance, payment status).
2. Load Balancing
Spread incoming traffic across many identical servers so no single one drowns.
┌→ Server 1
Users → [ LB ]┼→ Server 2
└→ Server 3 (LB picks which one)
Two types — L4 vs L7:
| L4 (Transport) | L7 (Application) | |
|---|---|---|
| Sees | IP + port only | Full HTTP (URL, headers) |
| Speed | Faster, dumber | Smarter, slightly slower |
| Can do | Raw distribution | Route /search vs /pay to different pools |
How it picks a server:
| Algorithm | Rule |
|---|---|
| Round-robin | Next server in turn |
| Least connections | The least-busy server |
| IP hash | Same user → same server (sticky) |
Swiggy: millions of requests spread across hundreds of servers. An L7 balancer routes /search to search servers and /payment to payment servers.
Azure: Load Balancer (L4), Application Gateway / Front Door (L7).
3. CDN — Content Delivery Network
Copy static content to servers around the world, so users get it from nearby.
❌ No CDN: user in Chennai → fetches image from a US server (slow, far)
✅ CDN: user in Chennai → fetches from a Chennai edge server (fast, near)
How content actually gets there — it's pull-based, not push:
- First user in a region requests
logo.png→ the edge has nothing → it fetches from your origin, stores a copy, serves it. - Next users in that region → served from the edge copy; origin untouched.
- Each edge manages its own cache, so another region still pulls from origin until its edge caches it too.
You never "upload" files to a CDN. The first request in each region fills the cache — which is why it needs almost no setup.
Three things beginners get wrong:
| Myth | Reality |
|---|---|
| "The CDN caches automatically" | On modern services (like Azure Front Door) caching is opt-in per route — you switch it on |
| "It's safe to cache everything" | Never cache authenticated or per-user pages — a shared edge cache can serve one user's data to another |
| "Staleness is controlled" | If your origin sends no Cache-Control, the default TTL can be days — set your own |
Swiggy: restaurant photos, app JS/CSS, menu images — all static, all cached at the nearest edge. Your biryani photo loads instantly because it's cached 20km away, not 12,000km. But the logged-in order page is never cached — that stays dynamic.
Key: a CDN is caching for location. Distance = latency, so serve from close by.
4. Partitioning / Sharding
Split one giant database into smaller pieces (shards), each holding part of the data.
The make-or-break decision is the partition key — how you split.
| Strategy | How | Watch out |
|---|---|---|
| Hash |
hash(key) decides the shard |
Even spread, but no range queries |
| Range | A–M here, N–Z there | Simple, but can create hotspots |
| Directory | A lookup table maps key → shard | Flexible, but the lookup is a dependency |
| ✅ Good | ❌ Bad |
|---|---|
| Scales beyond one machine | Cross-shard queries are painful |
| Each shard is smaller/faster | The partition key is ~irreversible |
Swiggy: orders partitioned by cityId or userId — Bengaluru's orders on one shard, Delhi's on another.
Azure: this is exactly the Cosmos DB partition key — the single most important, effectively permanent design choice you make up front.
⚠️ The trap: a bad key creates a hot partition — one shard takes all the traffic while the others sit idle. Choose a key that spreads load evenly.
5. Autoscaling
Automatically add servers when busy, remove them when quiet.
3 PM (quiet): ▪▪ (3 servers)
8:30 PM (peak): ▪▪▪▪▪▪▪▪▪▪ (auto-scaled to 20)
2 AM (dead): ▪ (scaled back to 1)
| Type | Rule |
|---|---|
| Reactive | Scale when CPU/queue crosses a threshold |
| Predictive | Scale ahead of a known pattern (the dinner rush) |
| ✅ Good | ❌ Bad |
|---|---|
| Pay only for what you use | Scaling isn't instant (there's lag) |
| Handles spikes automatically | Cold starts on new instances |
Swiggy: scale up before the 8 PM rush, scale down after midnight. Never pay for 500 servers at 3 AM.
Azure: VM Scale Sets, AKS autoscaler, App Service autoscale.
How they connect
CDN → serves static content from the edge (never hits your servers)
Load Balancer → spreads the dynamic traffic that remains
Caching → cuts DB reads for that traffic
Partitioning → splits the DB when caching isn't enough
Autoscaling → adjusts server count to match live demand
Each layer removes load before it reaches the expensive part: CDN catches the easy stuff → the load balancer spreads the rest → the cache skips the DB → partitioning shares the DB load → autoscaling flexes capacity.
The whole thing in one line
Scaling isn't one big trick — it's five layers peeling load away before it reaches the database. Serve static from the edge, spread the rest, cache what you can, split what you can't, and flex capacity to match the crowd.
Top comments (0)