This is Part 11 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear.
We've spent the last several articles making things faster by reducing work.
We stopped the database from answering the same question twice. We distributed reads across multiple replicas. We split data across shards so no single machine ever had to hold everything. We moved slow background tasks off the critical path so users don't have to wait for them.
Each of these solutions attacked the same underlying problem: the system was doing more work than it needed to, or concentrating too much work in one place.
But Part 10 ended by pointing at a different kind of problem entirely. One that has nothing to do with work.
A user in Bengaluru, sending a request to a server in Virginia, has to wait for that request to travel halfway around the world and come back. The server might be perfectly healthy. The database might be responding instantly. The cache might be working exactly as intended. And the user still waits.
Because the content is simply too far away.
That's the problem this article is about.
--
Section 1: When the Server Isn't the Problem
Imagine the application is running well. It's hosted in a data center in Virginia. Most of the early users are in the United States, and for them, the experience is fast. Pages load quickly. Images appear immediately. Everything feels responsive.
Then the application grows. Users in India start signing up. Then Germany. Then Brazil, Singapore, Japan.
And the complaints start coming in.
"The website is slow."
"Images take forever to load."
"It feels sluggish."
The engineering team checks everything they know to check.
CPU usage is normal. The application servers aren't overloaded.
RAM is fine. No memory pressure.
The database is responding quickly. Query times look healthy.
The cache hit rate is high. Most repeated queries aren't even reaching the database.
The message queue is draining normally. Background work isn't backed up.
Nothing is broken. Nothing is overloaded. The system, by every metric the team knows how to measure, looks fine.
And yet users in India are waiting three or four seconds for the page to load.
So the question becomes: if the server is healthy, why does the application still feel slow?
--
Section 2: Distance Has a Cost
The answer is something that no amount of server optimization can fix: physical distance.
When a user in Bengaluru opens a web page hosted in Virginia, their request has to travel thousands of kilometers across the internet. It hops through cables, routers, undersea fiber lines, and data centers spread across the globe. The server in Virginia receives it, processes it, and sends a response back. That response travels the same distance in reverse.
The server's processing time might be 10 milliseconds. But the round-trip across the network might add another 200 or 300 milliseconds on top of that. Load a page with twenty images, each making its own round trip, and those milliseconds compound into seconds.
User in Bengaluru Server in Virginia
| |
|--- request (travels ~13,000 km) -------> |
| | (processes in 10ms)
|<-- response (travels ~13,000 km) ------- |
|
Total experienced latency: 250-350ms per request
Page with 20 resources: potentially 2-4 seconds
This is network latency, and it is governed by physics. Data can only travel so fast. Light through fiber-optic cable moves at roughly two thirds the speed of light in a vacuum. The distance between Bengaluru and Virginia is real, and it takes real time to cross it.
No code change, no database optimization, no caching strategy, and no queue will change this. You cannot make light travel faster.
Server processing time is only part of the total time a user experiences. The network matters too, and when users are geographically far from the server, the network dominates everything else.
This means that every technique we've discussed so far, as valuable as it is, solves the wrong problem for these users. Their problem isn't that the server is slow. Their problem is that the server is far away.
If you can't bring the user closer to the server, the only remaining option is to bring the server closer to the user.
--
Section 3: Move the Content Closer
That phrase sounds obvious once you say it, but it has a real implication worth sitting with.
Most of what a web application serves is the same for everyone. A product image is the same image whether it's requested by someone in Berlin or someone in São Paulo. A company's logo is the same file regardless of who's asking for it. The JavaScript that makes the page interactive, the CSS that makes it look right, the fonts that make the text readable: all of that is identical for every single user.
When a user in India requests a product page, the application sends HTML, CSS, JavaScript, a dozen images, a font file, and more. The actual personalized piece of that response, the part that's different for each user, is often a small fraction of the total data being transferred.
The rest is identical content, being sent from the same server in Virginia, to millions of different users spread around the world.
That's where the waste lives. Not in the server working too hard, but in the same static content making the same long journey over and over again, in every direction, to every corner of the globe.
If a thousand users in Germany are all requesting the same product image, why is that image traveling from Virginia to Germany a thousand times? What if it only had to make that trip once, and then could be served from somewhere much closer to Germany for all subsequent requests?
That's the question that leads naturally to the solution.
--
Section 4: Meet the CDN
A Content Delivery Network, or CDN, is a network of servers distributed around the world. Instead of every user's request traveling all the way to the origin server, users can receive content from a server that's geographically nearby.
These distributed servers are often called edge locations or Points of Presence, sometimes abbreviated as PoPs. A CDN provider might operate hundreds of them, in cities across every continent. The idea is that no matter where a user is in the world, there's an edge location within a short network distance of them.
WITHOUT CDN:
User in India --> Internet --> Origin Server (Virginia)
User in Germany --> Internet --> Origin Server (Virginia)
User in Brazil --> Internet --> Origin Server (Virginia)
(Same content traveling the same long distances, repeatedly)
WITH CDN:
User in India --> Edge Location (Mumbai)
User in Germany --> Edge Location (Frankfurt)
User in Brazil --> Edge Location (São Paulo)
(Content served from nearby, origin server rarely involved)
The edge location has a cached copy of the content. When the user in India requests a product image, they get it from Mumbai, not Virginia. The round trip is a fraction of what it was. The image loads in milliseconds instead of hundreds of milliseconds.
This should feel familiar, because it's an idea you've seen before.
In Part 6, we learned not to make the database answer the same question repeatedly. We put a cache in front of it. Instead of recalculating an answer over and over, we stored it once and handed it out.
The CDN is that exact idea, applied to geography.
Instead of sending the same content across the world over and over, you store it at locations closer to the people who need it. The origin server answers once. The edge location serves the rest.
The content doesn't get faster to generate. It gets faster to receive.
--
Section 5: What Happens When the Content Isn't There?
The first time a user in Frankfurt requests a file that the Frankfurt edge location has never seen, the CDN doesn't have it yet. That's a cache miss.
When this happens, the edge location does what any cache does on a miss: it goes to fetch the content from the origin server, delivers it to the user, and stores a copy locally for next time.
First request to an edge location (Cache Miss):
User in Frankfurt
|
v
Frankfurt Edge Location: "I don't have this."
|
v
Origin Server (Virginia): returns the file
|
v
Frankfurt Edge Location: stores a copy
|
v
User in Frankfurt: receives the file
(this request was still slow, but it seeded the edge)
All subsequent requests (Cache Hit):
User in Frankfurt
|
v
Frankfurt Edge Location: "I have this." --> serves immediately
Every request after the first one is served from Frankfurt. The origin server doesn't get involved again until the cached copy expires.
The CDN sits between users and the origin server, absorbing the majority of requests for static content. The origin server, which might be doing a lot of other work, suddenly has far fewer requests to handle. Not because the content became less popular, but because the edge locations are handling most of it.
This matters especially during traffic spikes. A viral moment where millions of users suddenly request the same image or video doesn't flood the origin server. The edge locations absorb it. Each edge location serves its local cluster of users from its own cache, and the origin server sees only a fraction of the total traffic.
--
Section 6: The Trade-off: Freshness vs Distance
By this point in the series, you already know what's coming.
Every caching strategy introduces the same fundamental tension: the faster you serve content, the more you risk serving content that's no longer current.
CDNs are no different.
Imagine a company updates its website's logo. The new logo file is uploaded to the origin server. But edge locations around the world still have the old logo cached. Users in Tokyo, Lagos, and Buenos Aires continue to see the old version until the cached copy at each edge location expires and the new one is fetched.
For a logo change, that's a minor inconvenience. For a critical bug fix in a JavaScript file, it could mean users in some regions are running broken code for hours after the fix was deployed.
This is the same cache invalidation problem from Part 7, now playing out at a global scale across dozens of edge locations instead of a single cache layer.
CDN providers offer ways to handle this. You can set expiration times on content, telling the edge location how long to hold onto a cached copy before checking for a fresher version. You can issue a purge command that forces edge locations to drop their cached copies immediately, so the next request triggers a fresh fetch from the origin.
But there's a more fundamental limit to where a CDN helps.
Everything we've discussed so far assumes the content is the same for everyone. Static files, images, shared web pages: these are natural fits for edge caching, because serving one cached copy satisfies any user who requests it.
Personalized content is a different story.
A user's bank balance is unique to them. Their private messages are theirs alone. Their personalized dashboard reflects their specific account, preferences, and history. You cannot cache these at an edge location and serve them to other users, because they belong to one person.
Good candidates for CDN caching:
Product images, company logos, marketing pages,
CSS stylesheets, JavaScript files, video content,
font files, public documentation.
Poor candidates for CDN caching:
Bank balances, private messages, personalized feeds,
account settings, real-time inventory, session data.
The CDN is most powerful when many users want the same thing. When every user wants something different, the edge location can't help, and the request has to travel all the way to the origin anyway.
This is the honest picture of what a CDN provides: a dramatic improvement for the substantial portion of web traffic that is static and shared, and no improvement at all for requests that are inherently personal.
--
Conclusion
Let's look at the full map of what we've built across this series.
We started with one server. We've been adding layers ever since, each one solving a specific kind of problem.
The full picture:
Users around the world
|
v
[ CDN Edge Locations ] <-- static content served nearby
|
v (dynamic requests only)
[ Load Balancer ] <-- traffic distributed across servers
|
v
[ Application Servers ] <-- requests handled in parallel
| |
v v
[ Cache ] [ Message Queue ] <-- repeated work absorbed,
| slow work moved to background
v
[ Workers ]
|
v
[ Read Replicas ] <-- read traffic distributed
|
v
[ Database Shards ] <-- data distributed across machines
Each layer in that diagram exists because one specific problem demanded it.
Load balancers appeared because one server couldn't handle the traffic.
Caching appeared because the database kept answering the same questions.
Read replicas appeared because one database couldn't handle all the reads.
Sharding appeared because one database couldn't hold all the data.
Message queues appeared because users shouldn't wait for work that can happen later.
CDNs appeared because content was too far from the people who needed it.
In every case, the pattern was the same. A bottleneck appeared. We found the specific thing that was being concentrated too much in one place or one moment. Then we distributed it.
Traffic. Repeated work. Reads. Data. Background work. Content.
All of it, distributed.
But look at the application itself for a moment. We've been scaling the infrastructure around it, but the application code that runs on those servers has been growing too. Features get added every week. Teams grow. Engineers who joined recently struggle to understand parts of the codebase that were written before they arrived.
Deploying a change to fix a bug in the payment system requires redeploying the entire application, including the parts that handle user profiles, notifications, search, and everything else. A failure in one part can bring down the whole thing.
The infrastructure scales. The team scales. But the application, as a single unified codebase, starts to become a problem of its own.
What happens when one application becomes too big for one team, one codebase, and one deployment to handle?
That's the question Part 12 takes on.
Top comments (0)