This is Part 9 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear.
Look at what we've built so far.
We started with a single server. We added more application servers when one wasn't enough, spreading traffic horizontally across many machines. We added a cache to stop the database from answering the same question thousands of times. We taught that cache to stay honest as data changed underneath it. We added read replicas to distribute the remaining read traffic across multiple database copies, so the primary could focus on writes.
At each step, the system got more capable. At each step, a new bottleneck appeared just past the solution we'd just built.
For a while after adding read replicas, things were genuinely good. Reads scaled. The primary handled writes. The application felt fast.
But the application kept growing. And something that had nothing to do with traffic started becoming a problem.
The data itself.
--
Section 1: When One Database Becomes Too Big
Traffic is one dimension of growth. Data is another. And they don't always move together.
Every time a new user signs up, their record gets written to the database. Every order, every message, every transaction, every notification, every log entry accumulates. Most of it never gets deleted. Regulations sometimes require keeping it for years. Business needs require querying it at any time.
For a while, this is fine. A database that holds ten million user records is manageable. One that holds fifty million is still fine with the right hardware. But at some point, something shifts.
Backups start taking hours instead of minutes. A backup that used to finish before anyone noticed is now still running when the morning traffic surge begins.
Indexes grow large enough that keeping them in memory becomes difficult. An index that once fit comfortably in RAM now spills to disk, and suddenly queries that were instant start slowing down, not because the query is wrong, but because the index that makes it fast is too big to hold in memory all at once.
Maintenance operations that used to be routine become risky. Adding a column to a table with two billion rows isn't a quick command anymore. It locks the table, it takes hours, and the application has to work around it.
And writes keep coming. Every new user, every new transaction, every new event. The write load on the primary database grows steadily, not because users are doing anything unusual, but simply because there are more of them every day.
The engineers look at their monitoring dashboards. The read replicas are handling reads beautifully. But the primary database is under increasing strain, and not just from write volume. It's the sheer size of everything it has to manage.
So they do what engineers usually try first: they upgrade the hardware.
--
Section 2: Why Bigger Servers Stop Helping
More RAM, faster disks, more CPU cores. For a while, vertical scaling works. The database gets more room to breathe and the slowdowns ease.
But vertical scaling has a ceiling, and that ceiling is physical.
There are only so many CPU cores you can put in one machine. There is only so much RAM a single server can hold, and that limit is measured in terabytes, not infinite amounts. The fastest possible disk is still a single disk attached to a single machine. And even before you hit the absolute physical limits, you hit the economic ones. The price of a server doesn't scale linearly with its capability. A machine with twice the RAM doesn't cost twice as much. It costs three or four times as much, and at a certain point the cost becomes unreasonable.
More importantly, a bigger machine doesn't solve the fundamental problem. The data is all in one place. Every query, no matter how fast the hardware running it, still has to navigate the same enormous dataset. Every backup still has to copy the same enormous amount of data. Every index still has to track the same enormous number of rows.
You're not reducing the problem. You're just buying a bigger container for it.
Vertical Scaling:
Small Server → Medium Server → Large Server → ???
[ 64GB RAM ] [ 256GB RAM ] [ 1TB RAM ]
[ 1TB disk ] [ 8TB disk ] [ 64TB disk ]
Each jump costs significantly more.
Eventually, there is no larger option.
The problem is still the same size.
At some point, an engineer asks the question that changes the direction of the whole conversation.
What if we stopped trying to make the database bigger, and started splitting the data across multiple databases instead?
--
Section 3: One Database Can't Hold Everything: Database Sharding
Let's think through that idea slowly, because it's a significant shift from everything we've done before.
With read replicas, we created multiple copies of the same database. Every replica had all the data. The benefit was that reads could be spread across many machines. But the data itself still lived in one place, fully, on the primary.
This new idea is different. Instead of copying the data, we divide it.
Imagine your application has 90 million users. Right now, all 90 million live in a single users table in a single database. What if instead, you split them across three separate databases?
Database 1: Users with IDs 1 to 30 million
Database 2: Users with IDs 30 million to 60 million
Database 3: Users with IDs 60 million to 90 million
Each database now holds only one third of the total data. Each one is a third of the size. Each one has smaller indexes. Each one can be backed up in a third of the time. Each one receives only the writes that belong to its slice of users.
When a user logs in, the application looks at their user ID, figures out which database holds their data, and talks to that one.
Without Sharding:
All 90 million users
|
v
[ Single Database ] <-- enormous, under pressure
With Sharding:
Users 1-30M → [ Database Shard 1 ]
Users 30-60M → [ Database Shard 2 ]
Users 60-90M → [ Database Shard 3 ]
Each individual database is now a manageable size. As the total number of users grows past 90 million, you add a fourth shard. The existing databases don't have to grow. Only the new shard fills up with new data.
This is Database Sharding. Each individual database is called a shard, and together they hold the complete dataset that used to belong to one database. No shard has everything. Every shard has its piece.
The effect on writes is immediate and direct. Instead of all 90 million users' writes going to one primary, they're now distributed. User activity for the first 30 million users hits one database. Activity for the next 30 million hits another. The write load is divided along with the data.
And each shard can have its own read replicas, if needed. The full architecture composes cleanly with everything we've already built.
--
Section 4: Choosing a Shard
Once you accept the idea of splitting data across multiple databases, a very practical question appears: how does the application know which shard to look in?
This is where the shard key comes in. A shard key is the piece of information you use to decide where a particular piece of data lives. The application uses it to route every query to the right shard.
The example we just used divided users by their ID range. User ID is the shard key, and the range it falls into determines the destination.
User ID 4,523,901 → falls in range 1-30M → Shard 1
User ID 47,112,008 → falls in range 30-60M → Shard 2
User ID 83,400,551 → falls in range 60-90M → Shard 3
This works, and it's easy to reason about. But ID range isn't the only way to divide data. Different applications shard in different ways depending on what makes sense for their data.
A global application might shard by geography. Users in North America go to one database. Users in Europe go to another. Users in Asia go to a third. This keeps data physically close to the users it belongs to, which can reduce how far queries have to travel across a network.
A multi-tenant business application might shard by customer. Every piece of data belonging to Company A lives in one shard. Company B lives in another. Each customer's data is completely isolated from every other customer's.
In every case, the idea is the same: pick a property of the data that lets you divide it predictably, and make sure the application can always determine which shard holds what it needs.
Sharding by User ID range:
User 4M → Shard 1
User 47M → Shard 2
Sharding by Geography:
User in Germany → EU Shard
User in Brazil → Americas Shard
Sharding by Tenant:
Company A data → Shard A
Company B data → Shard B
The application now carries a small piece of routing logic it didn't have before. For every database operation, it has to answer the question: which shard does this belong to? Most of the time this is straightforward, as long as the shard key is always available. If you're looking up a user, you have the user ID. If you're looking up an order, the order is associated with a user, so you have the user ID. The routing decision is quick.
But the new complexity doesn't stop there. Not every problem is about where data lives. Some problems are about whether the data ends up in the right places to begin with.
--
Section 5: The Hidden Trade-off: Uneven Shards
Dividing data by ID range seems clean on paper. Users 1 to 30 million here, 30 to 60 million there. Equal thirds. But numbers don't tell the whole story.
Not all users are equally active.
Imagine your application has been around for several years. The oldest users, those with low user IDs from the early days, tend to be the most engaged. They've built up years of activity, thousands of posts, millions of interactions. They log in every day.
The newest users, those with high user IDs who joined recently, might have just signed up and haven't done much yet. Some of them will never come back.
Now look at what that means for your shards.
Shard 1: Users 1-30M
Old, active users. High write volume.
Large amount of historical data per user.
Frequent queries, heavy index usage.
Running hot.
Shard 3: Users 60-90M
New users, low engagement.
Minimal data per user.
Infrequent queries.
Barely doing anything.
The data is divided evenly by count, but the work is not divided evenly at all. Shard 1 is overloaded. Shard 3 has capacity to spare. You've added complexity to the system, but you haven't actually distributed the pressure evenly.
This problem has a name: a hotspot. A shard that receives disproportionately more traffic or stores disproportionately more active data than the others.
Hotspots happen when the shard key you chose doesn't distribute real-world activity evenly, even if it looks balanced on paper. ID ranges can create hotspots based on user age. Geographic sharding can create hotspots if one region has far more users than others. Tenant sharding can create hotspots if one company is ten times larger than every other.
Dealing with hotspots requires either choosing a better shard key, splitting the overloaded shard into smaller pieces, or distributing data using a more sophisticated strategy that mixes up the routing so no single shard ends up holding all the busiest users.
There are well-established approaches to solving the hotspot problem, but each one adds more complexity. The application's routing logic becomes more involved. Moving data between shards, if you decide the current division is wrong, is a large and careful operation. Cross-shard queries, where you need data that spans multiple shards, require the application to query several databases and assemble the result.
None of these problems are unsolvable. They're the trade-offs that sharding introduces in exchange for the problems it solves. And the same rule that has applied to every technique in this series applies here: sharding solves the problems of scale, and it introduces new problems in return.
--
Conclusion
Let's look at how far the architecture has come since Part 1.
We started with a single server handling a single user. We scaled application servers horizontally so that traffic could be spread across many machines. We added a cache so the database didn't have to keep answering the same questions. We built invalidation strategies to keep that cache accurate. We added read replicas to distribute reads so the primary database could focus on writes.
And now, in Part 9, we've taken the data itself and spread it horizontally across multiple databases. Each shard holds its piece of the whole. Storage scales because each database only grows with the users assigned to it. Writes scale because they're distributed across shards. Maintenance becomes manageable because no single database ever has to hold everything.
The full picture looks something like this.
[ Load Balancer ]
|
.-----------------+-----------------.
| | |
[ App Server ] [ App Server ] [ App Server ]
| | |
v v v
[ Cache ] [ Cache ] [ Cache ]
|
.--------+--------.
| |
[ Shard 1 ] [ Shard 2 ] [ Shard 3 ]
[ Primary ] [ Primary ] [ Primary ]
[ Replica ] [ Replica ] [ Replica ]
[ Replica ] [ Replica ] [ Replica ]
It's a long way from where we started.
But notice something that this architecture still doesn't handle gracefully.
Some operations don't belong to a single shard. Imagine a user sends a money transfer to someone who lives on a different shard. Or the application needs to generate a report that pulls data from every shard simultaneously. Or a new user signs up, and the welcome email, the onboarding notification, the recommendation engine seed, and the analytics event all need to happen as part of the same flow.
These are tasks that are either slow, span multiple shards, or shouldn't block the user from moving on.
Right now, the user has to wait for all of it.
What if they didn't have to?
What if the application could say: "I've saved your order. The rest will happen in the background." And then actually make that happen, reliably, even if it takes a few seconds?
That idea, of separating the work that has to happen immediately from the work that can happen whenever the system gets to it, is where Part 10 begins.
Top comments (0)