When people first learn about object storage, replication sounds like the obvious answer.
Need durability?
Keep three copies.
Need higher availability?
Keep more copies.
Simple.
For years, that's exactly what many storage systems did.
Then storage clusters became enormous.
Imagine storing:
10 PB
of data.
With three-way replication, you don't actually need 10 PB of storage.
You need:
30 PB
Suddenly, durability isn't just a technical problem.
It's a financial one.
At cloud scale, every extra copy costs millions of dollars in disks, racks, power, cooling, and networking.
Storage engineers started asking a different question.
Can we protect data without storing complete copies?
That question led to one of the biggest architectural shifts in modern object storage.
Replication is wonderfully simple
Suppose you upload:
vacation.jpg
Three-way replication looks like this:
Node A
vacation.jpg
Node B
vacation.jpg
Node C
vacation.jpg
If one node fails...
No problem.
Two copies remain.
If two nodes fail...
One copy still survives.
The recovery process is straightforward.
Copy the remaining version somewhere else.
Simple systems are often reliable systems.
That's one reason replication is still widely used.
The downside nobody likes paying for
Replication has one obvious drawback.
Storage efficiency.
Imagine a 100 GB object.
With three replicas, the cluster stores:
300 GB
That extra 200 GB buys durability.
But it also consumes:
- disks
- power
- rack space
- cooling
- network bandwidth
At small scale, this isn't a big deal.
At cloud scale, it becomes one of the largest operational costs.
A different idea
Instead of making complete copies...
What if we split the object into pieces?
Imagine a document divided into six fragments.
Now create three additional fragments containing recovery information.
The result looks something like this:
D1
D2
D3
D4
D5
D6
P1
P2
P3
No single fragment contains the entire object.
But together, they do.
More importantly...
If one or two fragments disappear, the missing data can be reconstructed.
Without storing three complete copies.
This is erasure coding
Erasure coding sounds complicated because it involves mathematics.
The underlying idea is surprisingly intuitive.
Instead of storing duplicate objects...
Store enough information to rebuild missing pieces.
It's similar to completing a puzzle.
Lose one piece?
You can still reconstruct the picture because enough information remains.
Why cloud providers love it
Let's compare.
Three-way replication:
100 TB data
300 TB storage required
An erasure coding scheme like:
8+4
stores:
100 TB data
150 TB storage required
The exact numbers vary depending on the coding scheme, but the pattern stays the same.
Much less storage.
Similar durability.
When you're operating thousands of storage nodes, that difference translates into enormous savings.
So why not use erasure coding everywhere?
Because nothing comes for free.
Replication is easy.
Need another copy?
Copy the object.
Erasure coding is different.
Lose one fragment...
The system has to reconstruct the missing data using the remaining fragments.
That requires:
- CPU
- network traffic
- coordination
Recovery becomes computational rather than simply copying bytes.
Another familiar trade-off
If you've followed this series, this pattern should look familiar.
Every architectural improvement solves one problem while introducing another.
Metadata simplified object discovery.
Immutable objects simplified updates.
Strong consistency simplified application logic.
Erasure coding reduces storage costs.
But it increases computational complexity.
Distributed systems rarely eliminate complexity.
They relocate it.
Why metadata matters yet again
Notice something interesting?
The actual fragments aren't the difficult part.
The difficult part is keeping track of them.
The system needs to know:
- Which fragments exist?
- Which nodes store them?
- Which fragment is missing?
- Which fragments are needed for reconstruction?
Those answers don't live with the object data.
They live in metadata.
Once again, metadata quietly becomes one of the most important parts of the storage system.
The hidden cost of saving space
Erasure coding dramatically improves storage efficiency.
But it also means:
- repairs take longer
- rebuilding requires computation
- network traffic increases during recovery
The storage bill goes down.
The engineering complexity goes up.
Cloud providers happily make that trade because hardware is expensive, while software can usually absorb additional complexity.
Key takeaway
Replication asks:
How many complete copies should we keep?
Erasure coding asks:
How little data can we store while still surviving failures?
Both protect your data.
They simply optimize for different things.
At cloud scale, that difference is worth billions of dollars.
Next in this series
Part 8: Why Rebalancing a Storage Cluster Is Harder Than It Looks
We'll explore what really happens after you add a new storage node—and why simply "moving some files" can create unexpected challenges.
Top comments (0)