S3 promises eleven nines of durability. Written out that is 99.999999999 percent. The way Amazon frames it, if you store ten million objects, you might expect to lose one of them once every ten thousand years. That number is easy to quote and hard to actually feel, so let me unpack what has to be true for it to hold.
First, durability and availability are not the same thing, and conflating them is the most common mistake here. Availability is whether you can reach your data right now. Durability is whether your data still exists at all. A system can be briefly unavailable, a request times out, you retry, and nothing is lost. Losing data is forever. S3's famous number is about durability, and durability is the harder promise because there is no retry that brings back bytes that are truly gone.
So how do you not lose bytes? The whole game is redundancy plus independence of failures. If you keep one copy of an object on one disk, the durability of your object is exactly the durability of that disk, which is not great. Disks die constantly at scale. The fix is to store the data in multiple places such that the failures are uncorrelated. S3 spreads every object across multiple devices in multiple facilities within a region. The key word is uncorrelated. Three copies on three disks in the same rack sharing the same power supply is not three independent failures, it is one. Real durability comes from spreading across separate storage nodes, separate racks, and separate data centers so that no single fire, power loss, or network partition can take out all the copies at once.
The next decision is how to add redundancy without paying for it three times over. The naive approach is replication: keep three full copies. Simple, but you pay three hundred percent storage overhead. The smarter approach for large scale systems is erasure coding. You split an object into data chunks, compute some parity chunks, and spread all of them across different nodes. With a scheme that produces, say, several data shards and several parity shards, you can lose several shards and still rebuild the original perfectly, while paying far less overhead than full triple replication. Erasure coding is how you get replication-grade durability at a fraction of the storage cost. The trade-off is more CPU to encode and decode, and more complexity when reconstructing, which is a fine trade at this scale.
But storing the data safely is only half of it. The other half is knowing when a copy has gone bad. Disks do not just die cleanly. They rot. Bits flip silently. So S3 continuously scrubs stored data in the background, reading it back, checking it against checksums, and repairing or re-replicating anything that has degraded before it ever becomes a problem. This is the part people forget. Durability is not something you set up once, it is an ongoing process of detecting and healing corruption faster than it accumulates. The system is always quietly checking its own work.
Now the architecture around all this. Conceptually S3 has a few pieces. There is a front end that handles the API requests. There is a metadata or index layer that maps your bucket and key to where the underlying data chunks actually live, which is essentially a giant, heavily sharded key to location lookup. And there is the storage layer, the fleet of nodes that hold the actual bytes. Object storage is deliberately flat. There are no real directories. What looks like a folder path is just part of the key string. That flatness is what lets it scale to trillions of objects, because you are not maintaining a tree, you are maintaining a massive distributed hash-like index.
A couple of honest trade-offs. S3 is not a filesystem and you should not treat it like one. There is no cheap rename, because a rename is a copy plus a delete. Listing is a prefix scan, not a directory read. And historically the consistency model was relaxed, though S3 now offers strong read after write consistency, which itself took real engineering to deliver at this scale. You get near infinite durable storage, but you give up the low latency random access and in-place mutation you would get from a block device.
How does the real S3 do it? Broadly this shape. Data spread across multiple facilities, erasure coding to get high durability without tripling storage cost, continuous background integrity checking to catch silent corruption, and a heavily partitioned metadata layer sitting in front of a flat storage fleet. The eleven nines is not one feature. It is the compounding effect of independent redundancy and relentless self-healing.
The lesson I take from S3 is that durability is a discipline, not a setting. You earn those nines by assuming everything will fail and designing so that no single failure, and no slow accumulation of small failures, can reach your data.
I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-s3-object-storage
Top comments (0)