When people picture Netflix, they picture the cloud. A giant data center somewhere, full of servers, pumping video out to phones and televisions all over the world. That mental model is wrong in a way that turns out to be the whole story. The interesting engineering at Netflix is not about running a big cloud. It is about the opposite. It is about how little video actually comes out of the cloud, and how much of it is served from a box sitting a few miles from your house, inside the same network your internet provider runs. Netflix figured out early that you cannot stream this much video out of a central location, no matter how large that location is. So they stopped trying, and built something else.
This piece works the problem the way a staff engineer would work it in a design review. We start from the requirements and the arithmetic, because at this scale the numbers rule out entire categories of design before you draw a single box. Then we build up the pieces: the split between the control plane and the data plane, what actually happens in the second after you press play, how the Open Connect content delivery network puts video inside your internet provider, why Netflix fills those caches ahead of demand instead of on demand, how the catalog gets encoded, how the client adapts to your fluctuating connection, and how the system fails without falling over. One note on numbers before we start. Every figure for subscribers, bitrate, throughput, and storage here is an industry-typical estimate chosen to drive the sizing, not a measured production metric from any specific quarter.
The problem, stated as requirements
The feature that a viewer sees is almost embarrassingly simple. Browse a catalog, press play, and watch a video that starts quickly and never stops to buffer, at the best quality your screen and your connection can support, resuming exactly where you left off on any device. That is the entire product surface for playback. The engineering does not live there.
The feature list is short. The non-functional constraints, startup time, no rebuffering, planetary concurrency, and the cost of raw bandwidth, are where the design actually happens.
It lives in the non-functional constraints, and those are worth writing down before anything else. Startup latency, the time from pressing play to the first frame, has to feel instant, well under a second, because a slow start reads as a broken app. Rebuffering, the stall in the middle of a show, is the single worst experience the product can deliver, and the system is measured on how rarely it happens. The service has to hold quality steady across connections that range from a fiber television to a phone on a train, and it has to do all of this for tens of millions of people watching at the same time in the same evening. Underneath every one of those constraints sits a cost problem that most systems never face at this magnitude. Video is enormous, and moving it is the dominant expense. A design that satisfies the viewer but ignores the per-gigabyte cost of delivery is not a viable design. When you state the constraints this way, the rest of the architecture reads as a sequence of answers to them.
The numbers that force the architecture
Do the arithmetic first, because it eliminates the obvious design immediately. Take north of 250 million paid memberships watching well over a billion hours a week. The number that matters is not total members, it is peak concurrency, the count of streams running at the same moment during prime evening hours. Assume, conservatively, tens of millions of simultaneous streams, and use 50 million as a round figure for the sizing.
Fifty million concurrent streams at roughly five megabits each is a quarter of a petabit per second, or 112 petabytes in one peak hour. No central cloud serves that, at any price, which is the entire reason Open Connect exists.
Each stream carries video at an average sustained bitrate of around five megabits per second, blending standard-definition phones near one to three megabits, high-definition laptops near five, and 4K televisions in the mid teens. Multiply. Fifty million streams times five megabits per second is 250 million megabits per second, which is 250 terabits per second of sustained delivery, and the true peaks with heavier 4K adoption run higher. Now turn that rate into volume and money, because that is where it becomes obvious. 250 terabits per second is about 31 terabytes every second, which is roughly 112 petabytes moved in a single peak hour. Priced at a nominal two cents per gigabyte of cloud egress, that one hour would cost on the order of two million dollars, every evening, before a single server bill. And the capacity side is worse than the cost side, because no single cloud region has an internet-facing pipe within an order of magnitude of a quarter petabit per second, and stringing enough regions together to fake one just moves the bottleneck onto the backbone links between them. That one calculation kills the obvious architecture. You cannot serve this from AWS, or from any centralized set of data centers, no matter how many. The storage math points the same way. A single hour of content, encoded across every resolution, bitrate, codec, and audio and subtitle track, produces on the order of hundreds of gigabytes to more than a terabyte of output, so the full encoded catalog runs to petabytes. The conclusion is forced, not chosen. The bytes have to live close to viewers, in thousands of places at once, and the only real design question is how they get there.
Two planes: control and data
The decision that organizes everything else is the separation of the control plane from the data plane. These are two different systems, running on two different sets of infrastructure, optimized for two completely different jobs, and keeping them apart is what makes the scale tractable.
The control plane is small, smart, and central on AWS. The data plane is enormous, simple, and everywhere. Almost none of the bytes and almost all of the logic live in different places.
The control plane runs in AWS and handles everything except the video bytes. Signup and billing, the recommendation and personalization systems, search, the device home screens, playback authorization, digital rights management license issuance, A/B testing, and the orchestration of the encoding pipeline all live here, spread across many hundreds of microservices. It is small in bytes and large in logic. The requests are frequent but tiny, a few kilobytes of JSON, and they demand rich, fast, consistent behavior. The data plane is the exact inverse. It is Open Connect, Netflix's own content delivery network, and its only job is to move video bytes from a cache to a screen. It is enormous in bytes and almost trivial in logic. An Open Connect appliance does not run personalization or authorization. It answers one kind of request, "send me these segments of this file," over HTTP, as fast as its network card allows. The insight is that these two workloads have nothing in common except that they cooperate for a few hundred milliseconds when you press play. Trying to serve both from one system would compromise both. Splitting them lets the control plane be clever and centralized while the data plane is dumb, cheap, and distributed to the edge.
The architecture end to end
With the two planes named, the full picture assembles cleanly. A Netflix client, whether a smart television, a phone app, or a browser, talks to both planes but for different things, and it is the client that stitches them together.
The client talks to the AWS control plane for everything except bytes, and pulls the actual video directly from an Open Connect appliance inside its own internet provider.
For browsing, search, and account actions, the client calls an API gateway in AWS that fans out to the microservices behind it, and those services read from their own data stores and caches. When the user presses play, the client makes a playback request to the control plane, which does the authorization and licensing work and, critically, decides which Open Connect appliances the client should stream from. That decision comes back as a ranked list of appliance addresses. From that point on, the control plane is out of the loop for bytes. The client opens connections directly to the chosen appliance and pulls the video over HTTP, segment by segment. The appliance itself was filled with that content ahead of time by the fill system, which we come to shortly, sourcing from encoded masters stored in AWS. So there are really three flows: a control flow of small, frequent metadata calls to AWS, a one-time-per-session steering handshake that hands the client a set of edge targets, and the heavy byte flow that never touches AWS at all and runs directly between the client and an appliance a short network distance away. Each flow scales on its own axis, which is exactly why the whole thing scales.
What happens when you press play
The second after you press play is where the control plane earns its keep, and it is worth walking step by step, because the sequence explains most of the system's design choices.
Pressing play is an authorization and steering handshake with AWS, followed by a direct pull from the closest healthy appliance. The bytes never route through the cloud.
First, the client, already signed in, sends a play request to the playback service in AWS. Second, that service authorizes the session: it checks that the membership is active, that the title is licensed in this country, and that the account has not exceeded its allowed number of concurrent streams. Third, it issues a digital rights management license, negotiating the specific scheme the device supports, so the video can be decrypted only on this authorized device. Fourth, and this is the step that makes Open Connect work, the steering service selects the best appliances for this specific client. It knows the client's internet provider and rough network location from the connection, it knows which appliances hold this exact title, and it knows the current health and load of each candidate, so it returns a ranked list of the appliances that are closest and least loaded. Fifth, the client picks the top appliance, requests the streaming manifest that lists the available quality levels, and begins pulling video segments directly from it over HTTP. If that appliance degrades mid-stream, the client walks down the ranked list to the next one. The division of labor is precise. AWS decides who you are, what you are allowed to watch, and where you should get it. The appliance simply serves the bytes, having been told nothing except which file and which byte ranges.
Open Connect: putting the bytes inside the ISP
Open Connect is the part that most people never see and that makes the arithmetic work. An Open Connect appliance is purpose-built hardware, a dense storage server running FreeBSD and NGINX, tuned to do one thing, saturate its network links serving video files. A single storage appliance holds on the order of 200 terabytes, enough for a large slice of the popular catalog, and flash-based appliances hold a smaller, hotter set for the titles everyone is watching right now.
Netflix does not stream video from the cloud. It streams from inside your internet provider's network, and everything else in the design exists to make that possible.
Appliances live either embedded inside an internet provider's network, where traffic never crosses the public internet, or at internet exchanges that serve many smaller providers at once.
There are two ways these appliances get close to you. The first is embedded deployment. Netflix ships appliances, at no cost, to internet service providers who install them inside their own networks, in the same facilities that serve their subscribers. When you stream, the bytes travel from that appliance to your home entirely within your provider's network, never crossing the congested public internet or a paid transit link. The provider saves enormous transit costs, Netflix gets the shortest possible path to the viewer, and both sides win. The second is deployment at internet exchange points, the neutral facilities where many networks interconnect, which covers smaller providers who cannot host an embedded appliance. Either way, the goal is the same: reduce the distance the video travels to nearly zero, and take it off the expensive shared backbone. Because these appliances sit at the very edge, one appliance failing is a local event, not a global one. The blast radius of any single box is the handful of viewers it was serving, who get steered to another appliance in the time it takes to open a new connection.
Proactive fill, not pull-through caching
Here is where Netflix diverges sharply from a conventional content delivery network, and the difference is the sharpest design decision in the system. A normal CDN is a pull-through cache. When a user requests something the edge does not have, that is a cache miss, and the edge fetches it from the origin, stores it, and serves subsequent requests locally. It reacts to demand.
A conventional CDN reacts to demand and pays for it with cache-miss storms at peak. Netflix predicts demand and fills the edge in the overnight trough, so peak hour asks the origin for almost nothing.
Netflix does the opposite. It fills the appliances proactively, before anyone requests the content, and it can do this because of a property most streaming workloads do not have: the catalog is finite and its popularity is predictable. Netflix knows tomorrow's popular titles today, because there is no user-generated flood and no live unpredictability in the on-demand catalog, and it knows, per region, roughly how much of each title will be watched. So every night, during the off-peak window when the provider's network is idle, the fill system computes which titles should sit on which appliances and pushes them out. By the time the evening peak arrives, the content is already there. A miss at peak is designed out rather than absorbed: when a title is not on the nearest appliance, the steering service simply points the client at another appliance that already holds it, so the request stays inside the edge fleet instead of turning into a live fetch from the origin. The fill traffic itself rode the cheap overnight capacity instead of competing with viewers. This inverts the usual CDN failure mode. A pull-through cache suffers most exactly when it is busiest, because a surge of new demand is a surge of misses hammering the origin. Netflix's proactive model spends its bandwidth when bandwidth is cheap and idle, and arrives at peak hour with nothing left to fetch. Prediction replaces reaction.
The fill hierarchy
Filling tens of thousands of appliances does not happen in one hop from a central origin, because that would recreate the very bottleneck the design avoids. The fill system is tiered, so that most fill traffic, like most viewing traffic, stays local.
Encoded masters live in AWS, but edge appliances fill from larger regional appliances nearby, so the central origin serves each title a handful of times, not millions.
The encoded masters, the finished output of the encoding pipeline, live in object storage in AWS. That is the single source of truth. But an embedded appliance deep inside a small provider does not pull from AWS directly. It pulls from a larger regional appliance closer to it, which in turn pulled from a fill source that pulled from AWS. Content flows down a hierarchy: from cloud storage to a small number of large regional caches, then out to the many edge appliances, each level filling from the level above and from peers at the same level where that is cheaper. The effect is that a given title is copied out of AWS only a handful of times, into the top of the hierarchy, and then replicated laterally and downward across the fleet using cheaper regional and peering links. The central origin never sees anything close to the real viewing load, because it only ever serves the fill tree, not viewers. This is the same principle as the byte path itself, applied recursively: keep every transfer as short and as local as possible, and never make the expensive central resource do work that a nearby cheaper resource can do instead.
Encoding the catalog
Before any of this can ship, the video has to be encoded, and Netflix does not store one version of a title. It stores dozens, because a phone on cellular and a 4K television have nothing in common in what they can accept.
A source master is chopped into chunks, encoded in parallel across a huge fleet of cloud workers into every quality and codec, then reassembled, validated, and published.
The pipeline starts with a high-quality source master delivered by the studio. Encoding a two-hour film into every needed variant serially would take an impractically long time, so the pipeline chops each title into short chunks and encodes the chunks in parallel across a large fleet of cloud compute, then stitches the results back together. Every chunk is encoded into many combinations of resolution, bitrate, and codec, producing the full set of streams a client might request, along with audio tracks in many languages and subtitle and caption files. Each output is validated automatically for quality and for artifacts before it is accepted, because a corrupt encode discovered by a viewer is a failure the pipeline is supposed to catch. The finished set of encoded files is packaged into the streaming format, encrypted for digital rights management, written to object storage in AWS as the master copy, and handed to the fill system for distribution to the edge. Encoding is expensive and slow, but it is done once per title, entirely offline, long before anyone presses play. That is the deeper pattern: the most costly work in the system happens ahead of time, on a predictable schedule, so that the live path at viewing time is nothing but reading a file that already exists.
The bitrate ladder, optimized per title
The set of quality levels a client can choose from is called the bitrate ladder, a set of rungs each pairing a resolution with a bitrate. The naive approach uses one fixed ladder for every title in the catalog, and it wastes a great deal of bandwidth.
A fixed ladder over-spends bits on simple content and starves complex content. A per-title ladder is tuned to how hard each specific title is to compress.
The problem with a single ladder is that content varies enormously in how hard it is to compress. A talking-heads drama or an animated cartoon with flat colors compresses to excellent quality at a low bitrate, while a fast, grainy action sequence needs far more bits to look the same. A fixed ladder set high enough for the action film wastes bandwidth on the cartoon, and a fixed ladder set low enough to be efficient on the cartoon looks bad on the action film. Netflix's answer is per-title encoding: analyze each title's actual complexity and build a ladder tuned to it, so simple content gets a lower-bitrate ladder that looks just as good and heavy content gets the bits it genuinely needs. The refinement goes further, down to optimizing shot by shot within a single title, since a film is not uniformly complex from scene to scene. The payoff is direct and large. Better quality at the same bitrate means fewer rebuffers and lower delivery cost, and at a quarter of a petabit per second of egress, even a few percent of bitrate saved is a vast amount of bandwidth and money. This is a case where investing heavily in offline computation, analyzing every title and every shot, buys a permanent reduction in the recurring cost of the live system.
Codecs: compatibility versus compression
Sitting underneath the ladder is the choice of codec, the compression format the video is encoded in, and Netflix supports several at once because it is caught between two opposing pressures.
Older codecs play everywhere but compress poorly. Newer codecs compress far better but only decode on recent hardware. Netflix ships all of them and lets the client pick.
On one side is compatibility. H.264, also called AVC, is the oldest of the widely used video codecs and decodes on essentially every device ever built, which makes it the universal fallback. On the other side is compression efficiency. Newer codecs like VP9, HEVC, and especially AV1 achieve the same visual quality at substantially lower bitrates, sometimes twenty to thirty percent less or better, which directly cuts delivery cost and rebuffering on constrained connections. The catch is device support. Efficient decoding at high resolutions needs dedicated silicon that only recent hardware has, and while a capable device can decode AV1 in software, that costs battery and CPU and does not stretch to a large 4K television stream, so a device that cannot decode a codec well must be served an older one instead. Netflix resolves the tension by not choosing. It encodes popular content in multiple codecs and lets the client advertise what it can decode, then serves the most efficient codec that device supports. A modern television gets AV1 and streams a 4K title at a fraction of the bits an older codec would need, while a decade-old device gets H.264 and still works. The cost is storage and encoding effort, since the same title is encoded several times over, but storage is cheap and one-time while bandwidth is expensive and continuous, so the trade strongly favors encoding more variants to send fewer bytes.
Adaptive bitrate on the client
Choosing which rung of the ladder to stream at, moment to moment, is not the server's job. It is the client's, and pushing that decision to the client is what lets the server stay dumb and scalable.
The client, not the server, picks the quality. It watches its own buffer and measured throughput and steps the bitrate up or down to keep the buffer full and the picture as high as the network allows.
This is adaptive bitrate streaming. The video is served as a sequence of short segments, each a few seconds long, and each segment is available at every rung of the ladder. The client downloads segments into a buffer that plays out to the screen, and before each segment it decides which quality to request next. The decision balances two signals. It estimates the available throughput from how quickly recent segments arrived, and it watches how full its own buffer is. If the buffer is healthy and throughput is strong, it steps up to a higher-quality rung. If the buffer is draining or throughput drops, it steps down to a lower rung to keep video flowing, because a slightly softer picture is vastly better than a stall. Netflix's own research pushed the field toward buffer-based control, using the buffer level itself as the primary signal rather than trusting throughput estimates alone, which are noisy and easily fooled by transient network behavior. The consequence for the architecture is profound. Because the client drives quality selection, the appliance does not need to understand networks, adapt, or make decisions. It just serves whichever segment was requested. All the intelligence about the messy, variable last mile lives on the one machine that can actually observe it, the client, and the fleet of servers stays a simple, stateless byte pump.
Steering and failover
At this scale, something is always failing. An appliance is being rebooted, a link is congested, a title was not filled to a particular box. The system is built to route around all of it continuously, without the viewer noticing, and this is where the earlier design decisions pay off.
The client holds a ranked list of appliances and can degrade quality rather than stall. Failure means switching sources or dropping a rung, never a broken stream.
Recall that the steering service handed the client a ranked list of appliances, not a single address. That list is the failover mechanism. The client streams from the top appliance, and if it observes that appliance slowing down or refusing connections, it simply moves to the next one on the list and continues, often mid-title, without interrupting playback. Because appliances are interchangeable byte servers and the client is the one holding state about where it is in the video, switching sources is cheap and invisible. Layered on top of that is the adaptive bitrate logic, which is itself a failure response: when the network between the client and even the best appliance degrades, the client drops to a lower rung rather than stalling. The two mechanisms compose into graceful degradation. The worst normal outcome is not a broken stream, it is a temporary drop to lower quality or a silent switch to a different source. The control plane in AWS is engineered for its own failures separately, running across multiple regions so that a regional outage does not take down authorization and steering, and Netflix famously validates all of this by deliberately injecting failures in production, killing instances and degrading services on purpose to prove the system routes around them. The design assumes components will fail and makes failure boring.
Where the data lives
Different data in this system has radically different shapes and access patterns, and Netflix uses different stores for each rather than forcing everything into one database. The video bytes are the obvious case, but the metadata is just as deliberate.
Each kind of data goes to the store that fits its shape: object storage for bytes, a wide-column store for the write-heavy viewing history, an in-memory cache for hot state, and a relational store for money.
The encoded video files are objects, large and immutable, so they live in object storage in AWS as the master copy and are replicated onto the appliances. There is no database involved in serving a byte; the appliance reads a file. Viewing history and playback state, the record of what every member watched and where they paused, is an enormous, relentlessly write-heavy, append-oriented dataset with no need for complex joins, which is the textbook case for a wide-column store, and Netflix runs this on Cassandra for exactly that reason: it takes writes at massive volume and scales horizontally across regions. The hot state that has to be read on nearly every interaction, the data behind the home screen, session state, and precomputed recommendation results, is served from a distributed in-memory cache so that the common path never waits on a disk. Billing and payments, where correctness and transactional guarantees matter more than raw scale, sit in a relational database, because money is the one place you want strict consistency and are willing to trade some scalability for it. The principle is the one that runs through the whole system. Let the access pattern choose the store. Immutable bytes go to object storage, high-volume writes go to a wide-column store, hot reads go to a cache, and transactional money goes to a relational database, and no single engine is asked to be good at all four.
The trade-offs, and what transfers
Every decision in this design bought something and cost something, and being able to name both sides is what separates understanding the system from reciting it. Splitting the control and data planes bought independent scaling but cost the complexity of coordinating two systems on every play. Proactive fill bought a peak with no cache misses but cost the storage to hold content that might not be watched and the machinery to predict popularity. Per-title and per-codec encoding bought lower delivery cost and better quality but cost enormous offline compute and multiplied storage. Client-side adaptation bought a dumb, scalable server fleet but cost the difficulty of getting the client algorithm right across billions of devices. In every case the design traded one-time and offline cost for recurring and online savings, which is the correct trade for a system whose recurring cost, moving petabytes of video, dwarfs everything else.
Four decisions from this design transfer to any large system: split control from data, precompute the predictable, push adaptation to the client, and degrade instead of failing.
The reason Netflix is worth this much attention is that its lessons are not about video. Four of them transfer directly to systems that have nothing to do with streaming. First, separate the control plane from the data plane when they have different shapes, so the smart, central part and the dumb, distributed part can each scale on their own terms. Second, precompute whatever is predictable, whether that is an encoding ladder or a cache fill, and move the expensive work off the live path and onto an offline schedule, because the cheapest work at request time is work you already finished. Third, push adaptation to the edge where the real conditions can be observed, so the core stays simple and stateless while the client absorbs the variability of the last mile. Fourth, design for graceful degradation rather than binary success, giving clients ranked fallbacks and quality levels to drop through, so that failure is a smaller picture, not a black screen. Learn to see which of these levers your own bottleneck sits on, and Netflix stops being a story about a video company and becomes a method you can apply to the next system you build.
I teach system design this way, from first principles with real diagrams and the trade-offs that only surface in production, as an interactive course at systemdesign.academy. The foundation lessons are free and need no signup.
Read the free lessons: https://systemdesign.academy
Top comments (0)