WhatsApp is the system design interview question that people think they understand and almost never do. The instinct is to treat it like a database problem. You imagine a table of messages, a fan-out on write, a feed that gets read later, and you reach for the same patterns you would use for a social timeline. That framing is wrong from the first line, and the interesting part of the system is exactly the part that framing hides. WhatsApp is not a store that people read from. It is a relay that forwards live traffic between two billion devices, holds each of them on a persistent connection, and is contractually forbidden from reading a single message it carries. Once you see it that way, the design stops looking like a database and starts looking like a telephone exchange with cryptography bolted to it.
This piece works the problem the way I would want a staff engineer to work it in a design review. We begin with requirements and the arithmetic, because the numbers here point somewhere surprising: the load that shapes the architecture is not the message rate, it is the count of connections you have to hold open while nothing is happening. From there we build up the pieces. How a single machine holds millions of live sockets, why the connection tier runs on Erlang, how store-and-forward covers the offline case, how delivery is made reliable over flaky mobile networks, why the server keeps almost no message history, what state it actually stores, how end-to-end encryption turns the server into a blind relay, how group messaging avoids an O(N) cost per message, how multi-device changes the fan-out, how media stays off the hot path, and how the whole thing behaves when a machine dies. One note before we start. Every latency, throughput, and capacity figure here is an industry-typical estimate used to drive the sizing, not a measured production metric from any specific service.
Start with the requirements, not the features
The user-facing product fits in three verbs. Send a message to one person, send a message to a group, and show the sender what happened to it. The receipts, the ticks that go from one to two to blue, are part of that third verb. Media is the first verb carrying a photo instead of text. That is the whole surface, and it is deceptively small.
The product is three verbs. The non-functional column, latency, an always-live connection, and end-to-end encryption, is where the design is actually decided.
The design does not live in the feature list. It lives in the non-functional constraints, and they are unusually demanding for something so simple to describe. Delivery has to feel instant, which for a chat app means a few hundred milliseconds end to end across the planet, not the single-digit milliseconds a local service targets, because the bytes have to physically cross oceans. The connection has to stay alive through the messy reality of mobile networks, where a phone hops between cell towers, loses signal in an elevator, and switches from wifi to cellular mid-sentence. And end-to-end encryption is not a feature you can add later, it is a constraint that reaches back and reshapes everything else, because it means the server is not allowed to understand the payload it routes. When you write those three constraints down before drawing any boxes, the rest of the design reads as a sequence of answers to them. The reason to insist on this order is that a candidate who starts from the feature list builds a message database, and a candidate who starts from the constraints builds a relay. Only one of those scales to two billion people.
The number that actually matters
Do the arithmetic before the architecture, because the binding constraint is not where intuition puts it. Take the widely cited figure of roughly a hundred billion messages a day. Divide by the number of seconds in a day and you get an average of about 1.16 million messages per second. That sounds enormous, and it is a lot, but it is a throughput number, and throughput you can shard. Spread 1.16 million messages a second across a fleet and each machine handles a manageable slice. Real traffic is spiky, so plan for several times that at peak, the classic example being the wall of New Year greetings that lands in a few time zones at once, but even a five times spike is a throughput problem with a throughput solution.
The message rate is large but shardable. The number that shapes the design is the count of always-open connections you hold while nothing is being sent.
The number that actually shapes the design is different. It is the count of devices you are holding a live connection to at rest, whether or not they are sending anything. With two billion-plus users, most of them idle most of the time, you are keeping on the order of a billion TCP connections open continuously. Not every registered device holds a socket at every instant, which is exactly why the push wake-up path described later exists, but the sizing case is near-full concurrency and that is what you build for. That is not a throughput problem, it is a memory and file-descriptor problem, and it does not go away when traffic is quiet. A server that can hold a million idle connections needs a fleet of a couple thousand machines to blanket the planet. A server that can only hold ten thousand needs two hundred thousand machines and a very different budget. So the first real engineering question for WhatsApp is not how fast can you move messages, it is how cheaply can you hold a connection that is doing nothing. Everything downstream depends on the answer.
The architecture, organized around the socket
Before answering that question, it helps to see the whole system, because every component in it exists to serve the connection. A client connects through an edge layer that terminates the connection and pins it to a specific connection server, the machine that will hold that socket for the life of the session. Behind the connection servers sit the back ends that answer the routing question and handle the cases the hot path cannot.
The edge terminates a socket, a connection server owns it, and three back ends handle routing, offline delivery, and wake-ups. Media never touches this path.
A session registry maps every connected user and device to the connection server currently holding its socket. When a message arrives for a recipient, the sending server consults this registry to find which server owns the recipient's connection, then hands the message across. This registry is the hottest piece of state in the system, read on essentially every message and rewritten every time a device connects, disconnects, or moves, which is why it lives in fast, distributed, in-memory storage rather than on disk. When the recipient is not connected, the message goes to an offline store to wait, and a push gateway sends a notification through Apple's or Google's push services to wake the app so it reconnects and drains what is waiting. Media takes a separate path entirely, going to a blob store while the message carries only a pointer. The organizing principle is the one that shows up in every clean design at scale. Keep the fast path short, and push the slow or occasional work, offline buffering, wake-ups, and large file transfers, off to the side where it cannot slow the common case.
The connection is the unit of scale
Return to the question the arithmetic raised: how cheaply can you hold a connection that is doing nothing. Because idle connections dominate, the entire connection tier is engineered around making an idle connection nearly free. The device opens one persistent, multiplexed TCP connection to the server and keeps it open for as long as it can. There is no polling loop asking are there new messages, which would waste battery and generate constant useless traffic across billions of clients. The connection simply sits there, and when a message needs to go either direction, it travels over the socket that is already open.
A device holds one socket, backed by one lightweight process holding kilobytes of idle state, which is why a single tuned box holds on the order of two million of them.
The famous milestone from WhatsApp's engineering history is holding more than two million connections on a single server. That number was not a stunt, it was the whole business model expressed as a systems achievement, because it is the difference between a fleet you can afford and one you cannot. Reaching it meant treating an idle connection as a few kilobytes of state and one very cheap unit of concurrency, then tuning the operating system, the network stack, and the runtime until millions of those could coexist on one box. A lightweight keepalive heartbeat runs over each connection so the server can tell the difference between a client that is quietly connected and one whose network died without a clean close, and reclaim the slot for the dead one. Hold that idle cost down and the arithmetic collapses in your favor: two billion devices divided by two million per box is a fleet in the low thousands. Let the idle cost creep up and no amount of clever routing saves you.
Why the connection tier runs on Erlang
That two-million-per-box result was not luck, it came from the runtime WhatsApp chose. The connection servers run on Erlang, for years on a customized FreeBSD, starting from a heavily modified version of the ejabberd XMPP server before evolving into their own protocol. That choice reads like a historical curiosity until you look at what the workload demands, at which point it looks less like a preference and more like the only sane option.
Millions of isolated processes, supervised so failures self-heal, and upgradeable live: the connection workload wants exactly what the Erlang runtime was built to provide.
The workload is millions of long-lived, independent, failure-prone connections on one machine. Erlang processes are not operating system threads, they are far lighter, scheduled by the runtime, and cheap enough that running one per connection is affordable at millions of connections. That one-process-per-connection model buys isolation for free. A bug or a malformed packet that crashes the process handling one connection takes down that one connection and nothing else, because processes share no memory and cannot corrupt each other. The runtime's supervision trees turn that isolation into self-healing: a supervisor watches its child processes and restarts any that die, back to a known-good state, so transient faults resolve themselves without an operator in the loop. This is the let-it-crash philosophy, and at this scale it is not sloppy, it is the cheapest possible fault handling. Hot code loading closes the loop by letting you load new code into a running node without dropping the open connections, so you can ship changes to a fleet holding billions of sockets without a maintenance window that would disconnect everyone at once. Pick the runtime for the workload, and a tier whose job is holding millions of fragile connections wants cheap isolated processes, supervision, and live upgrades. That is a precise description of Erlang.
Deliver if you can, park if you cannot
The core delivery logic is store-and-forward, and its elegance is that it stores as little as possible. When a message arrives at the server, the router asks the session registry a single question: is the recipient connected right now.
The router asks one question. If the recipient is online, relay the message immediately; if not, park it and send a push to wake the app.
If the recipient is online, the message is handed straight to the connection server holding their socket and pushed down to the device, with no durable write in the common path. If the recipient is offline, the message parks in the offline store, keyed by the recipient, and the push gateway fires a notification to nudge their app into reconnecting. When the device comes back, it reconnects, the server drains the queued messages down the fresh socket, and the device acknowledges each one. Here is the part that surprises people the first time they see it. Once a message has been delivered and acknowledged, the server deletes it. It does not keep a copy. Undelivered messages are held only for a bounded window, on the order of thirty days, after which they are dropped. The server is not a mailbox that accumulates history, it is a temporary holding area for the small subset of messages that are currently in flight to someone who is not looking. That single decision, store only what has not yet been delivered, is what keeps the storage footprint of a hundred-billion-message-a-day system astonishingly small.
WhatsApp's server is a relay that is never allowed to read the messages it delivers, and almost every design decision follows from that one constraint.
Making delivery reliable over a flaky network
A message crossing a mobile network can be dropped at any point. The socket can die mid-transfer, the app can be killed by the operating system, the radio can cut out. The delivery guarantee has to hold anyway, and the mechanism that makes it hold is worth spelling out, because it is the same mechanism that powers the ticks users watch.
Sent, delivered, and read are three separate acknowledgements flowing back over the connection, made safe by unique message ids and retry.
Every message carries a unique id generated by the sending client. Delivery is at-least-once: the server holds the message and keeps trying until it receives an acknowledgement, so a connection that dies mid-delivery loses nothing, the message is simply redelivered when the device returns. At-least-once means duplicates are possible, a message can be delivered, the ack can be lost, and the message redelivered, so the recipient deduplicates by id and silently drops any message it has already seen. That combination, retry until acked plus dedup by id, is what turns an unreliable network into reliable delivery. The ticks are this acknowledgement flow made visible. One tick means the server has accepted the message. Two ticks mean the recipient's device has acknowledged receipt. The two blue ticks are a separate read receipt, sent when the recipient actually opens the conversation. Ordering within a conversation is maintained as a per-chat sequence so messages display in the order they were sent even if they arrive out of order. None of this requires the server to understand the message, it only requires it to track ids and acks, which is exactly the amount of understanding a blind relay is allowed to have.
The server relays messages, it does not keep them
The transient storage decision deserves to be examined on its own, because it runs against the instinct of almost everyone who has built a messaging feature inside a normal product. The default assumption is that the server is the system of record, the durable place where all messages live and from which clients sync. WhatsApp deliberately does the opposite.
History lives on your devices. The server stores a message only until it is delivered, then forgets it, which removes an entire class of cost and liability.
Message history lives on your devices, in the local database on your phone, not on WhatsApp's servers. The server stores a message only for as long as it takes to deliver it, then deletes it. Compare that to the archive model, where the server retains every message forever, becomes queryable, and serves as the backing store for history and multi-device sync. The archive model is more capable, and it is what a naive design reaches for, but it is enormously expensive at this scale and it creates a standing liability: petabytes of message data that must be stored, encrypted at rest, and defended, and that becomes a target for attackers and a subject of legal demands. The relay model sidesteps all of it. There is far less to store, far less to secure, and far less to hand over, because there is simply no server-side history to hand. The clinching argument is that end-to-end encryption already prevents the server from reading messages, so retaining them would buy all of that cost and liability in exchange for capability the product cannot use anyway. When history lives on the client and the server cannot read the payload, a durable server archive is pure downside.
What the server actually stores
If the server keeps almost no messages, it is fair to ask what it does store, and the answer reveals the true shape of the system. The data model is dominated not by conversations but by routing and cryptographic material.
The interesting thing about the data model is what is missing. There is no table of everyone's messages, only routing state and keys.
The hottest state is the session route, the mapping from each connected user and device to the connection server holding its socket. It is read on every message and rewritten constantly as devices connect and disconnect, so it lives in fast distributed memory, historically Mnesia, the Erlang runtime's own distributed database, chosen precisely because it keeps this kind of state in memory close to the code that reads it. The offline queue is the only message store, and it is transient by construction, filled when recipients are away and drained and deleted as they return. Then there is the cryptographic material that makes end-to-end encryption work: each device's public identity key and a batch of one-time public prekeys, uploaded so that senders can establish encrypted sessions even when the recipient is offline. Finally there is durable account data, the phone number, profile, and contact relationships, which is genuinely long-lived. What is absent is the thing a chat app is assumed to have: a durable table of everyone's messages. That table does not exist here. Store the routing state hot and the message state briefly, and you have described a relay's data model, one dominated by where everyone is rather than by what everyone said.
The server is a key directory and a blind relay
End-to-end encryption is the constraint that reshapes everything, and WhatsApp implements it with the Signal Protocol. The property it guarantees is strong and simple to state: only the sender and the intended recipients can read a message, and the server, which routes every message, cannot. Achieving that while still allowing you to message someone whose phone is currently off is the clever part.
Encryption keys are agreed device to device. The server only holds public prekeys and forwards ciphertext it has no way to read.
Each device generates its own long-term identity key and publishes a batch of one-time prekeys to the server. These are public keys, useless to an attacker on their own. When you want to message someone, your device fetches their prekey bundle from the server and runs the X3DH key agreement, which derives a shared secret between the two devices without the recipient needing to be online, because their side of the exchange was pre-published. From that shared secret, the Double Ratchet algorithm derives a fresh key for every single message, advancing the key each time so that compromising one message key does not expose past or future messages. This is forward secrecy, and it is automatic. Through all of this the server plays exactly two roles. It is a directory that hands out public prekeys, and it is a relay that forwards an opaque ciphertext blob it cannot decrypt because it never holds the private keys. Keep the private keys on the endpoints and only the public prekeys on the server, and the consequence for system design is profound: compromising the server leaks who talks to whom and when, the routing metadata, but not the contents of a single conversation.
Groups would be O(N) per message without sender keys
Group messaging looks like it should be a small extension of one-to-one messaging, and in a world without encryption it would be. The server would receive one message and fan it out to every member. End-to-end encryption breaks that simplicity, because the server cannot read the message and therefore cannot be trusted to fan out anything meaningful, and every recipient device needs the message encrypted in a way only it can open.
Encrypting a group message once per member does not scale. A shared sender key moves that cost from every send to only membership changes.
The naive approach is to treat a group as a list of individuals and encrypt the message separately for each member, using the pairwise session you already have with each of them. That works and it is secure, but the cost is O(N) encryption operations for every message, all performed on the sender's device. WhatsApp group size has grown from two hundred and fifty-six members to over a thousand, so sending one message can mean a thousand encryptions, and in a chatty group that cost lands on the sender's phone and battery over and over. The Signal Protocol's answer, which WhatsApp uses, is Sender Keys. Each member generates a sender key and distributes it once to every other member, encrypted over the existing pairwise sessions. After that one-time distribution, sending a message is a single symmetric encryption with your sender key, and the server fans out the identical ciphertext to every member, each of whom already has the key to decrypt it. The expensive O(N) work now happens only when the group's membership changes, which is when keys must be redistributed, rather than on every message. Pay the encryption cost per membership change, not per message, and group messaging goes from O(N) per send to O(1) per send plus an O(N) cost amortized across everything sent between roster changes.
Every device is its own encrypted endpoint
Multi-device support, using WhatsApp on a laptop or a tablet while your phone sits in another room, looks like a product convenience and is actually a significant change to the cryptographic architecture. The naive implementation would make the companion devices mirrors of the phone, relaying everything through it, which is why for years the web client required the phone to be online. The modern design does something better and more demanding.
Multi-device is not a mirror of the phone. Each device is its own cryptographic endpoint, so the real fan-out is recipients times devices.
Each device is a first-class cryptographic endpoint with its own identity keys and its own sessions, registered independently in the system. Your phone, your laptop, and your tablet are three separate endpoints that happen to belong to the same account. This is why companion devices can send and receive while the phone is completely offline: they are not proxying through it, they are participating directly. The cost of this independence is that the fan-out multiplies. A message is no longer encrypted for each recipient, it is encrypted for each device of each recipient, so the true fan-out is the number of recipients times the number of devices each has. The Sender Keys mechanism extends to cover it, distributing the sending key to every device rather than every user, but the multiplier is real and the system has to carry it. Treat each device as a separate endpoint and you pay in fan-out, but you buy two things worth having: the phone stops being a single point of failure for the account, and each device works on its own schedule.
Media rides a separate, cache-friendly path
Photos, videos, and voice notes are large, and pushing them through the same real-time relay that carries text messages would be a mistake, because a single large transfer could occupy a connection for seconds while small messages wait behind it. Media therefore takes a completely separate path, and the message relay never carries the bytes.
Media never flows through the message relay. It is encrypted, uploaded to storage, and referenced from the chat message by a small pointer.
When you send a photo, your device encrypts the file locally, then uploads the ciphertext to a blob store fronted by a content delivery network. The file is addressed by a hash of its content, which enables an important optimization: if the same file has already been uploaded, perhaps a meme that is being forwarded across millions of chats, it does not need to be uploaded or stored again, the existing copy is referenced. The chat message that travels through the relay is tiny. It carries only a pointer to the blob, the decryption key, and the content hash, not the file itself. The recipient receives this small message, downloads the encrypted blob from the nearest CDN edge, and decrypts it locally with the key from the message. The encryption keeps the property intact end to end, because the blob store holds only ciphertext it cannot read. Keep large payloads off the real-time path, store them in a CDN-fronted blob store, reference them by a small pointer, and deduplicate by content hash, and a viral forward that reaches millions of people is uploaded once and served from cache everywhere.
Failure mode: the reconnect storm
A system this dependent on persistent connections has a failure mode that a request-response system does not, and it is worth designing for explicitly because it is the failure that turns a small problem into a large one. The danger is not that a connection server crashes. The danger is what its clients do in the second after it crashes.
When a box holding millions of connections dies, the real damage is every one of those clients trying to reconnect in the same instant.
When a connection server holding two million sockets fails, all two million connections drop at once. Every one of those clients notices immediately and tries to reconnect. If they all reconnect in the same instant, they create a thundering herd, a synchronized spike of connection attempts that slams into the surviving fleet all at once and can knock over the machines that were supposed to absorb the load, cascading the failure. On top of the connection load, the session registry takes a write storm as two million device-to-server mappings have to be rewritten to point at their new homes. The defenses are unglamorous and essential. Clients reconnect after a randomized backoff, spreading the herd across a window of time instead of a single instant, so the surviving fleet sees a smooth ramp rather than a wall. Planned maintenance drains connections gradually rather than dropping them, and crashes are contained by supervision so only the failed component restarts. And the fleet runs with enough spare headroom, planning for the loss of a machine, that it can absorb the reconnections of a full box without tipping over. Design for the reconnect, not just the crash, and a lost server becomes a blip instead of an outage.
The patterns that transfer
Every choice in this design bought something and cost something, and naming both sides is what separates understanding the system from reciting it. Holding persistent connections buys instant push but costs you the memory and the reconnect-storm risk of keeping billions of sockets open. Store-and-forward with delete-on-delivery buys a tiny storage footprint but costs you the ability to offer server-side history and search. End-to-end encryption buys real privacy and a smaller, safer server but costs you the ability to do anything server-side that requires reading messages, including simple spam filtering on content. Sender Keys buy cheap group messaging but add key-distribution complexity on membership changes. In every case the design traded capability for simplicity and scale, which is the right trade for a system whose job is to do one thing, relay messages, for a meaningful fraction of humanity.
Strip away the chat product and these four decisions reappear in any system that pushes to billions of live clients.
The reason WhatsApp is worth this much attention is that its lessons are not about chat. Four of them transfer directly to any large, real-time, push-based system, and you will build more of those than you expect. First, if clients need to be pushed to in real time, hold a persistent connection per client and back it with something cheap enough to run millions of them, rather than polling; the whole economics of the system rides on the cost of an idle connection. Second, use store-and-forward with a wake-up for the offline case, so a client that is not listening never forces you into a permanent server-side mailbox. Third, keep the delivery path to a single relay hop and push everything heavy, media, receipts, analytics, off to the side, because whatever sits on that path is multiplied by your entire connected population. Fourth, let the trust model drive the architecture, because a server that is not allowed to read its payload is a smaller, simpler, and safer thing to operate than one that is. Learn to see which of these levers your own bottleneck sits on, and the design stops being a memorized answer and becomes a method.
I teach system design this way, from first principles with real diagrams and the trade-offs that only surface in production, as an interactive course at systemdesign.academy. The foundation lessons are free and need no signup.
Read the free lessons: https://systemdesign.academy
Top comments (0)