System design case study: WhatsApp

Part 5 of 8 · week 5 of the learning path

A chat system is a store-and-forward pipeline: hold a persistent connection per user, store each message until the recipient's device confirms it, and push to a connected device the moment it arrives.

Learning objectives

• Apply the design method end to end on a real-time messaging product.
• Design delivery guarantees, ordering, group fan-out and presence.
• Explain how end-to-end encryption changes the architecture.

1. Requirements

• Functional: one-to-one and group chat, sent, delivered and read receipts, offline delivery, photos and videos, last seen and typing indicators, multiple devices per user.
• Non-functional: low latency for online users, no lost messages, order preserved within a chat, high availability, end-to-end encryption, and billions of users.

2. Estimates. Meta, then Facebook, reported in 2020 that WhatsApp delivers roughly 100 billion messages a day, which this exercise uses as the input.

Quantity	Calculation	Result
Average message rate	100 billion divided by 86,400	about 1.2 million per second
Peak rate at 3 times average	1.2 million times 3	about 3.5 million per second
Bandwidth at 1 KB per message	3.5 million times 1 KB	about 3.5 GB per second, 28 Gbit per second
Total if every message were kept	100 billion times 1 KB	about 100 TB per day
Backlog if 1% sit undelivered	1 billion times 1 KB	about 1 TB
Connection servers	500 million concurrent connections divided by about 1 million per server	about 500 servers, plan 2 to 3 times that for headroom and failover

The key insight is that the server only needs to hold undelivered messages, so storage tracks the backlog, not the total ever sent.

3. API and data model

• Over the open connection: send (message ID, chat ID, recipient, encrypted payload), ack (message ID, status of delivered or read), and ephemeral presence and typing events.
• Over HTTPS: registration, key upload and media upload.
• Data: users, devices, chats and group members, an inbox per recipient device (inbox sequence number, message ID, encrypted payload, expiry), and receipts.

4. High-level design

 Sender app                                   Receiver app
     |  (1) send, client message ID                ^
     v                                             | (6) deliver, device acks
 Connection server A                       Connection server B
     |  (2)                                        ^
     v                                             | (5) relay if connected
 +----------------+  (4) where is the user?  +-------------+
 |  Chat service  | -----------------------> | Session map |
 +----------------+                          +-------------+
     |  (3) store in recipient inbox
     v                                  (5) if offline: Push gateway -> notification
 +---------------+
 | Message store |
 +---------------+

Step numbers in the diagram match the list below.

The numbers on the diagram match the steps below, and the chat service is where most design decisions sit.

1. The sender's app sends the message over its open connection, with a message ID the client generated.
2. The connection server forwards it to the chat service.
3. The chat service stores it in the recipient's inbox.
4. The chat service asks the session map whether the recipient is connected, and to which server.
5. If connected, it relays through that connection server; if not, it asks the push gateway to send a notification.
6. The recipient's device receives the message and acknowledges it; the ack travels back and the sender sees delivered.

5. Deep dives

• Delivery guarantee: at least once with acknowledgements. The client-generated message ID lets the server drop duplicates when a send is retried. The message leaves the inbox only after the device acks, and unread messages expire after a retention period.
• Ordering: the server assigns an inbox sequence number per recipient, and the app shows messages in that order within a chat.
• Groups: fan out on write, storing one copy per member inbox; cap the group size so fan-out stays bounded. Media is uploaded once and referenced.
• Presence and typing: ephemeral, never stored, sent only to people who have that chat open, and throttled.
• Multiple devices: each device has its own keys and inbox, and the sender encrypts for each.
• End-to-end encryption: the apps use a protocol like the Signal protocol; the server holds public pre-keys and relays only ciphertext. This means no server-side search or content scanning, and backups need their own encryption.
• Media: the app encrypts the file, uploads it to object storage through a media service, and sends the link and key in the message; downloads use a CDN.
• Connection tier: servers hold the sockets and are nearly stateless. The session map is a sharded in-memory store keyed by user ID, kept fresh by heartbeats and expiry.

6. Trade-offs: store-and-forward is private and cheap but gives no server-side history; fan-out on write makes reads fast but costs writes for big groups; a persistent connection gives low latency but costs battery and connection capacity.

7. Failures and monitoring. If a connection server dies, clients reconnect with exponential backoff and jitter to avoid a thundering herd, and session entries expire; messages are safe in the inbox. If the push provider is down, messages stay queued. Watch connected sockets, p99 send-to-deliver latency, ack ratio, inbox depth, push error rate and reconnect rate.

Real-world note: WhatsApp engineers have written publicly about running on Erlang and holding more than a million connections on a single server, which is why the connection tier is so small for its size.

Design exercise: add a status feature (stories that disappear after 24 hours) and message search to this design. Say where the data lives, how long it lives, and what end-to-end encryption forces you to do on the device.

Quiz 5 (5 questions, pass mark 4)

1. The server keeps a message only until the recipient's device acknowledges it. What does that mean for storage sizing?
   • A) You must store the full history
   • B) Storage tracks the undelivered backlog, not the total ever sent
   • C) No storage is needed at all
   • D) Storage depends only on images
2. A client retries a send after a timeout. How does the server avoid delivering it twice?
   • A) Drop every retry
   • B) Lock the user
   • C) Use a client-generated message ID and deduplicate
   • D) Sort by timestamp
3. What does a delivered tick mean?
   • A) The server got it
   • B) The recipient read it
   • C) The message was encrypted
   • D) The recipient's device received and acknowledged it
4. The recipient is offline. What happens?
   • A) The message waits in their inbox and a push notification is sent; it is delivered when the device reconnects
   • B) The message is dropped
   • C) The sender must resend
   • D) The server decrypts and stores the history
5. A connection server fails and millions of clients reconnect at once. What reduces the damage?
   • A) Reconnect immediately in a tight loop
   • B) Randomized exponential backoff on reconnect
   • C) Disable push
   • D) Add a bigger cache

Quiz 5 answers

1. B the server keeps only undelivered messages, so storage follows the backlog.
2. C
3. D
4. A
5. B

Sources and further reading

• WhatsApp is now delivering roughly 100 billion messages a day, TechCrunch, October 2020, the source of the volume figure.
• One million and 1 million is so 2011 on the WhatsApp blog, on connections per server with FreeBSD and Erlang.
• WhatsApp Encryption Overview, the technical white paper on registration, sessions, groups and media.
• Signal Protocol specifications, including X3DH and the Double Ratchet.

The delivery, inbox and group fan-out design is a teaching design built on these facts, not WhatsApp's published architecture.

---

Part 5 of 8 in the System Design Learning Path series. Next: part 6.