"Design WhatsApp" flips the usual system-design instinct on its head. Every other design is request/response — a client asks, a server answers, done. Chat isn't like that: the server has to push a message to a recipient who never asked for it, over a connection that's been sitting idle, possibly to a phone that's offline — and it has to arrive once, in order, and report back the two grey ticks and the blue ones.
This is the condensed walkthrough; the full guide (estimates, protocol, data model, and the full production .NET 9 SignalR code) is on my site 👇
Full guide: https://prepstack.co.in/blog/design-whatsapp-system-design
The design at a glance
| Concern | Decision |
|---|---|
| Transport | Persistent WebSocket per client to a connection server |
| Routing |
Connection registry (userId -> server) + a backplane (pub/sub) |
| Offline | Store-and-forward — persist to inbox, deliver on reconnect |
| Ordering | Per-chat sequence number assigned by the server |
| Receipts | sent -> delivered -> read, each an ack flowing back |
| Delivery |
At-least-once + client dedup by clientMsgId
|
Why chat is different: the server pushes
In every request/response system the client drives. In chat, the server drives — it pushes to a recipient who isn't asking. That single inversion is why you need persistent connections, a registry to find them, a backplane to route between them, and an inbox for when they're gone. Everything else is receipts and ordering on top.
Estimate it — 500M daily users, ~40 messages/day:
Messages: 500M x 40 / 86,400s ~ 230,000 messages/sec (peak ~1M/sec)
Concurrent connections: hundreds of millions online at once
-> at ~100k sockets/server, that's thousands of connection servers
Two drivers fall out: thousands of connection servers to hold the sockets, and a routing layer so a message on server A reaches a recipient on server B.
User A ==(WebSocket)== [ Connection server 1 ]
| SEND
v
[ Message service ] -- persist (seq++) --> [ Message store ]
| look up B in [ Session registry (Redis) ]
v
[ Backplane: pub/sub or Kafka ]
| route to B's server
v
User B ==(WebSocket)== [ Connection server 2 ] -- push DELIVER --> B
| (B offline?) -> write to B's inbox, deliver on reconnect
The hard parts
Persistent connections & routing. Each client keeps one WebSocket to a connection server (through a load balancer that supports sticky, long-lived connections). The session registry (userId -> serverId, in Redis) records where each user is parked. Deliver A->B: look up B's server, publish onto the backplane, B's server pushes it down B's socket. On drop, the client reconnects (maybe to a different server) and the registry updates.
Ordering — one sequence per chat. Network races mean two messages arrive "at the same time." Don't trust client clocks. The server assigns a monotonic seq per chat at persist time; everyone orders by seq. The client's clientMsgId handles the other direction — dedup, so a retried send doesn't duplicate.
Receipts (the ticks). Three states, three acks: sent (one grey — server persisted), delivered (two grey — device received + acked), read (two blue — chat opened + acked). Each ack routes back to the sender the same way messages do.
Presence (online / last-seen / typing). Driven by connection state + heartbeats: active socket = online, last heartbeat = last-seen, typing is ephemeral. The trap is fan-out — pushing every presence change to everyone is enormous. Only send presence to users with the chat open, and throttle (typing especially). Cheap to track, expensive to broadcast.
Offline delivery — store-and-forward. No active session? Write the message to B's inbox (persistent). On reconnect, B drains the inbox in order, acking each. Because delivery is at-least-once, B's client dedups by msgId.
Group messaging. A group message fans out to each member: route to their connection server if online, else write to their inbox. Small groups are cheap; very large groups reintroduce the celebrity/fan-out problem from the news-feed design — which is exactly why real chat apps cap group size.
Scaling gotchas
- Connection servers scale horizontally (~100k sockets each); use a WebSocket-aware load balancer.
- The backplane must sustain the full message rate; partition by chat/user.
- Presence fan-out is the sneaky cost — throttle and scope it.
- Reconnection storms: a dead connection server dumps all its clients onto the survivors at once — stagger with jittered backoff or you thundering-herd yourself.
- E2E encryption: if the server routes ciphertext it can't read, that rules out server-side search/previews; receipts and ordering must work on opaque blobs.
I shipped this in production (Mattrx)
Mattrx isn't a chat app, but its real-time layer is built on exactly these primitives. Two features need the server to push: the live campaign dashboard (activity, KPI ticks, "who else is viewing this campaign" presence) and Mattrx Insights, the agentic AI assistant that streams its answer as it thinks. V1 polled the API every 10s and ran Insights as request/response — dashboards lagged and users watched a spinner for the ~4.2s an answer took. We rebuilt both on a single SignalR hub with a Redis backplane, so any of the N hub servers can reach any client — exactly the connection-server + registry + backplane spine above.
| Metric | Before | After |
|---|---|---|
| Live campaign updates | ~10s HTTP polling (stale, wasteful) | SignalR push, sub-second |
| Insights AI answer | request/response spinner (~4.2s) | streamed; first token p95 ~300ms, full p95 1.8s |
| Collaboration presence | none | live "who's viewing this campaign" |
| Transport | repeated polls per client | one persistent WebSocket + Redis backplane |
| Reconnect behaviour | full page reload | resume connection + replay missed events |
Insights streaming rides mediator.CreateStream, so the same MediatR pipeline that governs AI answers feeds the socket; the Redis backplane means a token produced on hub server 3 reaches a client parked on hub server 7; and presence is pure connection state — OnDisconnectedAsync cleans it up. (Full .NET 9 SignalR hub + Redis presence tracker is in the post.)
The model to carry forward
Chat is the system where the server pushes. Once you accept that inversion, the design writes itself: persistent connections to hold the recipients, a registry to find which server holds whom, a backplane to route between servers, and an inbox for when the recipient is gone. Layer receipts (acks flowing back), ordering (a server-assigned per-chat sequence), and scoped presence on top, and cap your groups so fan-out stays sane.
Three habits it teaches: lead with "the server pushes" (it frames every later decision); separate holding connections from routing between them (that spine is reusable across every real-time system); make order the server's job (a per-chat sequence is the only clock everyone can trust).
The full guide has the estimates, protocol, data model, all the hard parts in depth, scaling gotchas, the complete production .NET 9 SignalR hub, and the "when it's overkill" section:
https://prepstack.co.in/blog/design-whatsapp-system-design
Originally published on PrepStack.
Top comments (0)