A chat message is easy to send when both users are online. The difficult part is ensuring that one logical message survives retries, reaches multiple devices, appears in a consistent order, and remains recoverable when connections or services fail.
Imagine Alice sends Bob a message. Bob's laptop is connected, but his phone is offline. The server accepts and durably stores the message, then its acknowledgement to Alice is lost. Alice's app cannot tell whether the send succeeded, so it retries. Meanwhile, Bob's laptop receives the message, and his phone will reconnect hours later.
What should happen? Alice must not create a second logical message by retrying. Bob's laptop should not display duplicates. Bob's phone should eventually catch up, subject to the system's retention and access policies. And the sender's sent, delivered, and read indicators must not pretend to mean the same thing.
That is the real challenge behind designing a WhatsApp-like messaging platform. This article develops an illustrative architecture for a real-time chat system; it does not claim to describe WhatsApp's private implementation.
1. Requirements before architecture
Start with one-to-one messages, group conversations, low-latency delivery to connected clients, offline recovery, multi-device synchronization, delivery and read receipts, and media attachments. Presence and typing indicators improve the experience, but they are not part of the durable-message guarantee. Broadcast channels, message editing, and end-to-end encryption introduce additional design constraints that we will address separately.
The critical non-functional requirements are:
- Durability: an acknowledged send has satisfied a clearly defined durable-write contract, even if the recipient is offline.
- Idempotency: retrying the same logical send does not create another accepted message.
- Ordering: messages have a consistent order within a conversation, without requiring one global order for the entire platform.
- Recovery: disconnected devices can reconcile missed changes from durable state.
- Scalability: socket connections, message writes, history reads, group delivery, and media bytes can scale independently.
- Security: authenticate devices, authorize conversation operations, and define encryption and retention policies explicitly.
We should not promise unconditional instant delivery, exactly-once network transmission, or zero data loss under every possible disaster. The guarantees depend on the storage write policy, failure model, retention window, and client behavior.
A scale model, not a claim about WhatsApp
For a sizing exercise, assume 500 million daily active users, 200 million concurrent connections, and 100 billion messages per day. That is approximately 1.16 million messages per second on average. If the effective stored message record averages 1 KB, the raw logical ingest is about 100 TB/day, or 3 PB over 30 days, before replication, indexes, attachments, and other overhead. Peaks may be substantially higher.
These are illustrative assumptions, not published operating figures. They tell us to plan separately for persistent sockets, sustained writes, fan-out amplification, reconnect bursts, and storage growth. A hypothetical 50,000 sockets per gateway would imply about 4,000 gateways for 200 million connections before spare capacity; actual density depends on workload, memory, TLS, runtime, network throughput, and failure headroom.
2. Start simple, then find the breaking point
A small chat application can start with clients connecting to an application server backed by a relational database. The server authenticates requests, writes messages, and forwards them to connected recipients.
Alice ──> App Server ──> Message Database
│
└──────> Bob (if connected)
This works at modest scale. But once users are connected to many application instances, Alice's server cannot assume Bob is connected to the same one. Long-lived sockets consume a different kind of capacity from database writes. Offline delivery and group messaging also need durable recovery rather than in-memory queues.
The next architecture separates connection management, message acceptance and storage, and delivery/synchronization. These boundaries solve different problems; they are not microservices added merely for appearance.
3. WebSocket gateways and connection routing
WebSocket is a practical default for interactive chat because it supports persistent bidirectional communication. The client can send messages and acknowledgements while the server pushes new events over the same connection.
Server-Sent Events plus HTTP requests can work for primarily server-to-client applications; long polling is another compatible fallback. Neither is inherently invalid, but frequent bidirectional chat tends to favor WebSocket. HTTP/2 Server Push is not the relevant modern alternative.
Alice's phone ──> Load Balancer ──> Gateway A ──┐
│
Bob's laptop ──> Load Balancer ──> Gateway B ────┼──> Messaging Service
│ │
Bob's phone (offline) │ v
│ Message Store
Connection Registry <────────┘
A gateway authenticates the device session, maintains heartbeats and bounded socket buffers, and registers its route. The connection registry conceptually maps:
user_id -> {
device_id/session_id -> gateway_id, connection_id, session_epoch
}
The registry is ephemeral routing information, not the source of truth for messages. One user can have several devices and several sessions. A stale registry entry may send a delivery attempt to a dead connection; the gateway can reject it, the routing layer can re-check the route, and durable sync remains the fallback.
Session epochs or equivalent generation identifiers prevent old sockets from silently replacing newer connections. Gateways must also drain connections during deployments and bound their outbound queues. If Bob's laptop stops reading, retaining unlimited pending messages in gateway memory is not reliable delivery; it is an eventual memory incident. Disconnect the slow session when necessary and let it recover from durable storage.
4. Define the message-send contract
Alice generates a stable client_message_id before her first send attempt:
{
"type": "message.send",
"conversation_id": "conv-42",
"client_message_id": "alice-phone-unique-123",
"content_type": "text",
"payload": "Hello Bob"
}
The server authenticates Alice, checks that she can send to the conversation, validates the request, and atomically resolves the client-generated request identity to one canonical server message ID and one conversation-order position. It then commits the message according to its durable-write contract and returns an acceptance acknowledgement.
The sender acknowledgement means the server durably accepted the logical message. It does not mean Bob's device received it, the message was read, a push notification succeeded, or every downstream fan-out task finished.
The idempotency mapping might be scoped by (sender_id, sender_device_id, client_message_id); its precise scope and retention policy are part of the protocol. Retrying with the same ID but different content should be rejected or handled under an explicitly defined policy, not silently treated as a new message.
The lost-ACK problem
Alice Messaging Service Durable Store
| SEND (client_id=123) | |
|----------------------------->|--- atomic accept/write --->|
| |<--------- committed -------|
| ACK is lost X <----------|
| | |
| RETRY (client_id=123) | |
|----------------------------->|--- resolve same identity -->|
|<--- original message_id -----| |
A naïve SELECT followed by INSERT is insufficient: two concurrent requests can both observe that the ID does not exist. Use a uniqueness constraint, conditional write, transaction, or equivalent atomic mechanism. The deduplication record, canonical message identity, and ordering decision must be coordinated so a crash does not create two accepted versions of the same logical send.
This is exactly-once logical acceptance, under the defined idempotency scope and retention assumptions—not exactly-once packet delivery. Network sends, event deliveries, and client replays can still happen more than once.
5. Ordering is per conversation, not global
If Alice and Bob send at nearly the same time, their devices may observe different network arrival orders. Timestamps are not a reliable strict-order mechanism because clocks differ and concurrent requests race.
For this design, an ordering authority assigns each accepted message a monotonically increasing conversation sequence:
Conversation conv-42
seq 418 -> Alice: Hello
seq 419 -> Bob: Hi
seq 420 -> Alice: Are you free?
Possible implementations include an ordered log partition, a single conversation owner/sequencer protected by fencing, or an appropriate storage-side conditional allocation mechanism. Consistent hashing can help route requests to an owner, but routing to the same instance is not proof of exclusive write authority when instances fail or ownership changes.
Strict contiguous numbering is a stronger promise than merely increasing order. If we promise contiguous committed sequences, the sequence allocation and durable acceptance protocol must not leave permanent unexplained holes after crashes. An implementation can instead define visible committed order separately from provisional reservations. The important point is to state which contract the client may rely on.
Suppose Bob receives sequence 420 before 419. His client tracks the highest contiguous, safely applied sequence, buffers newer events within limits, and requests the missing range. It should not advance its checkpoint past an unresolved gap simply because 420 arrived. If targeted repair fails, it can fetch conversation history or reconcile through device sync.
6. Persist first; deliver afterward
The central write path is:
Client send
|
v
Authenticate + authorize + deduplicate
|
v
Assign canonical identity and conversation order
|
v
Durable message acceptance
| \
v v
Sender ACK Durable delivery/sync work
|
Online device routing
|
Offline catch-up / push hint
A successful write must not be followed by a best-effort event publish that can disappear forever if the process crashes. If delivery workers or the sync feed depend on an event stream, use a transactional outbox, storage-derived change feed, or another recoverable handoff with appropriate atomicity and replay guarantees. Otherwise the database can contain an accepted message that downstream systems never learn about.
The event bus might be Kafka, but Kafka is not mandatory for every messaging architecture. It can handle asynchronous group fan-out, notification work, receipt propagation, and operational events. If a feature relies on Kafka, an outage delays that feature. If Kafka is on the synchronous acceptance path, its availability also affects sender ACKs. Both designs are possible; the dependency boundary must be explicit.
Consumers should tolerate at-least-once event processing, maintain idempotent effects, and track lag. In-memory producer buffers alone do not provide durable outage recovery.
7. Offline delivery and multi-device sync
Bob's laptop is online, so the system can attempt immediate delivery to that session. Bob's phone is offline, so there is no usable live route. Neither condition changes whether Alice's message was durably accepted.
When Bob's phone reconnects, it authenticates and presents its own durable sync cursor. The server returns changes after that checkpoint: messages, membership changes, relevant receipts, and other supported conversation events. The device applies each batch safely, deduplicates by stable identity, and advances its checkpoint only after successful local application.
Bob laptop: cursor 847 ── receives new event ──> 848
Bob phone: cursor 842 ── offline
Bob phone reconnects ── fetch events after 842 ──> apply ──> 848
A cursor can represent a device-scoped ordered change feed rather than the sequence number of any one conversation. Conversation sequence and device sync cursor are different ordering domains. They should not be treated as interchangeable integers.
Realtime pushes and mobile notifications are wake-up hints, not authoritative history. A missed WebSocket event or failed APNs/FCM notification must not prevent the next sync from discovering the accepted message.
A device offline beyond the retained incremental change history may have an expired cursor. The server must explicitly require a snapshot or full reconciliation path, respecting deletion, membership, and retention semantics. No design can promise recovery of content already removed under a valid retention or deletion policy.
8. Sent, delivered, and read are separate events
A useful state model is:
CLIENT_PENDING -> SERVER_ACCEPTED -> RECIPIENT_DELIVERED -> READ
These labels are product semantics, not automatic network guarantees.
In this illustrative design:
- Server accepted: the canonical message has passed the durable write contract.
- Delivered: at least one eligible recipient device has acknowledged that the app received and safely accepted the message locally.
- Read: a recipient action or client state meets the product's defined read criterion.
A gateway successfully writing bytes to a socket is not sufficient evidence that the recipient application processed them. Likewise, delivered-to-one-device and delivered-to-all-devices are different possible products. We choose the former for this example and track richer per-device state internally if needed.
Read state can often be represented by a monotonic read_up_to_sequence for each user and conversation, provided the product defines what that high-watermark means. Large groups may aggregate, restrict, or omit detailed per-member read receipts because otherwise receipt traffic can overwhelm message traffic.
Receipt events can be late or duplicated. Processing must be idempotent and must not regress a valid later state to an earlier one.
9. Group messaging changes the cost model
One-to-one delivery has a small number of recipients. A group send may need to notify hundreds or thousands of devices; a broadcast channel may have an audience orders of magnitude larger.
Two common distribution strategies are:
Write fan-out: create recipient-oriented delivery or inbox work when a message is accepted. This makes individual catch-up and notifications convenient but amplifies writes with audience size.
Read fan-out: store the message once in a canonical conversation or channel log, then let members fetch it. This reduces per-message write amplification but shifts cost toward reads, indexes, and caching.
A hybrid design often makes sense: a canonical conversation log plus recipient-oriented sync indexes for interactive chats, while very large broadcast channels use store-once, notify-and-fetch behavior. There is no universal group-size cutoff that decides the architecture.
Group membership is also a correctness problem. If Carol is removed while Alice sends a message, which membership version applies? Bind acceptance and fan-out to a defined membership snapshot or equivalent serialized rule. Otherwise different workers may produce different recipient sets for the same message. Encryption-enabled groups add key-management and membership-epoch concerns on top of that authorization decision.
The sender's ACK should not wait for thousands of devices to acknowledge delivery. Durable acceptance and downstream group fan-out are separate operations, with explicit lag monitoring and replay behavior.
10. Store messages for ordered reads and bounded partitions
The durable message store needs high write throughput, efficient history pagination, and access by conversation and sequence. One possible physical layout is:
Partition key: (conversation_id, bucket_id)
Sort key: message_sequence
Value: message_id, sender_id, payload/ciphertext, metadata
Buckets may use sequence ranges, time windows, or bounded-size policies. This avoids an indefinitely growing physical partition for a long-lived conversation, though bucketing does not automatically fix a hot current bucket. Very active conversations can still overload their current storage replicas or ordering authority.
A relational database can be suitable at modest scale, particularly when transactional requirements dominate. At extreme append volume, a wide-column or distributed key-value store may better match partition-local history reads. Cassandra, ScyllaDB, DynamoDB, and relational systems have different conditional-write, indexing, transaction, and consistency characteristics; choosing a brand does not itself prove the protocol correct.
The canonical conversation log and a per-user or per-device sync index are different views. If a sync index is derived asynchronously, the durable handoff and replay path must ensure that acknowledged messages eventually become discoverable by authorized devices, subject to retention and availability constraints.
11. Presence, typing, and push notifications are not durable messages
Presence is usually derived from active sessions with heartbeat-refreshed leases or TTLs. It is approximate, affected by mobile background behavior, and subject to privacy rules. If any permitted active session is live, a product may show a user as online—but this is a policy, not a universally accurate fact.
Typing indicators are short-lived events that can be coalesced, rate-limited, or dropped under load. Persisting a backlog of stale typing events adds cost without value.
Mobile push notifications are also best-effort wake-up mechanisms. They may be delayed, duplicated, suppressed, or unavailable. A push payload must respect privacy and, where enabled, end-to-end encryption constraints. The client still fetches authoritative state from the messaging service after waking.
These workloads should degrade before the system sacrifices the integrity of already accepted messages.
12. Media messages take a separate byte path
Sending a photo should not mean streaming megabytes through the same gateway buffers that handle text messages and receipts.
Sender ── authorized upload ──> Media Service / Object Storage
|
└── message.send(media reference) ──> Message Service
|
Recipient <── authorized download / CDN <─┘
The client uploads the media through an authorized, potentially resumable object-storage path. The message contains a reference and the metadata needed for retrieval. The service must define when that reference is considered valid, how orphaned uploads are cleaned up, and how access is checked on download.
For private content, a CDN requires its own appropriate access-control strategy; an object-storage presigned URL is not automatically a secure CDN authorization scheme. Media can fail independently of text chat, and clients should surface that failure rather than treating a missing upload as a successfully delivered attachment.
13. End-to-end encryption changes the trust model
End-to-end encryption (E2EE) means the service should not possess the plaintext keys needed to read protected message content under the chosen threat model and protocol. Servers still need to authenticate users and devices, route ciphertext, store encrypted payloads, coordinate device enrollment, and manage operational metadata.
Multiple devices and groups make key distribution more complicated. Adding or removing a device, changing group membership, and restoring backups all need explicit cryptographic policies. Encryption also constrains server-side search, moderation, and content-derived analytics.
E2EE is not achieved merely by storing encrypted strings in a database or enabling TLS. A real deployment requires a reviewed cryptographic protocol, secure client key handling, device verification and recovery semantics, and careful metadata-leakage analysis. This article treats E2EE as an architectural constraint rather than inventing a complete secure messaging protocol.
14. What happens when components fail?
The gateway crashes. Connected clients lose their sockets, reconnect with backoff and jitter, register new routes, and resume from durable sync cursors. An accepted message is not dependent on the dead gateway's memory.
Alice's ACK is lost. She retries using the same client-generated ID. Atomic idempotency returns the original canonical message and order, rather than writing a duplicate.
Bob receives the same event twice. His device deduplicates by canonical message ID and applies the event idempotently. At-least-once delivery is compatible with one logical chat bubble.
Bob receives sequence 103 before 102. His client detects the gap, buffers within a limit, fetches missing history, and does not advance a contiguous checkpoint incorrectly.
The message store cannot meet its write contract. The service must not send a success ACK for an uncommitted message. Clients can retry safely, and already acknowledged data retains only the durability guarantee the chosen storage configuration actually provides.
Kafka or a delivery worker is down. Kafka-dependent fan-out, push, or receipt processing may lag. Recoverable outbox or change-feed state preserves pending work, and workers replay idempotently after recovery. If Kafka is on the acceptance path, new sends may also be blocked.
Millions of devices reconnect together. Gateways, authentication, registries, and sync APIs face a recovery surge. Use jittered reconnect, admission control, bounded catch-up batches, and capacity headroom rather than allowing recovery to create a second outage.
One conversation becomes extremely hot. Current-bucket write pressure and sequencer contention can persist even with historical bucketing. Apply workload-aware limits or specialized handling while preserving the stated ordering contract.
A region fails. Cross-region routing alone is not enough. The system must define replication lag, recovery point and time objectives, and how a new conversation writer gains authority without a stale writer continuing to accept conflicting sequences.
15. Multi-region messaging and split-brain prevention
A defensible baseline gives each conversation one current write authority, often associated with a home region. Users may connect to nearby gateways around the world, but accepted writes for that conversation pass through its authoritative ordering path. Other regions can handle routing, caching, delivery, and replication.
Active-active writes across regions are possible, but require a more sophisticated ordering or conflict contract. Two regions must not independently claim to own the same strict conversation sequence during a network partition. Lease ownership must be backed by effective fencing or another mechanism that rejects stale writers; a lease timestamp alone is not sufficient if old owners can still write.
Cross-region replication may be asynchronous, which means regional disaster recovery can involve a nonzero recovery point objective. Claiming both immediate local writes everywhere and strict global per-conversation ordering without explaining the coordination cost would be misleading.
16. Putting the architecture together
Mobile / Web / Desktop Clients
|
Load Balancer
|
WebSocket Gateways
| |
| Connection Registry
v
Messaging Service
(auth, idempotency,
conversation ordering)
|
Durable Message Store
|
Durable Outbox / Change Feed
|
Event Stream / Workers
/ | \
Live fan-out Sync index Push hints
| | |
Gateways Sync API APNs / FCM
|
Reconnecting
devices
Separate: conversation/membership metadata,
presence/typing, media object storage + CDN,
key/device management if E2EE is enabled
Return to Alice and Bob. Alice sends M with a stable client ID. The service authorizes the send, assigns a canonical message identity and conversation order, commits the durable message and its recoverable downstream handoff, then attempts to ACK Alice. The ACK disappears, but the write does not. Alice retries, and the service returns the original result.
Bob's connected laptop receives M and acknowledges receipt according to the product's delivery rule. Bob's offline phone later reconnects and obtains M through its own sync cursor. If the laptop gets a duplicate event, it ignores the duplicate logical message. If the phone discovers a sequence gap, it repairs it from authoritative history. None of these cases requires pretending that a WebSocket is a durable queue.
The design succeeds because it draws precise boundaries: durable acceptance is not device delivery; device delivery is not reading; live routing is not offline recovery; and per-conversation order is not global order. Once those contracts are explicit, component choices become much easier to evaluate.
References and further reading
For deeper explanations of the design decisions and their failure boundaries, see SeeItFlow's Messaging Platform module:
Top comments (0)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.