You send a message in a Slack channel and everyone in that channel sees it within a fraction of a second, whether they have the app open on a laptop, a phone, or both. Making that feel instant, for millions of connected users across millions of channels, is a very different problem from a normal request-response web app. The web is built around clients asking servers for data. Real-time messaging needs servers to push data to clients that did not ask.
The core problem
HTTP is pull. A browser requests, the server responds, the connection closes. To get a message to a user the instant it is sent, you would have to poll constantly, which is wasteful and still laggy. Real-time chat inverts the flow: the server must be able to reach out to a specific set of connected clients the moment something happens. That requires long-lived connections and a way to route a message to exactly the right connections among millions.
Key design decisions
Hold persistent connections with WebSockets. Each client opens a WebSocket to a gateway server and keeps it open. Now the server can push at any time with no polling. The cost is state: every open connection consumes memory and lives on a specific machine, so a fleet of gateway servers holds millions of these connections between them.
Separate the connection layer from the message logic. Gateway servers do one job: own client connections and shuttle bytes. The logic of "who is in this channel and who should receive this message" lives behind them. This split lets you scale connection handling and business logic independently, and it means a gateway can stay dumb and fast.
Solve the routing problem with a pub/sub backbone. Here is the crux. A message sent to channel C must reach every online member of C, but those members are scattered across many gateway servers. The clean answer is publish/subscribe. When a message is published to channel C, an internal pub/sub system fans it out to every gateway that currently holds a subscriber for C, and each gateway pushes it down the right sockets. No gateway needs to know about any other gateway; they all just subscribe to the channels their users care about.
Persist first, then fan out. The message is written to durable storage before or alongside delivery, so history survives and offline users can catch up. Real-time delivery is a fast path layered on top of a durable write, not a replacement for it. A user who was offline reconnects and pulls the messages they missed from the store.
Presence and delivery guarantees
Presence (who is online) is its own noisy subsystem, because status changes constantly and naively broadcasting every change to everyone melts under load. It is usually handled with heartbeats and heavy batching. For delivery, each client tracks the last message it has seen, so on reconnect it asks for everything after that point. Clients dedupe by message id because at-least-once delivery means the same message can arrive twice.
How the real systems do it
Slack maintains long-lived connections and, on connect, sends the client the state it needs to get current, then streams events over the socket after that. The internal architecture separates the edge that holds connections from the services that own channels, messages, and presence, with a pub/sub layer moving events between them. Discord and WhatsApp follow the same broad pattern: persistent connections at the edge, a fanout backbone in the middle, durable storage underneath, and per-client cursors so nobody misses a message across a reconnect.
The idea to carry into an interview: real-time messaging flips the web's pull model to push, which forces persistent connections, which forces you to solve fanout across a fleet. Pub/sub is the piece that makes routing to the right connections tractable, and durable storage plus per-client cursors is what makes it reliable rather than merely fast.
I wrote the full breakdown, with the connection lifecycle and the fanout diagram, here: https://www.systemdesign.academy/interview/design-slack
Top comments (0)