DEV Community: aditya emmadishetty

Designing a Real-Time Chat System (Slack-style)

aditya emmadishetty — Tue, 28 Jul 2026 08:43:31 +0000

External link Design real time chat system

This is a write-up of a system design exercise I worked through for real-time
communication between users, modeled loosely on Slack. It covers the
requirements, the evolution from polling to WebSockets, how to scale
WebSocket connections across multiple servers, and the database schema.

To keep the core design easy to reason about, I've deliberately scoped out
multi-device support and strict message ordering guarantees — these
are called out as improvements at the end rather than folded into the main
design.

Requirements

Multiple users, multiple channels
Users can DM each other or message in a channel
Real-time chat — messages should be delivered live, not on refresh
Historical messages can be scrolled through (pagination)

Architecture at a Glance

Before diving into the individual pieces, here's the end state we're
building toward: clients hold a persistent WebSocket connection to one of
several WS servers, a discovery layer tracks which server each user is on,
and a Chat Service is the single source of truth for persistence.

flowchart LR
    A[User A] -- WebSocket --> WS1[WS Server 1]
    B[User B] -- WebSocket --> WS2[WS Server 2]
    C[User C] -- WebSocket --> WS2

    WS1 <--> DS[Discovery Service<br/>user_id to server mapping]
    WS2 <--> DS

    WS1 <--> MB[(Message Broker<br/>Pub/Sub)]
    WS2 <--> MB

    MB --> CS[Chat Service]
    CS --> DB[(Database)]

    style DS fill:#f9f0d9,stroke:#c9a227
    style MB fill:#dde8f5,stroke:#4a7ab5
    style DB fill:#e2f0e2,stroke:#4a8a4a

WS Servers hold the actual live connections and know nothing about routing — they just push to whichever local sockets they're told to.
Discovery Service is the "phone book": given a user_id, it tells you which WS server (if any) that user is currently connected to.
Message Broker is the pub/sub layer that lets a message published on one WS server's side reach a subscriber connected to a different WS server.
Chat Service + DB is the durability layer — every message is persisted here regardless of whether anyone was online to receive it live.

Data Model

Four core entities:

User
  id
  name

Message
  id
  from
  channel_id

Channel
  id
  name
  type: direct / group

ChannelUserMapping
  id
  channel_id
  user_id

A few useful queries this schema supports directly:

-- messages in a channel
SELECT * FROM messages WHERE channel_id = ?

-- users in a channel
SELECT DISTINCT(user_id) FROM channel_user_mapping WHERE channel_id = ?

Member joins/leaves are handled as separate operations against
ChannelUserMapping, rather than being baked into the channel or message
tables.

Getting to Real-Time: Three Approaches

1. Short Polling

The simplest approach: the client repeatedly calls an endpoint like
GET /messages?channel_id=?&from=?, and the server returns new messages if
any exist, or an empty response otherwise.

Problem: the client is guessing when to ask. Poll too infrequently and
messages feel delayed; poll too frequently and you're hammering the server
with mostly-empty responses. It's simple but not truly real-time.

sequenceDiagram
    participant B as User B
    participant S as Server

    loop every few seconds
        B->>S: GET /messages?channel_id=X&from=Y
        S-->>B: [] (no new messages)
    end
    Note over B,S: eventually...
    B->>S: GET /messages?channel_id=X&from=Y
    S-->>B: [new message]

2. Long Polling

The client keeps the connection open, and the server only responds once a
new message actually shows up (or a timeout elapses, after which the client
reopens the connection).

Problem: this is still fundamentally client-initiated. The server can
only respond — it can't push new data on a connection unless the client
asked first. That still doesn't give you real bidirectional, low-latency
communication, and the connection churn (constantly closing and reopening)
is wasteful.

sequenceDiagram
    participant B as User B
    participant S as Server

    B->>S: GET /messages?channel_id=X&from=Y
    Note over S: connection held open...
    Note over S: new message arrives
    S-->>B: [new message]
    B->>S: GET /messages?channel_id=X&from=Y (reopen)
    Note over S: connection held open again...

3. WebSockets

WebSockets open a persistent, bidirectional connection between the client
and server. Once established, either side can send data at any time. This is
the right building block for real-time chat — it's the actual mechanism
underneath both Slack and most modern chat apps. (Server-Sent Events were
also considered, but ruled out since they're server-to-client only — the
client still needs a separate channel to send messages.)

sequenceDiagram
    participant B as User B
    participant S as Server

    B->>S: WebSocket handshake (upgrade)
    S-->>B: connection established (persistent)
    Note over B,S: connection stays open
    S->>B: push: new message (server-initiated!)
    B->>S: send: new message

Scaling WebSocket Connections

A single server can only hold so many concurrent open socket connections. If
you have many users, you need multiple servers holding connections, which
introduces a new problem: if user A is connected to server 1 and sends a
message to user B, who is connected to server 2 — how does server 1 know
where to deliver it?

Discovery Service

The fix is a discovery service that tracks which server each user's
active WebSocket connection lives on:

When a client first loads the app, it registers with the discovery service, recording which WS server it has an open connection to.
When user A sends a message, it's routed to a Chat Service, which persists it to the DB and also asks the discovery service where user B is connected.
If B has an active connection, the message is routed to the correct WS server and pushed down B's socket in real time.
If B isn't connected anywhere, the message is just persisted — B will see it next time they load the channel or reconnect.

sequenceDiagram
    participant A as User A
    participant WS1 as WS Server 1
    participant CS as Chat Service
    participant DS as Discovery Service
    participant WS2 as WS Server 2
    participant B as User B

    A->>WS1: send message to B
    WS1->>CS: forward message
    CS->>CS: persist to DB
    CS->>DS: where is B connected?
    DS-->>CS: WS Server 2
    CS->>WS2: deliver message
    WS2->>B: push message (live)

Channel / Group Fan-out

For channels (as opposed to 1:1 DMs), the same idea extends via pub/sub:
each WS server subscribes to the channels its connected users are members
of. When a message is published to a channel, it's fanned out to every WS
server that has at least one relevant subscriber, and each of those servers
pushes it to its own connected sockets.

This is fan-out-on-write: the fan-out work happens at message-send time,
not at read time. It's the standard approach for chat systems at Slack's
scale (as opposed to fan-out-on-read, which is more common for large,
algorithmic feeds).

flowchart TB
    A[User A sends to Channel] --> CS[Chat Service]
    CS -->|1. persist| DB[(Database)]
    CS -->|2. publish to channel topic| MB[(Message Broker)]

    MB -->|subscribed| WS1[WS Server 1]
    MB -->|subscribed| WS2[WS Server 2]

    WS1 --> B[User B]
    WS1 --> C[User C]
    WS2 --> D[User D]

    style CS fill:#dde8f5,stroke:#4a7ab5
    style MB fill:#f9f0d9,stroke:#c9a227

Note the fan-out is per WS server that has relevant subscribers, not per
individual user — if 50 channel members happen to be connected to the same
WS server, that's a single publish to that server, which then pushes to its
50 local sockets.

Avoiding Duplicate Persistence

One subtlety: a message needs to be persisted (Chat Service → DB) and
also delivered live (via the WS/pub-sub path) — these are two different
concerns and should be handled by two different components, not conflated:

Chat Service is the single source of truth for persistence — it writes every message to the DB exactly once.
The WS/pub-sub queue is purely for real-time fan-out to connected clients and should not also be treated as a persistence layer. Keeping these responsibilities separate avoids double-writes and keeps recovery logic simple — if delivery fails, the message is still safely in the DB and will show up on next fetch.

Handling Scale: Sharded Discovery Services

If the number of users is large enough that a single discovery service (or
a single set of WS servers) can't hold all the connection state, you shard
it — e.g., partitioning users across DiscoveryService1, DiscoveryService2,
etc. Each shard registers itself as a subscriber to the relevant channels /
topics in a shared message broker (SQS/Redis/Kafka-like), so that a message
originating on one shard can still be routed to a user connected via a
different shard.

flowchart LR
    A[User A] --> DS1[Discovery Service 1]
    B[User B] --> DS1
    C[User C] --> DS2[Discovery Service 2]
    D[User D] --> DS2

    DS1 <--> MB[(Shared Message Broker<br/>topics = channels)]
    DS2 <--> MB

    style MB fill:#f9f0d9,stroke:#c9a227

Each discovery service shard subscribes to the broker on behalf of its own
users' channels, so a message published by a user on DiscoveryService1 can
still reach a subscriber whose connection lives on DiscoveryService2.

Putting It Together (End-to-End Flow)

Client loads the app → registers its WebSocket connection with the discovery service.
User A sends "Hi" to a channel → goes to the discovery service / message-fan-out layer.
It checks which channel members have active connections and pushes the message to the WS servers those connections live on.
In parallel, the message is sent to a queue → picked up by a worker → persisted to the DB.
Any client without an active connection simply reads the message from the DB the next time they open the channel.

Afterthoughts / Future Improvements

These were intentionally left out of the core design to keep it focused, but
are worth calling out as natural next steps:

Multi-device support. Users are often connected from more than one
device at once (phone + laptop). The discovery mapping would need to go
from user_id → single connection to user_id → set of connections, and
delivery/read-state would need to account for multiple active sessions per
user.
Strict message ordering guarantees. With multiple WS servers and a
pub/sub fan-out layer in between, messages sent close together from
different servers aren't guaranteed to arrive or be persisted in send
order. A more complete design would assign each channel's messages a
monotonically increasing sequence number at write time (e.g., by
serializing writes per channel, or using a partitioned log like Kafka
keyed by channel_id), and have clients render/order by that sequence
number rather than by arrival time.
Offline push notifications for users with no active connection at all.
Pagination strategy for scrolling through history — cursor-based
pagination on (channel_id, created_at) rather than naive SELECT *.
Presence and typing indicators, which piggyback on the same connection
infrastructure but weren't part of the core requirements here.

Designing a User Online/Offline Presence System: From Naive Polling to Redis TTLs

aditya emmadishetty — Thu, 23 Jul 2026 11:32:35 +0000

External Github link for diagrams

How I designed a presence system from scratch — including the mistakes I nearly shipped

Every chat app, social network, and collaboration tool has that little green dot next to a user's name. It looks trivial. It is not. In this post I'll walk through how I designed a presence (online/offline) system for a web app — starting from the naive approach, working through why it breaks at scale, and landing on a design that's actually used in production systems at places like Slack and Discord.

If you've ever wondered "why does this need more than a boolean column in the users table," this is for you.

The Naive Approach: A Boolean in the Database

The first instinct is simple: add an is_online column to your users table, set it to true on login, false on logout.

flowchart LR
    A[User] -->|login| B[Login Service]
    B -->|UPDATE users SET is_online=true| C[(Database)]
    A -->|logout| B
    B -->|UPDATE users SET is_online=false| C

This immediately falls apart for one obvious reason: users don't log out. They close the tab, their laptop dies, their WiFi drops. Your database now has a permanent record of a user being "online" who logged off three days ago.

Attempt Two: Client-Side Heartbeat Polling

The fix is a heartbeat: a script on the client pings the server every N seconds. If the server has heard from the user recently, they're online.

sequenceDiagram
    participant Client
    participant Server
    participant DB
    loop every 30 seconds
        Client->>Server: POST /heartbeat
        Server->>DB: UPDATE user_activity SET last_active_at = now()
    end

To check if someone is online, you query:

SELECT * FROM user_activity
WHERE last_active_at > NOW() - INTERVAL '1 minute'

This works — but it introduces a new problem: you're now writing to your primary database on every heartbeat, from every connected user, every 30 seconds.

For a modest 10,000 concurrent users, that's over 300 writes per second hitting your primary DB, forever, just to track a boolean-ish state. This is the kind of load that starts showing up in your slow query logs and your DBA's Slack messages.

The questions worth asking at this point

What happens if the database goes down? Does presence tracking take core traffic down with it?
How many concurrent connections/requests can the database realistically absorb?
Do we actually need this data to be durable at all?

That last question is the key insight that unlocks the next design.

The Insight: Presence Data Doesn't Need to Be Durable

Presence is fundamentally ephemeral state. Nobody cares if you lose a user's online status during a server restart — they'll just send another heartbeat in 30 seconds and it'll self-heal. This means presence doesn't belong in your primary transactional database at all. It belongs in an in-memory store.

Enter Redis.

flowchart LR
    A[User] -->|heartbeat| B[Server]
    B -->|SET presence:id EX 30| C[(Redis)]
    B -.->|no impact| D[(Primary DB)]
    style D stroke-dasharray: 5 5

The trick isn't just "use a faster database" — it's using a native TTL (time-to-live) expiry instead of storing a timestamp and comparing it yourself:

SET user:123 1 EX 30

Now "is this user online" is just:

EXISTS user:123

No timestamp math, no WHERE last_active < NOW() - INTERVAL. If the key exists, they're online. If Redis silently expired it because 30 seconds passed without a heartbeat, they're offline. Redis does the work for you.

Why this design survives a Redis outage

Because presence data is disposable, if Redis goes down, you just spin up a fresh instance. There's no backup to restore, no data loss to worry about — the system self-heals as soon as clients send their next heartbeat.

Implementation

Here's the actual service, in Python with FastAPI and redis-py:

import redis
from datetime import timedelta

class PresenceService:
    def __init__(self, redis_client: redis.Redis):
        self.redis = redis_client
        self.ttl = timedelta(seconds=30)

    def mark_online(self, user_id: str) -> None:
        self.redis.set(f"presence:{user_id}", "1", ex=self.ttl)

    def is_online(self, user_id: str) -> bool:
        return self.redis.exists(f"presence:{user_id}") == 1

    def mark_offline(self, user_id: str) -> None:
        self.redis.delete(f"presence:{user_id}")

from fastapi import FastAPI, Depends

app = FastAPI()
presence_service = PresenceService(redis_client)

@app.post("/api/heartbeat")
def heartbeat(user_id: str = Depends(get_current_user_id)):
    presence_service.mark_online(user_id)
    return {"status": "ok"}

Note the get_current_user_id dependency — this pulls the user ID from the authenticated session, not from a request parameter. Otherwise anyone can call this endpoint and mark any user online, which is a spoofing vulnerability, not just a bug.

The Fan-Out Problem: Checking Many Users at Once

A single EXISTS call is fine for checking one user. But what happens when you load a chat app and need to show online status for 200 friends at once? Looping and calling Redis 200 times means 200 network round trips.

flowchart TB
    A[Client loads friend list] --> B[Friend Service]
    B --> C[(Primary DB<br/>list of friend ids)]
    B --> D[Chat/Presence Service]
    D -->|pipelined EXISTS x N| E[(Redis)]
    E -->|batched result| D
    D -->|green/gray dots| A

The fix is pipelining — batching all the EXISTS checks into a single round trip:

def are_online(self, user_ids: list[str], chunk_size: int = 500) -> dict[str, bool]:
    result = {}
    for i in range(0, len(user_ids), chunk_size):
        chunk = user_ids[i:i + chunk_size]
        pipe = self.redis.pipeline()
        for uid in chunk:
            pipe.exists(f"presence:{uid}")
        exists_flags = pipe.execute()
        result.update(dict(zip(chunk, map(bool, exists_flags))))
    return result

Chunking (e.g. 500 keys per pipeline) matters even though Redis has no hard cap on pipeline size — an unbounded pipeline for a user with tens of thousands of connections is a real memory and latency risk, both client-side and server-side.

The Real Fix: Stop Polling, Push Instead

Heartbeat polling every 30 seconds works, but it has two structural weaknesses:

It's wasteful. Every connected client is sending a request every 30 seconds whether or not anything changed.
It's slow to detect disconnects. If a user closes their laptop, you won't know they're offline until their TTL expires — up to 30 seconds of showing a stale "online" status.

The better design — and where systems like Slack and Discord actually land — is to tie presence directly to a persistent connection, like a WebSocket:

sequenceDiagram
    participant Client
    participant WSGateway as WebSocket Gateway
    participant Redis

    Client->>WSGateway: connect
    WSGateway->>Redis: SET presence:id (no expiry needed)
    Note over Client,WSGateway: connection stays open

    Client--xWSGateway: disconnect (tab closed / network drop)
    WSGateway->>Redis: DEL presence:id
    Note over Redis: instant offline detection

The connection's lifecycle is the presence signal. Connect → online. Disconnect → offline. No polling loop, no 30-second detection lag. The TTL-based Redis key still has value here as a safety net — if a disconnect event is ever missed (e.g. gateway crash), the TTL ensures the key eventually expires instead of leaving a permanently stale "online" status.

Publishing Presence Changes to Friends

The last piece is getting presence updates to the people who need to see them, without broadcasting every status change to everyone.

flowchart LR
    A[User goes online/offline] --> B[Presence Service]
    B -->|publish event| C[Pub/Sub - Redis or Kafka]
    C -->|only to subscribed viewers| D[Friends currently viewing this user's status]

The naive approach — re-fetching every friend's status on a timer, or broadcasting every change to every friend regardless of whether they're looking — doesn't scale once friend lists get large. The scalable version: publish a change event, and only push it to clients that are actively subscribed to that specific user's presence (i.e., have them visible on screen right now).

Architecture Evolution, Summarized

Stage	Approach	Problem
1	Boolean column in `users` table	Never goes false on disconnect
2	Heartbeat + `last_active_at` timestamp in DB	Crushes primary DB with write load
3	Heartbeat + Redis with TTL	Still polling; 30s detection lag
4	WebSocket connection lifecycle + Redis TTL as safety net	Production-grade
5	+ Pub/sub fan-out to active viewers only	Scales to large friend graphs

Closing Thoughts

What looks like "just a green dot" touches almost every hard problem in distributed systems: ephemeral vs. durable state, polling vs. push, fan-out at scale, and graceful degradation. Working through the naive version first — and feeling why it breaks — is what makes the Redis TTL design click, rather than just being a pattern you memorized.

This is the first in a series where I'm documenting real system design problems as I work through them — from scratch notes to working code. If you found this useful, I'm exploring turning this into a full course on building production-ready systems for backend engineers. Follow along for the next post, where I'll cover the WebSocket gateway implementation in detail.