DEV Community

Cover image for Design a Scalable Chat Application Combine everything into one architecture
Tanu Priya
Tanu Priya

Posted on

Design a Scalable Chat Application Combine everything into one architecture

#architecture #backend #scalability #systemdesign

Building a chat application may seem simple at first.

A user sends a message, another user receives it, and the conversation continues.

But once your application grows from a few users to millions, the architecture becomes much more interesting.

You need to think about:

  • How do users connect in real time?
  • How are messages delivered?
  • Where are messages stored?
  • What happens when a server crashes?
  • How do you support millions of concurrent connections?
  • How do users receive messages when they're offline?
  • How do you monitor the entire system?

This is where all the concepts from system design come together.

In this article, we'll design a scalable chat application from requirements to a production-ready architecture.

Step 1: Understand the Requirements

Before choosing technologies, we need to understand what we're building.

Let's assume we're designing a WhatsApp- or Discord-like chat system.

Functional Requirements

Our application should support:

  • One-to-one messaging
  • Group chats
  • Real-time message delivery
  • Message history
  • Online/offline status
  • Read receipts
  • Typing indicators
  • Media messages
  • Push notifications

Non-Functional Requirements

The system should also provide:

  • Low latency
  • High availability
  • Scalability
  • Message durability
  • Fault tolerance
  • Observability

These requirements will guide our architecture.

Step 2: Estimate the Traffic

Let's assume:

10 Million Daily Active Users
1 Million Concurrent Users
100 Messages per User per Day
Enter fullscreen mode Exit fullscreen mode

That gives us:

10M × 100 = 1 Billion Messages / Day
Enter fullscreen mode Exit fullscreen mode

Average messages per second:

1,000,000,000 / 86,400

≈ 11,500 messages/second
Enter fullscreen mode Exit fullscreen mode

But traffic is not evenly distributed.

Peak traffic could be several times higher.

Let's design for:

50,000+ messages/second
Enter fullscreen mode Exit fullscreen mode

This tells us that a single server or database will eventually become a bottleneck.

Step 3: High-Level Architecture

A scalable chat system might look like this:

                         ┌──────────────┐
                         │    Users     │
                         └──────┬───────┘
                                │
                         WebSocket / HTTPS
                                │
                                ▼
                      ┌──────────────────┐
                      │  Load Balancer   │
                      └────────┬─────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        ┌───────────┐    ┌───────────┐    ┌───────────┐
        │ Chat Node │    │ Chat Node │    │ Chat Node │
        └─────┬─────┘    └─────┬─────┘    └─────┬─────┘
              │                │                │
              └────────────────┼────────────────┘
                               ▼
                        ┌─────────────┐
                        │ Message Bus │
                        │ Queue/PubSub│
                        └──────┬──────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        Message Store      Presence         Notifications
              │             Service            Service
              ▼
         Chat Database
Enter fullscreen mode Exit fullscreen mode

Each component has a specific responsibility.

This separation makes the system easier to scale independently.

Step 4: Real-Time Communication

Chat applications need a persistent connection between the client and server.

Using traditional HTTP polling would create unnecessary requests.

A better approach is:

Client
   │
   │ WebSocket Connection
   │
   ▼
Chat Server
Enter fullscreen mode Exit fullscreen mode

Once connected, either side can send data instantly.

User A ─── Message ───→ Chat Server
                              │
                              ▼
                       User B receives it
Enter fullscreen mode Exit fullscreen mode

WebSockets are useful for:

  • New messages
  • Typing indicators
  • Read receipts
  • Online presence
  • Live updates

Step 5: Scaling WebSocket Connections

A single chat server cannot maintain millions of connections.

Instead:

                      Load Balancer
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
          Chat Node 1    Chat Node 2    Chat Node 3
Enter fullscreen mode Exit fullscreen mode

Each node manages a subset of active connections.

For example:

Node 1 → 100,000 Connections
Node 2 → 100,000 Connections
Node 3 → 100,000 Connections
Enter fullscreen mode Exit fullscreen mode

As users grow:

More Users
     │
     ▼
Add More Chat Nodes
Enter fullscreen mode Exit fullscreen mode

This provides horizontal scalability.

The load balancer can route new WebSocket connections to healthy nodes.

Step 6: Sending a Message

Let's follow a message from User A to User B.

User A
  │
  │ "Hello!"
  ▼
Chat Server
Enter fullscreen mode Exit fullscreen mode

The server first validates:

  • Is the user authenticated?
  • Is the sender allowed in this conversation?
  • Is the message valid?

Then the message enters the processing flow.

Client
   │
   ▼
Chat Gateway
   │
   ▼
Authentication
   │
   ▼
Message Service
Enter fullscreen mode Exit fullscreen mode

A message might look like:

{
  "messageId": "msg_123",
  "conversationId": "conv_456",
  "senderId": "user_A",
  "content": "Hello!",
  "timestamp": "2026-08-23T12:00:00Z"
}
Enter fullscreen mode Exit fullscreen mode

The messageId is important for deduplication and idempotency.

Step 7: Store the Message First

A message should not be considered successfully sent until the system can reliably persist it.

User A
   │
   ▼
Message Service
   │
   ▼
Message Database
   │
   ▼
Message Stored ✅
Enter fullscreen mode Exit fullscreen mode

After successful storage:

Message Service
       │
       ├──→ Deliver to recipient
       │
       ├──→ Update conversation
       │
       └──→ Trigger notifications
Enter fullscreen mode Exit fullscreen mode

This improves durability.

If a chat server crashes after the message is stored, the message is not lost.

Step 8: Delivering Messages in Real Time

Now we need to find User B.

What if User B is connected to a different server?

User A
   │
   ▼
Chat Node 1
   │
   │ Message for User B
   ▼
        ?
   │
   ▼
Chat Node 3
   │
   ▼
User B
Enter fullscreen mode Exit fullscreen mode

The system needs a way for chat servers to communicate.

This is where a message broker or Pub/Sub system can help.

                  Message Broker
                  /      |      \
                 ▼       ▼       ▼
              Node 1   Node 2   Node 3
Enter fullscreen mode Exit fullscreen mode

The message can be published to the appropriate channel.

The node holding User B's connection delivers it.

User A
   │
   ▼
Node 1
   │
   ▼
Message Broker
   │
   ▼
Node 3
   │
   ▼
User B
Enter fullscreen mode Exit fullscreen mode

Step 9: Managing User Presence

Presence tells us whether a user is:

  • Online
  • Offline
  • Away
  • Last seen recently

A fast in-memory store can manage this information.

user_123 → Chat Node 2
user_456 → Chat Node 3
Enter fullscreen mode Exit fullscreen mode

When a user connects:

user_123 → ONLINE
Enter fullscreen mode Exit fullscreen mode

When the connection closes:

user_123 → OFFLINE
Enter fullscreen mode Exit fullscreen mode

A distributed presence store allows any chat server to find the user's active connection.

Step 10: What If the User Is Offline?

If User B is offline:

User A
   │
   ▼
Message Service
   │
   ├── Message Database ✅
   │
   └── User B Offline
Enter fullscreen mode Exit fullscreen mode

The message remains stored.

The system can then send a push notification.

Notification Service
        │
        ▼
   Push Provider
        │
        ▼
     User B 📱
Enter fullscreen mode Exit fullscreen mode

When User B reconnects:

User B
   │
   ▼
Chat Service
   │
   ▼
Fetch Unread Messages
   │
   ▼
Display Conversation
Enter fullscreen mode Exit fullscreen mode

This ensures offline users don't lose messages.

Step 11: Supporting Group Chats

Group chats introduce another challenge.

Imagine a group with:

10,000 Members
Enter fullscreen mode Exit fullscreen mode

If one user sends a message, how should it be delivered?

There are two common approaches.

Fan-Out on Write

When a message is sent:

Message
   │
   ├──→ User 1 Inbox
   ├──→ User 2 Inbox
   ├──→ User 3 Inbox
   └──→ Thousands More
Enter fullscreen mode Exit fullscreen mode

Advantages

  • Fast reads
  • Easy message retrieval

Disadvantages

  • Expensive writes for large groups

Fan-Out on Read

Store the message once:

Group Message Store
        │
        ▼
Members read when needed
Enter fullscreen mode Exit fullscreen mode

Advantages

  • Efficient writes
  • Better for very large groups

Disadvantages

  • More expensive reads

A real system may use a hybrid approach.

Step 12: Database Design

Chat applications generate massive amounts of data.

A simple message table might look like:

Messages
------------------------------------
message_id
conversation_id
sender_id
content
created_at
status
Enter fullscreen mode Exit fullscreen mode

A useful query is:

SELECT *
FROM messages
WHERE conversation_id = ?
ORDER BY created_at DESC
LIMIT 50;
Enter fullscreen mode Exit fullscreen mode

This suggests that messages should be efficiently organized around:

conversation_id + timestamp
Enter fullscreen mode Exit fullscreen mode

This supports fast retrieval of recent messages.

Step 13: Database Partitioning and Sharding

As the number of messages grows, a single database may become too large.

We can partition messages:

Messages
   │
   ├── Partition 1
   ├── Partition 2
   └── Partition 3
Enter fullscreen mode Exit fullscreen mode

Eventually, we may shard data across multiple database servers.

For example:

hash(conversation_id) % 4
Enter fullscreen mode Exit fullscreen mode 
Conversation A → Shard 1
Conversation B → Shard 2
Conversation C → Shard 3
Conversation D → Shard 4
Enter fullscreen mode Exit fullscreen mode

Using conversation_id as a shard key keeps most messages from the same conversation together.

This makes message retrieval easier.

Step 14: Adding Caching

Some data is accessed frequently.

For example:

  • User profiles
  • Conversation metadata
  • Active conversations
  • Presence information

Instead of querying the database every time:

Application
     │
     ▼
    Cache
     │
     ├── Hit → Return Data ⚡
     │
     └── Miss → Database
Enter fullscreen mode Exit fullscreen mode

Caching reduces:

  • Database load
  • Latency
  • Infrastructure cost

But the database remains the source of truth.

Step 15: Reliability and Fault Tolerance

Failures are inevitable.

Suppose one chat node crashes:

Chat Node 2 ❌
Enter fullscreen mode Exit fullscreen mode

The load balancer should stop routing new connections to it.

Load Balancer
     │
     ├── Node 1 ✅
     ├── Node 2 ❌
     └── Node 3 ✅
Enter fullscreen mode Exit fullscreen mode

Users connected to the failed node reconnect to healthy servers.

For critical components, we need redundancy.

                  Load Balancer
                         │
            ┌────────────┼────────────┐
            ▼            ▼            ▼
         Node 1        Node 2        Node 3
Enter fullscreen mode Exit fullscreen mode

We also need:

  • Health checks
  • Timeouts
  • Retries
  • Circuit breakers
  • Database replicas
  • Automatic failover

The goal is to minimize the impact of individual failures.

Step 16: Handling Duplicate Messages

Distributed systems can deliver the same message more than once.

For example:

Message Sent
    │
Network Timeout
    │
Client Doesn't Know if Server Received It
    │
Client Retries
Enter fullscreen mode Exit fullscreen mode

Without protection:

Hello!
Hello!
Enter fullscreen mode Exit fullscreen mode

To prevent this, clients can generate a unique messageId.

messageId = msg_123
Enter fullscreen mode Exit fullscreen mode

If the server receives the same message again:

msg_123 already processed

→ Ignore duplicate
Enter fullscreen mode Exit fullscreen mode

This is called idempotency.

It is essential for reliable messaging.

Message Ordering

Users expect:

Hello
How are you?
Enter fullscreen mode Exit fullscreen mode

Not:

How are you?
Hello
Enter fullscreen mode Exit fullscreen mode

Ordering becomes difficult when multiple servers process messages.

A practical approach is to assign a sequence number within each conversation.

Conversation A

Message 1 → Sequence 101
Message 2 → Sequence 102
Message 3 → Sequence 103
Enter fullscreen mode Exit fullscreen mode

Clients can then display messages in the correct order.

Step 17: Security and Authentication

Every connection must be authenticated.

A common flow:

User Login
    │
    ▼
Authentication Service
    │
    ▼
Access Token
    │
    ▼
WebSocket Connection
Enter fullscreen mode Exit fullscreen mode

The server validates the token before allowing the user to connect.

The system should also verify:

  • Is the sender part of the conversation?
  • Does the user have permission to access this group?
  • Is the message within allowed limits?

Security should be enforced on the server—not trusted to the client.

Step 18: Observability

With multiple services and servers, debugging becomes difficult without visibility.

We need:

Logs 

Message received
Message stored
Message delivered
Delivery failed
Enter fullscreen mode Exit fullscreen mode

Metrics

Track:

  • Messages per second
  • Delivery latency
  • Active WebSocket connections
  • Error rate
  • Database latency
  • Queue size

Traces

Follow a message across the system:

Client
   │
   ▼
Chat Gateway
   │
   ▼
Message Service
   │
   ▼
Database
   │
   ▼
Message Broker
   │
   ▼
Recipient Chat Node
Enter fullscreen mode Exit fullscreen mode

This helps identify bottlenecks quickly.

Putting Everything Together

Now we can combine everything into one architecture.

                              USERS
                                │
                    WebSocket / HTTPS
                                │
                                ▼
                        LOAD BALANCER
                                │
             ┌──────────────────┼──────────────────┐
             ▼                  ▼                  ▼
         CHAT NODE 1        CHAT NODE 2        CHAT NODE 3
             │                  │                  │
             └──────────────────┼──────────────────┘
                                │
                     ┌──────────┴──────────┐
                     ▼                     ▼
              PRESENCE SERVICE       MESSAGE SERVICE
                     │                     │
                     ▼                     ▼
               IN-MEMORY STORE      MESSAGE BROKER
                                           │
                          ┌────────────────┼───────────────┐
                          ▼                ▼               ▼
                     MESSAGE DB      NOTIFICATIONS      DELIVERY
                          │
                    ┌─────┴─────┐
                    ▼           ▼
                 REPLICA     REPLICA
Enter fullscreen mode Exit fullscreen mode

Across the entire architecture:

                    OBSERVABILITY LAYER

        Logs ──────┐
        Metrics ───┼──→ Dashboards + Alerts
        Traces ────┘
Enter fullscreen mode Exit fullscreen mode

And reliability mechanisms protect the system:

Redundancy
Retries
Timeouts
Circuit Breakers
Failover
Health Checks
Enter fullscreen mode Exit fullscreen mode

Complete Message Flow

Let's follow one message end to end.

1. User A sends a message 

User A → WebSocket → Chat Node
Enter fullscreen mode Exit fullscreen mode

2. The server authenticates and validates it 

Token Valid? ✅
Conversation Access? ✅
Message Valid? ✅
Enter fullscreen mode Exit fullscreen mode

3. The message is stored 

Message Service → Database
Enter fullscreen mode Exit fullscreen mode

4. The message event is published 

Database
   │
   ▼
Message Broker
Enter fullscreen mode Exit fullscreen mode

5. Find User B 

Presence Store
   │
   ▼
User B → Chat Node 3
Enter fullscreen mode Exit fullscreen mode

6. Deliver in real time 

Message Broker
   │
   ▼
Chat Node 3
   │
   ▼
User B 📱
Enter fullscreen mode Exit fullscreen mode

7. If User B is offline 

Message Stored ✅
        │
        ▼
Push Notification
Enter fullscreen mode Exit fullscreen mode

8. Track the entire process 

Logs + Metrics + Traces
Enter fullscreen mode Exit fullscreen mode

This is the complete journey of a message.

How Does This System Scale?

As usage grows, individual components can scale independently.

More Connections?
→ Add Chat Nodes

More Messages?
→ Add Message Processing Workers

More Database Load?
→ Add Replicas / Partition / Shard

More Cache Load?
→ Expand Cache Cluster

More Notifications?
→ Add Notification Workers
Enter fullscreen mode Exit fullscreen mode

This is one of the biggest advantages of separating responsibilities.

Scale the bottleneck, not the entire system.

Common Trade-Offs

There is no perfect architecture.

Every decision involves trade-offs.

Decision Benefit Trade-Off
WebSockets Real-time communication Connection management complexity
Message Broker Decouples services Additional infrastructure
Caching Lower latency Cache consistency
Sharding Horizontal database scale Operational complexity
Replication Higher availability Replication lag
Retries Handles temporary failures Can cause retry storms
Fan-out on write Fast reads Expensive for large groups
Fan-out on read Efficient writes Slower reads

Good system design is about understanding these trade-offs.

Final Takeaways

Designing a scalable chat application brings together nearly every major system design concept.

You start with requirements and traffic estimation.

Then you design for:

  • WebSockets for real-time communication
  • Load balancing for scalable connections
  • Message brokers for cross-server delivery
  • Databases for durable message storage
  • Caching for low-latency access
  • Partitioning and sharding for massive datasets
  • Redundancy and failover for reliability
  • Retries and idempotency for safe message delivery
  • Logs, metrics, and traces for observability

The final architecture is not about using every technology possible.

It's about combining the right components to build a system that is real-time, scalable, reliable, and easy to operate.

The complete system design mindset: 

Requirements
     ↓
Estimate Scale
     ↓
Choose Architecture
     ↓
Identify Bottlenecks
     ↓
Add Caching
     ↓
Scale Components
     ↓
Design for Failure
     ↓
Add Observability
     ↓
Continuously Improve
Enter fullscreen mode Exit fullscreen mode

*And that's how individual system design concepts come together to build a scalable real-world application. *

Top comments (0)