Enterprise chat architecture is the set of backend systems that move messages between users reliably, in order, and fast: a real-time connection layer for live clients, a durable event pipeline for ordering and delivery, tiered storage for history and media, and a presence layer for who's online. To keep this concrete, the examples here come from a real system instead of a generic diagram: CometChat's architecture, which runs across 113,000+ apps in 163 countries at a 99.999% uptime SLA. CometChat is a chat and messaging platform for developers, and working from an actual stack means naming real databases and real numbers, not vague claims about ‘scalable infrastructure.’
Get those four systems right and chat feels instant. Get one wrong and it feels instant right up until real load arrives. This is the engineering piece, not the buyer piece: if you're an architect or senior engineer deciding whether to build in-app chat or buy it, here's how the system actually works under the hood, and the parts most teams underestimate until they're debugging them at 2am
The four systems chat actually is
In-app chat is four coupled systems wearing one trench coat. Each has its own failure mode, and they fail on different nights.
Connection layer: persistent WebSocket connections, one per active client, carrying events in both directions.
Event pipeline: the durable log that decides ordering, guarantees delivery, and fans each message out to recipients.
Storage: where message history, metadata, and media live, each with a different access pattern.
Presence: who's online, typing, idle, or gone, updated in real time.
Build any one of these badly and you don't find out on launch day. You find out at scale, in production, in front of users who are not impressed.
Message delivery guarantees: what ‘delivered’ has to mean
A message delivery guarantee is the contract for whether a message can be lost, duplicated, or reordered when something fails. There are three flavors: at-most-once (fast, drops messages), at-least-once (safe, duplicates messages), and exactly-once (the one everyone wants on the whiteboard and nobody gets for free).
Real chat backends aim for effectively-once: idempotent producers, automatic retries, offline queuing, and sync across devices, so a flaky network produces one message on every device - not zero, and not three. CometChat's pipeline is built on exactly this set: effectively-once semantics using idempotent producers, retry logic, offline message queuing, and synchronization across a user's devices.
The happy path is not the problem. The problem is the client that dropped into a tunnel mid-send, reconnected four minutes later, and expects its message to land once, in the right order, on the phone and the laptop and the web tab it forgot to close. Ordering and deduplication under failure are the actual product. Everything else is UI.
This is why serious stacks put a real commit log in the middle. CometChat uses Apache Kafka as the event backbone: guaranteed message ordering, no message loss, sub-second latency, and configurable retention. The log is what lets you make promises about ordering that survive a retry without one, ‘in order’ is a hope, not a guarantee.
Presence: the feature that lies to you
Presence is the system that tracks who's online, typing, or idle, in real time, across every device. It sounds simple. It is not, and it tends to prove that in front of users.
Presence is high-frequency, low-value, and unbounded - the worst combination in distributed systems. Every keystroke can fire a typing event. Every reconnect can fire an online event. Open a group with 3,000 members and, done naively, you've signed up to push 3,000 presence updates to every one of them, continuously. That's not a chat feature. That's a self-inflicted DDoS with a green dot on it.
The fixes are unglamorous and mandatory: debounce typing events, batch presence changes, scope presence to the conversations a user is actually looking at, and treat ‘last seen’ as eventually consistent instead of real-time. CometChat handles presence and typing indicators over its WebSocket Gateway - persistent bi-directional connections with automatic reconnection and session recovery and exposes WebSocket-level control so you can define custom presence logic instead of accepting "is the socket open" as the answer. Because "is the socket open" is almost never the question you actually mean.
Fan-out at scale: where the math stops being cute
Fan-out is the work of turning one sent message into delivery events for every recipient's active connections. For 1:1 chat it's trivial. For a 10,000-member group where 3,000 people are online, each on two devices, one message becomes 6,000 delivery events (3,000 × 2) and something has to route all of them without tipping over.
Two things keep this sane. First, a connection layer that scales horizontally: CometChat runs on a stateless service design, so you add capacity by adding compute, not by re-architecting or migrating data. Second, the event log in the middle. With Kafka handling routing and ordering centrally, delivery is guaranteed once - not renegotiated per connection while 6,000 sockets wait.
Concrete numbers, since ‘highly scalable’ is a phrase and not a fact: CometChat's Docker Swarm tier is built for up to ~200,000 monthly active users and ~20,000 peak concurrent connections, at sub-100ms message latency. Past that - 200k+ MAU or multi-region - the deployment moves to Kubernetes. The whole platform is engineered from 10,000 to 250,000+ MAU with linear scaling. Those are the kind of numbers you want in a capacity plan, not adjectives.
Storage: one database is a lie you tell yourself early
Chat storage is polyglot by necessity. No single database is good at hot reads, durable history, media blobs, and rate-limit counters at the same time, and the classic early mistake is making one try. It works beautifully in staging and falls over the first quarter you have real history to query.
Here's how CometChat's on-premise stack splits the job:
| Layer | What it stores | Technology | Why this one |
|---|---|---|---|
| Distributed SQL | Messages, conversations, users — the source of truth | TiDB (PD + TiKV with Raft, MySQL-compatible SQL layer) | ACID transactions, automatic sharding, multi-region replication, strong consistency |
| Document store | Moderation policies, custom metadata, webhook config, audit logs | MongoDB | Flexible schema, native JSON, rich queries on semi-structured data |
| In-memory | Cache, sessions, auth tokens, rate-limit counters | Redis clusters (cache + session/rate-limit) | Sub-millisecond latency for the highest-frequency operations |
| Object storage | Files, images, large binary attachments | S3-compatible (S3, MinIO, Ceph, GCS) | Lifecycle policies, versioning, encryption at rest, tiered cost |
| Event log | Ordering and delivery backbone | Apache Kafka | Guaranteed ordering, no message loss, configurable retention |
The part people miss: message history grows forever and gets queried constantly. The database that felt fine at 10 million messages is a different animal at 10 billion, and by then you're not choosing a database - you're migrating one, live, while people are talking. Distributed SQL with automatic sharding exists so that day never arrives.
Data residency: the requirement that rewrites your topology
Data residency is the rule that a user's data must be stored and processed inside a specific legal jurisdiction. It is not a settings toggle. It's an architectural constraint that shows up before you write a line of code, because it dictates where your databases physically sit and how your regions replicate.
If you serve regulated industries or the EU, this decides your deployment model, not the other way around. CometChat supports the full range: managed cloud, private cloud in a specific region, hybrid, and fully air-gapped on-premise where data never leaves your infrastructure. It's built to meet GDPR, HIPAA, SOC 2, ISO 27001, CCPA, and PIPEDA, with encryption at rest, configurable key management, and secure WebSocket connections. Multi-tenancy gives you logical isolation with per-tenant configuration, rate limits, and quotas, which matters the moment one deployment serves several business units that legal insists cannot see each other's data.
The reframe worth keeping: residency isn't a feature you bolt on later. Retrofitting jurisdiction into a single-region system is a rebuild, not a config change. Decide it first.
Where it breaks at 2am
The four systems are the easy part to name and the hard part to keep alive. The failures that actually page you:
Reconnection storms. A region blips, thousands of clients reconnect at once, and your connection layer plus session store take the full spike simultaneously. Without session recovery and stateless gateways, this cascades.
Presence fan-out. The green dot that felt free in testing becomes the single largest source of traffic in production.
Group fan-out. Big groups turn one message into thousands of writes and pushes. Linear-looking code, quadratic-feeling bill.
Ordering under retry. Messages arrive, but out of order or twice, because the retry path didn't go through the log. Users notice instantly.
History queries. The read that was instant at launch times out once there's real backlog to scan.
Cross-region replication lag. Two users, two regions, one conversation, and a few seconds of disagreement about what was said.
None of these show up in a demo. All of them show up at scale. That's the whole reframe of operating chat instead of shipping it: the work isn't the feature, it's the years the feature has to keep working.
Build vs buy, without the sales pitch
Building this in-house is entirely possible - it's also months of work before your first message, and years of the failures above before it's boring. If chat is your core product and your differentiator, build it. If chat is a feature your product needs to have and keep working, the math usually points the other way.
Either way, now you know what the system is actually made of, and which parts to interrogate before you commit. If any of this resonated, that's the part worth getting right first.
CometChat's enterprise chat infrastructure, in one place
CometChat's enterprise infrastructure is the four systems above, already built and already surviving the 2am failures - a stateless WebSocket Gateway for connections and presence, an Apache Kafka backbone for ordered, no-loss delivery, polyglot storage (TiDB, Redis, MongoDB, S3-compatible object store) tuned per access pattern, and multi-tenancy with per-tenant isolation, rate limits, and quotas.
It runs where your compliance team needs it to: managed cloud, region-pinned private cloud, hybrid, or fully air-gapped on-premise where data never leaves your infrastructure - built to meet GDPR, HIPAA, SOC 2, ISO 27001, CCPA, and PIPEDA, with encryption at rest, configurable key management, and secure WebSocket connections. It scales linearly from 10,000 to 250,000+ monthly active users at sub-100ms message latency, on Docker Swarm up to ~200k MAU and Kubernetes beyond that.
The point isn't the feature list. It's that presence fan-out, ordering under retry, reconnection storms, and residency-driven topology are already handled - so your team ships the product and skips the years of learning which of these breaks first. Speed becomes sustainability. Shipping becomes operating. That's the whole trade.
Want to see the stack instead of read about it? Start for free and build against it, or contact sales if you're sizing a deployment for real load, data residency, or an on-premise rollout.
FAQ
What is enterprise chat architecture? It's the backend that delivers messages reliably, in order, and fast at scale - a real-time connection layer, a durable event pipeline for ordering and delivery, polyglot storage for history and media, and a presence layer. Enterprise adds data residency, compliance, and multi-tenancy on top.
What message delivery guarantee does chat need? Effectively-once: one message on every device, even after a dropped connection and retry. Pure exactly-once is expensive and rare, so real stacks combine idempotent producers, automatic retries, and offline queuing to reach effectively-once in practice.
How many concurrent connections can a chat backend handle? It depends on architecture, not vibes. CometChat's Docker Swarm deployment targets ~20,000 peak concurrent connections and ~200,000 MAU at sub-100ms latency; beyond that, Kubernetes handles 200k+ MAU and multi-region, scaling linearly to 250,000+ MAU.
What database should I use for chat? More than one. Distributed SQL (like TiDB) for the message source of truth, Redis for hot cache and sessions, a document store for metadata and audit logs, object storage for media, and a log like Kafka for ordering. No single database does all of that well.
Do I need Kafka for chat? You need a durable, ordered event log; Kafka is the common choice. It's what lets you guarantee ordering and no message loss centrally, so those promises survive retries and reconnects instead of being renegotiated per connection.
How do I meet data residency requirements for chat? Decide the deployment model before you build. Options are managed cloud, region-pinned private cloud, hybrid, and air-gapped on-premise. CometChat supports all four and is built to meet GDPR, HIPAA, SOC 2, ISO 27001, CCPA, and PIPEDA. Retrofitting residency into a single-region system is a rebuild, so choose first.
Top comments (0)