DEV Community: Ably

Why your AI chat reconnects but your session doesn't

Maddy Quinn — Wed, 27 May 2026 10:05:10 +0000

TL;DR: WebSockets are the right protocol for production AI chat. But the connection is stateless at the session level. When it drops — AWS ALB defaults to 60 seconds, Cloudflare to 100 seconds on Free and Pro plans — all in-flight tokens, tool call results, and agent context disappear. Reconnection logic restores the socket. It doesn't restore the session. That's the gap this post covers.

WebSockets are the right protocol for production AI chat. But that fact doesn't prevent the failure most teams hit first. An enterprise load balancer closes the idle connection at 60 seconds during a tool execution wait. Your reconnect logic fires in under a second, the agent keeps running server-side, and the client receives nothing from the gap. No tokens, no tool call results, no context.

The reconnected socket has no view of what happened while it was down. Three conditions cause this routinely: a proxy timeout mid-task, a page reload mid-generation, and a mobile network handoff. Each breaks for the same underlying reason: the WebSocket protocol handles transport, not session state, and reconnection logic doesn't change that.

Key takeaways

WebSockets are the right protocol for production AI chat: bidirectional, persistent, and suited to live steering and tool calls in ways SSE isn't.
A WebSocket connection is stateless at the session level. When it closes through a proxy timeout, page reload, or device switch, all state disappears with it.
Reconnection logic re-establishes the transport. It does not recover the tokens, tool calls, or agent context in flight when the connection is dropped.
What fills the gap is a session layer: infrastructure that persists conversation state against a session ID and replays it to reconnecting clients.

What WebSockets get right for AI chat

The protocol question is worth settling early, because the rest of this piece argues about the infrastructure layer above it. For production AI chat, the choice is WebSockets or SSE. Both stream tokens to the client, but only WebSockets let signals flow the other way.

WebSockets are bidirectional. When your user cancels mid-stream, that signal travels back on the same channel; tool call confirmations and workflow approvals work the same way. When a workflow pauses for human input mid-execution, that input must arrive in-band, not via a polling endpoint.

SSE is a one-way stream. For simple chatbots on stable networks, that doesn't matter. Add tool calls, mid-stream cancellation, or multi-device continuity, and it does.

Where production AI connections actually fail

Not all connection drops come from bad network conditions. The more common causes in production are infrastructure defaults designed for HTTP requests, not AI chat. A response can be mid-generation for tens of seconds, and most defaults weren't built for that.

AWS Application Load Balancer idle timeout. AWS ALB closes connections idle for 60 seconds by default, per the AWS Application Load Balancer documentation. For standard HTTP that's generous. For an agent waiting on a downstream API, 60 seconds of silence is routine, and the connection closes without warning. Your user's response stops mid-sentence with no explanation.

Cloudflare proxy timeout. On Cloudflare Free and Pro plans, WebSocket connections terminate after 100 seconds of inactivity, as documented in Cloudflare's WebSocket troubleshooting guide. Enterprise plans can raise this limit; on Free and Pro plans, the ceiling is fixed.

Mobile network handoffs. Switching from WiFi to cellular drops the underlying TCP connection immediately, taking the WebSocket with it. On mobile this happens during normal use: walking between coverage areas, backgrounding the tab, entering a building.

Page reload and tab crash. Your user reloads mid-generation, or the browser crashes, both of which are routine. The connection closes, and any session state tied to it is gone unless something stored it.

Why reconnection logic doesn't fix the session problem

The standard reconnection pattern re-establishes the socket. Transport recovers in milliseconds. But it cannot restore the state that was in flight when the connection dropped.

Token stream position. The response kept generating while the connection was dark. Those tokens went nowhere. When the client reconnects, it arrives mid-sentence or finds nothing at all.

Tool call results. Some chat responses depend on realtime data: a lookup, a search, or an action your user triggered. If the connection dropped while the agent was waiting for that result, the response either never came — or ended before it could use the information.

Agent context. In a multi-turn exchange, the agent accumulates context: what was asked, what was answered, and what's in progress. When a session drops and reconnects without state recovery, the agent and the client are at different points in the same conversation. Your users experience this as a loss of thread: a response that ignores what came before, or one that repeats something already answered.

The pattern most teams reach for is a Redis buffer: sequence number tracking, offset storage, and deduplication keys between the agent and the client. It handles full page reloads. It tends to break on deploy-triggered reconnects, mobile handoffs that hit the reconnect window twice, and anything that generates messages faster than the buffer drains.

Even Vercel's AI SDK lead built a pluggable interface to fill this gap. Every team reaching this point builds the same infrastructure from scratch and chooses to own it indefinitely. Reconnection handles the protocol layer; session state sits one layer above it, and it's a separate problem entirely.

What production AI chat needs from the transport layer

Any viable approach to production AI sessions needs to satisfy four requirements. These are implementation-neutral: what any infrastructure option has to provide, regardless of vendor.

Persistent state storage. Conversation history, token positions, tool call inputs and outputs, and agent state must be stored against a stable session ID and survive connection drops. The session ID is the anchor: the same session must be addressable after any reconnect, from any device.

Offset-based replay. A returning client requests messages from its last received serial. The infrastructure delivers everything missed, in order, with no duplicates. The client supplies its offset; the infrastructure fills the gap.

Protocol fallback. When a WebSocket upgrade is blocked by a proxy or firewall, the transport degrades to HTTP streaming or long-polling automatically. This should not require per-deployment configuration.

Multi-device delivery. Any authenticated device subscribing to a session ID receives the current state plus history. The session is not bound to the tab, browser, or device that opened it.

How Ably AI Transport solves the session layer problem

Thankfully, you don't need to build the infrastructure. Ably AI Transport is the durable session layer — the thing that makes the user experience survive what the WebSocket protocol cannot. The session lives in Ably; your application talks to it.

The five failures raised in this post each map directly to a capability:

Connection drops from proxy timeouts, mobile handoffs, and page reloads. The transport degrades automatically — WebSocket first, then HTTP streaming, then long-polling — so the session survives the infrastructure defaults that break standard WebSocket connections. No per-deployment configuration required.
→ Reconnection and recovery

Tokens generated while the client was disconnected. The token stream is stored against the session. On reconnect, the client receives everything it missed in order, with no duplicates. The developer doesn't track offsets or implement catch-up logic.
→ Token streaming

Tool call results and agent context lost mid-task. Agent state, tool call inputs and outputs, and conversation history are all published to the session as they generate. A reconnecting client recovers the full context, not just the tokens.
→ Reconnection and recovery

Mid-stream steering and human-in-the-loop signals. Cancellations, approvals, and human input travel back to the agent on the same session channel. The bidirectional requirement that rules out SSE is covered without a separate signaling mechanism.
→ Human in the loop

Sessions tied to a single tab or device. Any authenticated device subscribing to the session ID receives current state plus history. A conversation started on desktop continues on mobile without restart.
→ Multi-device sessions

Get started: Vercel AI SDK · Core SDK

Frequently asked questions

When is SSE still the right choice for AI chat?

SSE is a reasonable starting point for chatbots that follow a simple request-response pattern: a user submits a message, the server streams tokens, no interruption required. It deploys more easily than WebSockets, carries no persistent connection overhead, and works well on stable networks.

The constraints appear when your application starts adding agentic behaviour: tool calls, mid-stream cancellation, multi-device continuity, and background tasks that complete while the user is offline. At that point, SSE's unidirectional architecture stops being a trade-off and becomes a blocker.

What timeout values should I configure to prevent AI connection drops in production?

Set your AWS ALB idle timeout to at least 3,600 seconds for WebSocket connections. The 60-second default was designed for HTTP requests, not long-running agent tasks. On Cloudflare Free and Pro plans, the WebSocket timeout is fixed at 100 seconds. Send heartbeat pings at around 25-second intervals to stay well below that threshold.

For Nginx, the equivalent setting is proxy_read_timeout. These three changes cover most production timeout failures for AI chat deployments.

Does reconnection logic solve the session recovery problem?

Reconnection logic solves the transport problem. It doesn't solve the state problem. Exponential backoff and heartbeats re-establish the socket.

But they can't recover tokens generated during the gap, tool call results that arrived while the client was disconnected, or context accumulated across multiple steps. Preventing duplicate messages on reconnect requires sequence numbers or idempotency keys at the session layer, not the WebSocket layer. A client that reconnects without a session layer arrives at an empty context and either loses the conversation or restarts it.

How does Ably replay missed messages after a WebSocket reconnect?

Ably assigns every published message a serial number. When a client reconnects, the transport layer uses the internal untilAttach mechanism to fetch messages published during the gap. This bounds the history query to the exact reconnection point.

Ably delivers everything missed in order, with no overlap between historical and live messages. The client doesn't track its own offset or implement catch-up logic. Every plan includes two minutes of ephemeral history by default. Persisted channels extend this to 72 hours on Standard plans, or up to 365 days on Pro and Enterprise plans.

Have you hit this in production? Curious what the failure looked like - was it the proxy timeout, a page reload, or something else that first surfaced it?

Ably AI Transport is now available

Ably Blog — Tue, 20 Jan 2026 17:49:29 +0000

Today we’re launching Ably AI Transport: a drop-in realtime delivery and session layer that sits between agents and devices, so AI experiences stay continuous across refreshes, reconnects, and device switches — without an architecture rewrite.

The gap: HTTP streaming breaks down for stateful AI UX

AI has moved from “type and wait” requests to experiences that are long-running and stateful: responses stream, users steer mid-flight, and work needs to carry across tabs and devices. That shift changes what “working” means in production. It’s not just whether the model can generate tokens, it’s whether the experience stays continuous when real users behave like real users do.

Most AI apps still start with a connection-oriented setup: the client opens a streaming connection (SSE, fetch streaming, sometimes WebSockets), the agent generates tokens, and the UI renders them as they arrive. It’s low friction and demos well.

But HTTP streaming really solves only the first part of the problem, and it’s not a good place to end.

First: continuity. When output is tied to a specific connection, the experience becomes fragile by default. Refreshes, network changes, backgrounding, multiple tabs, device switches, agent handovers (even agent crashes) are normal behaviour. And they’re exactly where teams see partial output, missing tokens, duplicated messages, drifting state, and “start again” recovery paths. That’s where user trust gets lost.

Second: capability. A connection-first transport layer doesn’t just make UX fragile. It limits what you can build. Once you want true collaborative patterns like barge-in, live steering, copilot-style bidirectional exchange, multi-agent coordination, or a seamless human takeover with full context, you need more than “a stream.” You need a stateful conversation layer that can support multiple participants, resumable delivery, and shared session state.

So teams patch it: buffering, replay, offsets, reconnection logic, session IDs, routing rules for interrupts and tool results, multi-subscriber consistency, and observability once production incidents start. It’s critical work — but it’s not differentiation.

What Ably AI Transport does

AI Transport gives each AI conversation a durable bi-directional session that isn’t tied to one tab, connection or agent. Agents publish output into a session channel, clients subscribe from any device, and Ably handles the delivery guarantees you’d otherwise rebuild yourself: ordered delivery, recovery after reconnects, and fan-out to multiple subscribers.

It’s deliberately model and framework-agnostic. You keep your agent runtime and orchestration. AI Transport handles the delivery and session layer underneath.

The key shift: sessions become channels

In a connection-oriented setup, the “session” effectively lives inside the streaming pipe. When the pipe breaks, continuity becomes a headache.

With AI Transport, the session is created once and represented as a durable channel. Agents and clients can join independently. Refresh becomes reattach and hydrate. Device switching becomes another subscriber joining the same session. Multi-device behaviour becomes fan-out rather than custom routing. Agents and humans become truly connected over a transport designed for AI bi-directional low latency conversations.

How Ably AI Transport ensures a resilient, stateful AI UX

Resumable, ordered token streaming: A great AI UX depends on durable streaming. Output is treated as session data, so clients can catch up cleanly after refreshes, brief dropouts, and network handoffs.

Multi-device continuity: Conversations are user-scoped, not tab-scoped. Multiple clients can join the same session without split threads, duplication, or drifting state.

Live steering and interruption: Modern AI UX needs control, not just output. Interrupts, redirects, and approvals route through the same bi-directional session fabric as the response stream, so steering works even across reconnects and devices.

Presence-aware sessions: Once agents do real work, wasted compute becomes a serious cost problem. Presence provides a reliable signal for whether the user is currently connected (or fully offline across devices), so you can throttle, defer, or resume work accordingly.

Agents that collaborate and act with awareness: As soon as you have more than one agent (or an agent plus tools/workers), coordination becomes the product. Shared session state and routing prevent clashing replies, duplicated context, and “two brains answering at once,” so multiple agents can communicate directly with users coherently.

Seamless human takeover when it really matters: When an agent hits a boundary (risk, uncertainty, or policy) a human should be able to step in with full context and continue the session immediately. The handoff keeps the same session history and controls, so there’s no repeated questions, no “start again,” and no losing track of what happened mid-flight.

Identity and access control: Beyond toy demos, you need to know who can read, write, steer, or approve actions. Verified identity plus fine-grained permissions let multi-party sessions stay secure without inventing a bespoke access model.

Observability and governance: When AI UX breaks in production, it’s rarely obvious where. Built-in visibility into session delivery and continuity makes failures diagnosable and auditable instead of “black box streaming incidents.”

Concrete examples

Multi-device copilots: A user starts a long-running answer on desktop, switches to mobile mid-response, and the session continues without restarting. Steering and approvals apply to the same session regardless of device.

Long-running agents: A research agent runs multi-step tool work for minutes. If the user disconnects, the work continues; when the user returns, the client hydrates from session history instead of resetting.

Getting started (low friction)

You can get a basic session running in minutes:


js
import Ably from 'ably';

// Initialize Ably Realtime client
const realtime = new Ably.Realtime({ key: 'API_KEY' });

// Create a channel for publishing streamed AI responses
const channel = realtime.channels.get('my-channel');

// Publish initial message and capture the serial for appending tokens
const { serials: [msgSerial] } = await channel.publish('response', { data: '' });

// Example: stream returns events like { type: 'token', text: 'Hello' }
for await (const event of stream) {
  // Append each token as it arrives
  if (event.type === 'token') {
    channel.appendMessage(msgSerial, event.text);
  }
}

AWS us-east-1 outage: How Ably’s multi-region architecture held up

Ably Blog — Fri, 24 Oct 2025 12:24:17 +0000

Resilience in action: zero service disruption

During this week’s AWS us-east-1 outage, Ably maintained full service continuity with no customer impact. This was our multi-region architecture working exactly as designed; error rates were negligibly low and unchanged throughout. Any additional round trip latency was limited to 12ms, which is below the typical variance in any client-to-endpoint connection, and well below our 40–50ms global median; this is imperceptible to users and below monitoring thresholds. There were no user reports of issues. Taken together this means there was zero service disruption.

The technical sequence

Ably provides a globally-distributed system hosted on AWS with services provisioned in multiple regions globally. Each scales independently in response to the level of traffic in the regions, and us-east-1 is normally the busiest region.

From the onset of the AWS incident what we saw was that the infrastructure already existing in that region continued to provide error-free service. However, issues with various ancillary AWS services meant that our control plane in the region was disrupted, and it was clear that we would not be able to add capacity in the region as traffic levels increased during the day.

As a result, at around 1200 UTC we made DNS changes so that new connections were not routed to us-east-1; traffic that would have ordinarily been routed there (based on latency) were instead handled in us-east-2. This is a routine intervention that we make in response to disruption in a region. Pre-existing connections in us-east-1 remained untouched, continuing to serve traffic without errors and with normal latency throughout the incident. Our monitoring systems, via connections established before the failover, confirmed this directly.

Latency impact: negligible

We continuously test real-world performance in multiple ways. Monitors operated by Ably, in proximity to regional datacenter endpoints, indicated that the worst case impact on latency - which would have been clients directly adjacent to the us-east-1 datacenter, but which now have to connect to us-east-2 - was 12ms at p50. We also have real browser round-trip latency measurements using Uptrends, which more closely simulate real users, with actual browser instances publishing and receiving messages between various global monitoring locations.

These measurements taken during the incident are shown below; real-world clients experienced even lower latency impact, since from each of the cities tested, there is negligible difference in distance, and latency, between that location and us-east-2 versus us-east-1. Taken across all US cities that are monitoring locations, the measured latency difference averaged 3ms. That actual difference is substantially lower than normal variance in client connection latencies, and is therefore imperceptible to users and well below monitoring thresholds.

We restored us-east-1 routing on 21 October following validation from AWS and our own internal testing.

The architecture at work

This incident validated our multi-region architecture in production:

Each region operates independently, isolating failures
Latency-based DNS adapts routing to regional availability
Existing persistent connections are unaffected if the only change is to the routing of new connections
A further layer of defense, not used in this case, provides automatic client-side failover to up to five globally-distributed endpoints

That final layer matters. Even if us-east-1 infrastructure had failed entirely (it didn’t), client SDKs would have automatically failed over to alternative regions, maintaining connectivity at the cost of increased latency. It didn’t activate this time, since regional operations continued normally, but it’s a core part of our defense-in-depth strategy.

Lessons reinforced

The key takeaways for us from this incident:

A genuinely distributed system spanning multiple regions, not just availability zones, is essential for ultimate continuity of service
Planning for, and drilling, responses to this type of event is critical to ensuring that your resilience is real and not just theoretical
A multi-layered approach, with mitigations both in the infrastructure and SDKs, ensures redundancy and continuity even without active intervention. AWS continues to be an outstandingly good global service, but occasional regional failures must be expected. Well-architected systems on AWS infrastructure are capable of supporting the most critical business needs

Keep your realtime apps running smoothly, even when the internet breaks. Try Ably for free today!

How doxy.me turned realtime from a liability into a strategic asset

Ably Blog — Thu, 17 Jul 2025 16:30:49 +0000

When Realtime Breaks, Virtual Care Breaks.

Here’s the hard reality of telehealth: when infrastructure cracks, care collapses. A missed chat ping or delayed check-in notification isn’t just a glitch, it’s a broken line of communication between a provider and a patient. That can be the difference between timely diagnosis and uncertainty, between trust and frustration.

Doxy.me runs over 250,000 virtual visits every day. Their core product delivers browser-based telemedicine that works without downloads or installations. But behind the scenes, one part of the stack had become a chronic liability: realtime infrastructure.

It wasn’t just fragile - it was feared. VP of Engineering, Ben Anderson-Dukes recalled their engineering team viewing it as a “black box.” Over time, that black box became a bottleneck. Small issues like ghost check-in chimes and out-of-order messages became symptoms of deeper instability. Lag was unpredictable. Full-day outages were a looming threat. Worse still, this fragile infrastructure had become the second-largest operating expense after video delivery.

That wasn’t sustainable. But replacing their realtime provider wasn’t a simple procurement problem. It required a mindset shift from seeing realtime as a necessary evil to treating it like a core product surface that deserved strategic investment.

A Strategic Overhaul, Not Just a Quick Fix.

That partner was Ably. Unlike previous vendors, Ably took a collaborative, hands-on approach from day one, offering architectural guidance and flexible support throughout the migration.

“We needed a partner who could not only help us rebuild realtime into something reliable, scalable, and secure, but one that was developer-friendly and of a similar mindset to doxy.me.”

With support from external dev agency Walter Code, doxy.me and Ably planned a phased migration away from their existing realtime provider to Ably’s modern WebSocket-based infrastructure. The process was methodical: time it right, plan it right, implement it right. Nothing was rushed.

“Initially, we thought this would take a year. But with Ably, we went from design to 100% migrated in under six months.”

Despite handling more than 250,000 calls a day, the migration was completed with zero downtime. The transition not only modernized their infrastructure but also demystified it. Engineers gained new visibility into system behavior, and the team collectively regained confidence.

“They helped us not only get there, but raise the bar in internal education about realtime as well.”

Real Results, Not Just Promises.

Post-migration, the transformation was visible across both technical and business metrics:
• 65% reduction in realtime infrastructure costs
• 95% fewer patient queue issues
• 99% drop in app crashes caused by signaling issues
• 100% elimination of ghost check-in chimes

But beyond metrics, the real value came from stability. doxy.me’s CTO, initially skeptical, came to view realtime as a stable part of core infrastructure. The engineering team moved from firefighting to forward planning.

“Support tickets dropped, realtime errors disappeared, and our Datadog logs became clean and readable.”

The financial impact was just as compelling:

“We saw a full ROI in under six months, despite using an external team to handle the migration. That’s unheard of.”

“For me personally, it’s been a really great win. It’s bolstered Engineering’s reputation internally. There was much kudos served all-round.”

Looking Forward: Realtime as Innovation Engine.

With Ably in place, doxy.me isn’t just maintaining a stable stack, they’re building on top of it. New use cases are now in scope, including advanced presence, session orchestration, and collaboration tooling designed to make virtual care more intuitive and human.

“Even now, we’re exploring roadmap innovations together.”

One of doxy.me’s largest customers, accounting for nearly 30% of realtime traffic, now operates smoothly with no degradation in performance. Internally, engineers have better observability, faster diagnostics, and more freedom to innovate.

“Ably is helping us push realtime beyond the basics, into new opportunities that let providers be with their patients more reliably and securely.”

At its core, doxy.me is about connection - between patient and provider, between care and access.

“We believe that providers are the real heroes, and that doxy.me is their superpower.”

With Ably behind the scenes, that superpower now has a reliable, resilient realtime engine built for scale.

“Ably has helped us turn realtime from a liability into a strategic asset.”

Doxy.me built this on top of Ably Pub/Sub, the core messaging product in the Ably platform. Please see our docs if you're interested in the technical details.

Achieving low latency with pub/sub

Ably Blog — Wed, 22 Jan 2025 22:39:19 +0000

In pub/sub messaging systems, getting messages to flow quickly between publishers and users isn't just critical to its general performance, but central to its basic usability. Achieving this at scale introduces some extra challenges that require thoughtful architecture design and strategies for handling unexpected behavior (e.g. traffic spikes).

To better understand the best practices we can implement to our architecture to overcome these challenges, let's revisit how the pub/sub pattern works.

What is pub/sub?

Pub/sub (or publish/subscribe) is an architectural design pattern used in distributed systems for asynchronous communication between different components or services. Although publish/subscribe is based on earlier design patterns like message queuing and event brokers, it is more flexible and scalable. The key to this is the fact that pub/sub enables the movement of messages between different components of the system without the components being aware of each other’s identity (they are decoupled).

For a deeper dive into pub/sub, including examples and comparisons to other messaging patterns, see our guide: What is pub/sub?.

Why latency is crucial to pub/sub realtime systems

Latency is the time it takes for data to travel from the backend (like a datacenter) to the end-user’s device. Latency levels of <100ms are hard to achieve in general, but for pub/sub systems, it’s essential that those speeds remain not only consistent but also undetectable so that users remain engaged and don’t quit the app entirely. Applications that focus on, for example, crucial broadcasting updates, realtime chat, and live streaming services, need to be able to deliver seamless experiences with ultra-low latency to maintain their user bases.

This becomes especially important at scale for a global audience: If a pub/sub system can’t maintain these speeds as it scales up and reaches a global user base, message delays could render it unusable, even if your infrastructure has the raw capacity. Serving a single region is significantly simpler than achieving consistent low latency across a distributed global audience, where factors like inter-region data replication and network variability come into play. Global median latency is a good measure of average global latency if you’re operating at scale, and it’s the metric we use to measure our speeds at Ably.

Some architectural decisions you can make to achieve low latency are:

Global datacenter coverage: The physical proximity of datacenters or edge points of presence (PoPs) to end users significantly impacts round-trip times for messages. If you distribute datacenters and PoPs globally, you can drive down latency for your users.
Protocol efficiency: The choice of protocol affects how efficiently messages are transmitted. For example, WebSocket is highly efficient for realtime communication compared to HTTP long polling. (WebSockets are a particularly good protocol for achieving low latency in pub/sub systems since they maintain an open connection between the client and server without the need for frequent HTTP responses. For a deeper dive into how WebSockets compare to other protocols in pub/sub systems, check out our guide Pub/Sub vs WebSockets.)
Network robustness: A reliable, fault-tolerant underlying network infrastructure can ensure consistent low latency even under high traffic volumes.

Challenges to achieving low latency

The most straightforward obstacle to latency is network speeds - latency is inherently affected by the distance between clients and the server. The farther a client is from a datacenter, the longer it takes for messages to reach them.This is a critical consideration for global systems, where distances between users and datacenters can span continents. But there are other factors that can affect end-user latency:

Message routing: Poorly optimized routing can lead to bottlenecks, especially in use cases with high fanout where a single message is delivered to thousands or millions of subscribers.
Load balancing: If you don’t have a load balancer, or an improperly configured one, imbalances can cause overloading of certain nodes, resulting in delays for subscribers.
System resource contention: High message volumes can strain CPU, memory, and storage resources, leading to increased latency. This is particularly true during traffic spikes.
Encoding: Inefficient message encoding increases latency by slowing down the system’s ability to translate data into a transmittable format and back again.

Best practices for achieving low latency

Best practices for achieving low latency are, on paper, straightforward fixes to the points discussed above. However, making these changes to your architecture requires significant engineering effort and potentially a rehaul of your existing infrastructure. Here’s what we recommend you do:

Use a globally-distributed architecture

Deploying servers in multiple regions reduces the physical distance between clients and the server, minimizing network latency. Make sure that your infrastructure includes a combination of core datacenters and edge points of presence (PoPs). This secures fast round-trip and consistent round trip times for users anywhere in the world.

Optimize message routing

Efficient routing algorithms, such as consistent hashing, can ensure that messages are delivered to subscribers quickly and reliably. For systems with high fanout, prioritize techniques that minimize duplication and ensure messages are processed efficiently.

Have a load balancer

Dynamic load balancing distributes traffic evenly across servers, preventing overloading. For pub/sub systems, load balancers must account for both connection count and message throughput.

Use message delta compression

Compressing messages reduces their size, enabling faster transmission over the network. Use lightweight, efficient compression algorithms to minimize processing overhead.

Autoscale to reduce resource consumption

Optimize resource usage by scaling infrastructure elastically during traffic spikes. Use dynamic autoscaling to add capacity on demand and maintain a significant resource buffer.

Have redundancy and failover

Build redundancy into servers and have failover mechanisms that reroute traffic during outages. For global systems, failover strategies should account for regional redundancies to make sure that if one region experiences an outage, traffic can seamlessly shift to another without impacting users worldwide. This minimizes latency spikes during failover events and ensures uninterrupted service.

How Ably can help

For many people, building a system with all of these components from the ground up is impractical and is a huge time and skill investment. That investment also tends to be more expensive than initially expected because of maintenance costs and other challenges - like scalability and data integrity - that make maintaining a low enough latency even more difficult.

At Ably, our team is very familiar with the amount of work building a low-latency pub/sub system takes - and all the edge cases around optimum performance. We’ve made it our mission to provide the most reliable realtime service for you - and Ably Pub/Sub is devoted to pub/sub use cases.

Choosing a managed pub/sub service like Ably can save you and your team the headache of managing the architectural challenges of low latency at scale. Performance is one of Ably’s core pillars, and it’s built into what we do. Here’s how:

Predictable performance: A low-latency and high-throughput global edge network, with median latencies of <50ms.
Guaranteed ordering & delivery: Messages are delivered in order and exactly once, with automatic reconnections.
Fault-tolerant infrastructure: Redundancy at regional and global levels with 99.999% uptime SLAs. 99.999999% (8x9s) message availability and survivability, even with datacenter failures.
High scalability & availability: Built and battle-tested to handle millions of concurrent connections at scale.
Optimized build times and costs: Deployments typically see a 21x lower cost and upwards of $1M saved in the first year.

Low latency is non-negotiable for any pub/sub system that aims to deliver realtime experiences at scale. If you’re looking for a solution that scales up and ensures some of the lowest latencies in the business, Ably provides a robust and reliable platform to power your pub/sub needs. Sign up for a free account to try it for yourself.

How to use Ably LiveSync’s MongoDB Connector for realtime and offline data sync

Carolina Carriazo — Thu, 16 Jan 2025 13:01:19 +0000

In light of the recent deprecation of MongoDB Atlas Device Sync (ADS), developers are seeking alternative solutions to synchronize on-device data with cloud databases. Ably LiveSync offers a potential alternative and can replace some of ADS’s functionality, enabling realtime synchronization of database changes to devices at scale. LiveSync allows for a large number of changes to MongoDB to be propagated to end user devices in realtime and store the changes in any number of local storage options - from an embedded database to in-memory storage.

For instance, imagine an inventory app that needs to broadcast stock updates to multiple devices in realtime. Ably LiveSync allows you to automatically subscribe to inventory changes in your database and broadcast this data to millions of clients at scale, allowing them to remain synchronized with the state of your inventory in realtime.

This article explains why on-device storage is critical, explores existing solutions, and demonstrates how Ably LiveSync’s MongoDB connector can help with a brief code tutorial.

Why keep information on-device?

Local storage is a must for apps that need offline access or fast performance— like e-commerce inventory apps, or news apps downloading content for offline browsing. But not every app needs it. If your app is always online or just streams read-only data, you can skip the complexity of a local database. Thankfully, with Ably, you can adapt to your use case, whether you need offline support or just realtime updates. Some of the benefits of on-device storage are:

Offline access: Storing data directly on the device ensures users can seamlessly access and interact with information even when they have no internet connection or are in areas with poor connectivity. This is particularly crucial for users who frequently work in offline environments or travel to locations with unreliable network coverage.
Performance: Applications demonstrate significantly improved response times and reduced latency when accessing data stored locally, as opposed to making time-consuming server calls across the network. This local data access eliminates network-related delays and provides instantaneous data retrieval for critical operations. Cost efficiency: Users experience substantial savings on their data usage and associated costs since the application doesn't need to repeatedly download information from remote servers. This is especially beneficial for users with limited data plans or in regions where mobile data is expensive.
User experience: Users benefit from a consistently smooth and reliable application experience, maintaining uninterrupted access to their data regardless of their network status or connection quality. This reliability helps build user trust and satisfaction with the application.

Options for storing information on device

Modern mobile operating systems provide a variety of ways to store information on device:

iOS: Includes UserDefaults, CoreData, and SQLite, with flexibility for additional solutions based on specific needs.
Android: Provides shared preferences, Room database, and file storage.
Cross-platform frameworks: With React Native, react-native-async-storage is a popular starting library for simple needs. However, for advanced use cases requiring NoSQL-like abilities, some good choices here would be RealmDB (which, unfortunately, as we know, is being deprecated), UnQLite, LevelDB, or Couchbase.

Regardless of your choice of an on-device database and storage methodology, you can use Ably LiveSync to synchronize data from your managed or on-premises database to mobile devices in realtime. This includes MongoDB - as well as Atlas. While we currently support only MongoDB and PostgresSQL, we are working on adding support for other database engines.

What is Ably LiveSync?

Ably LiveSync lets you monitor database changes and reliably broadcast them to millions of frontend clients, keeping them up-to-date in realtime.

LiveSync works with any tech stack and prevents data inconsistencies from dual-writes while avoiding scaling issues from "thundering herds" — sudden surges of traffic that can overwhelm your database.

How to persist data locally with Ably

Let’s explore how to build a simple in-store management app that tracks product inventory using React Native and SQLite for local storage and a Mongo Atlas for our cloud database. Despite Mongo being a document storage and SQLite being a relational database, the two can be used in combination. We are going to use the Ably SDK callback methods to store documents and changes inside our local SQLite database.

Setting up Ably

For simplicity, we’ll stick to TypeScript. Before anything else, create a new React Native project using the CLI:

npx @react-native-community/cli@latest init AwesomeStore

Creating a MongoDB integration rule with Ably

Now we need to create a new channel that streams database changes to your clients. This ensures realtime updates whenever your MongoDB data changes. To create an integration rule that will sync your MongoDB database with Ably, you’ll first have to sign up for an Ably account.

Once that’s done, you should have access to your Ably dashboard. Create an app or select the app you wish to use. Navigate to the Integrations tab > Create a new integration rule > MongoDB. Fill out the Connection URL with your MongoDB connection URL; Database name with your db name (for this example, SQLiteDatabase); and Collection with your collection name (for this example, products). For more information on this process and the parameters involved, check out our docs.

This sets up a new channel, built on top of our core Ably Pub/Sub product, which streams changes (through MongoDB change streams) from your database to your clients.This essentially ensures that any change that occur in your database will be delivered to any device subscribed to a channel.

Creating the local datastore

We’ll create a new file in our project called datastore.js and initialize SQLite:

export const createTables = async (db: SQLiteDatabase) => {

  const query = `CREATE TABLE IF NOT EXISTS products(
        id INT32
        name TEXT NOT NULL
        description TEXT
        quantity INT32
    );`;

  await db.executeSql(query);
};

After the tables are created, we need a way to retrieve store products and update their stock:

export const getProducts = async (db: SQLiteDatabase): Promise<ToDoItem[]> => {
  try {
    const products: StoreProduct[] = [];
    const results = await db.executeSql(`SELECT id, name, description, quantity FROM ${tableName}`);
    results.forEach(result => {
      for (let index = 0; index < result.rows.length; index++) {
        products.push(result.rows.item(index))
      }
    });
    return products;
  } catch (error) {
    console.error(error);
    throw Error('Failed to get products!');
  }
};

export const saveOrUpdateProducts = async (db: SQLiteDatabase, products: StoreProduct[]) => {
  const insertQuery =
    `INSERT OR REPLACE INTO ${tableName}(id, name, description, quantity) values` +
    products.map(i => `(${i.id}, '${i.value}', '${i.description}', '${i.quantity}')`).join(',');

  return db.executeSql(insertQuery);
};

Receiving database changes from MongoDB over Ably

Let’s take a look at how we can receive changes from the configured Ably channel. More information can be found in our documentation, but this is the important snippet we need - setting up the Ably Realtime SDK:

import * as Ably from 'ably';

// Instantiate the Realtime SDK
const ably = new Ably.Realtime('[your API key]');

// Get the channel to subscribe to
const channel = ably.channels.get('store:1');

// Subscribe to messages on the 'store:1' channel
await channel.subscribe((message) => {
    // Print every change detected in the channel
  console.log('Received a change event in realtime: ' + message.data)
});

We need to write a new function that will take the payload of message.data and store it in our database:

const addOrUpdateProduct = async (data: any) => {
    try {
      let product: StoreProduct = {
            id: data.fullDocument.id,
            name: data.fullDocument.name,
            description: data.fullDocument.description,
            quantity: data.fullDocument.quantity,
      }
      const db = await getDBConnection();
      await saveOrUpdateProducts(db, product);
    } catch (error) {
      console.error(error);
    }
  };

We can call our new function in our message subscription:

// Subscribe to messages on the 'store:1' channel
await channel.subscribe((message) => {
      // Print every change detected in the channel
      console.log('Received a change event in realtime: ' + message.data)
      if (data.ns.coll === "products") {
            await addOrUpdateProduct(message.data)
      } else if (data.ns.coll === "store") {
            // Another function which updates changes for the `store` collection
      } else {
            console.warn("Unknown collection");
      }
});

The full workflow

With this setup, the app listens for realtime updates from your MongoDB collection and persists changes locally, ensuring an up-to-date inventory system even when offline.

Final thoughts: What makes Ably different

I hope that gives you a good overview of what Ably LiveSync’s MongoDB Connector can do! Besides providing a potential alternative to Atlas Device Sync, Ably, as a realtime communications platform, is built for scalability and reliability. Here are some features of Ably Pub/Sub, the backbone upon which LiveSync’s database connector is built:

Predictable performance: A low-latency and high-throughput global edge network, with median latencies of <50ms.
Guaranteed ordering & delivery: Messages are delivered in order and exactly once, even after disconnections.
Fault-tolerant infrastructure: Redundancy at regional and global levels with 99.999% uptime SLAs. 99.999999% (8x9s) message availability and survivability, even with datacenter failures.
High scalability & availability: Built and battle-tested to handle millions of concurrent connections at scale.
Optimized build times and costs: Deployments typically see a 21x lower cost and upwards of $1M saved in the first year.Try Ably today and explore our MongoDB connector.

Low latency at scale: Gaining the competitive edge in sports betting

Ably Blog — Mon, 06 Jan 2025 09:27:04 +0000

The sports betting industry has grown rapidly in recent years, fueled by changing regulations, advancements in technology, and a rising demand for realtime interactions from consumers. For many fans, in-play betting adds another dimension to how they can engage, making them feel closer to the action. When you then consider the increasing number of global followers many teams now have, it’s easy to understand why global revenues are projected to continue expanding at a compound annual growth rate (CAGR) exceeding 10%. To sustain this growth, data providers and betting companies face a key challenge: consistently ensuring fast, low latency delivery of data and services, even as they scale operations to meet growing demand – every fan needs to receive the same data at the same time.

Low latency - the rapid transmission of data with minimal delay—plays a crucial role in the sports betting ecosystem. A single delay in odds updates or live bet placement can impact user retention and create financial exposure for operators. This challenge becomes even more pressing as companies expand into new geographies, requiring infrastructure that can deliver consistent, low latency experiences worldwide.

Why low latency matters in sports betting

In sports betting, every second matters. For bettors, much of the appeal of sports betting lies in the immediacy of their interaction with live events. Odds must be delivered and displayed in realtime, and delays can result in missed opportunities for bettors and revenue loss for operators.

If odds are not updated instantly after a critical game event, operators risk users betting on outdated odds. For data providers, failure to deliver realtime data can strain relationships with clients, harm reputation and even have legal implications.

The importance of a consistent user experience

Scaling betting operations isn’t just about onboarding new users—it’s about ensuring these users have the same experience, regardless of location or time.

There are two key infrastructure requirements involved in making this possible: Consistent low latency, and the ability to handle disconnections.

When it comes to latency, users demand the same experience whether they’re betting from Europe, Asia, or the Americas. A lack of consistency can create an uneven playing field – or even cause users to abandon platforms as they develop a mistrust for their data. For both data providers and betting companies, this means achieving similar levels of low latency to clients in diverse geographies. From an infrastructure perspective, this means having a global network of points of presence which devices can connect to and achieve low latency, wherever your users are.

When it comes to handling disconnections and subsequent reconnections, this is particularly key when serving users in markets with unstable network conditions – but is also useful when a user changes data networks or is travelling. For betting operators, they need a strategy to define how a bet is treated if it is placed as a user loses connectivity, and the odds change while they are offline for example – should the original odds be ‘retained’, or should the odds be refreshed upon reconnection? Operators need to consider how to ensure that users with worse network connectivity don’t become ‘worse off’ as a result.

How Genius Sports delivers critical data at a global scale

Genius Sports serves betting companies that demand instantaneous data delivery for live sporting events. Before adopting their current realtime infrastructure, they relied on a traditional centralised system to deliver live data to its betting clients. Maintaining live data performance at low latency and on a global scale, meant locating ever larger and more costly on-premise facilities, close to customers. The system struggled with scalability and latency consistency, especially during high-demand events like global tournaments. As costs and latency demands increased, Genius Sports needed a new realtime solution.

To ensure a consistent user experience, the company switched to a cloud-native distributed infrastructure that could handle their need to reliably serve customers – regardless of their location. By leveraging a WebSocket-based realtime data streaming solution, Genius Sports is now able to deliver on the high expectations of both its B2B clients and the end-users who rely on their ultra-low latency data delivery without the headache of having to manage and scale the realtime infrastructure.

Responding to the growing competition for fan engagement

As the sports betting market becomes more competitive, betting companies are incorporating innovative ways to deliver realtime experiences to keep their customers ‘on-platform’. This means investing in infrastructure that can not only handle the fast-paced nature of betting and provide instant, low latency, updates but also enable the creation of interactive experiences.

Sportsbet faced a unique challenge with its innovative “Bet With Mates” feature, a product allowing friends to pool resources and bet together. A critical component was a chat feature that mirrored the functionality and speed of modern messaging applications like WhatsApp.

Sportsbet’s existing infrastructure could not deliver the rich user experience their customers expected, especially for features like realtime updates, reactions, and comments. As well as latency needs, Sportsbet also had very stringent security and data handling requirements. For Sportsbet, opting for a cloud-native realtime platform that handled both the underlying infrastructure and latency complexities but also made building their chat solution easy, was game changing. The solution saved Sportsbet valuable time and effort compared to them building a basic implementation from scratch.

Low latency, high stakes

It is clear that the sports betting industry is at an inflection point. As regulatory changes continue to unlock new opportunities, companies must rise to the challenge of delivering low latency experiences at scale. Whether it’s B2B providers like Genius Sports ensuring data consistency or B2C platforms like Sportsbet creating seamless user experiences, the ability to operate in realtime, no matter where and when, will define success.

How Sportsbet handles 4.5M daily chat messages on its 'Bet With Mates' platform

Ably Blog — Fri, 20 Dec 2024 12:35:55 +0000

Sportsbet is a leader in the Australian wagering market. Through their best-in-class platform, which includes ‘Bet With Mates’, they bring excitement to life for sports and racing enthusiasts - replicating the experience of punting with friends in the pub, no matter where they are.

But after launching 'Bet With Mates', Sportsbet customers were still off the platform (second-screening) to talk about their bets and banter with each other in WhatsApp and other chat apps. Sportsbet wanted to introduce that functionality into ‘Bet With Mates’ and provide everything their customers needed without ever having to leave the platform.

Sportsbet needed a chat feature that met their customers’ high expectations of messaging applications.

Selecting the right chat solution for 'Bet With Mates'

When deciding which solution to use for the 'Bet With Mates' chat, there were a few key criteria.

It had to be feature-rich including reaction and reply functionality and also update in realtime.

As an extremely event-driven business with huge traffic spikes during major events like the Australian Football League, National Rugby League finals, and the Melbourne Cup, the solution needed to be highly performant and scalable.

It had to demonstrate great frontend performance figures, integrate well into Sportsbet’s build pipeline, and be future-proofed for other realtime use cases that developers were planning.

As well as latency needs, Sportsbet also had very stringent security and data handling requirements, so the solution needed to be hosted within Australia on a dedicated cluster.

And finally, the team decided they needed to deliver the new product in less than four months so that it would be live and in customer hands before the next AFL season launch!

Why Sportsbet chose Ably

Based on their requirements, Sportsbet decided to move ahead with Ably after comparing them to other providers. Some key benefits for Sportsbet included:

Great customer support: Early assessments of Ably’s documentation encouraged Sportsbet to quickly move to build a proof of concept. This involved dedicated support from the Ably team to consult on requirements, providing access to SDKs for their chosen tech stack, and creating a sandbox environment to conduct some integration testing and analysis.
Easy to get started: Sportsbet started out building a prototype using Ably React Hooks with their existing react client and were impressed with how quickly they could get a basic chat feature going without having to build services. They then added some components that published events including bet placements and group activity as well as features like reactions and comments in the same message stream.
Support for data security: Ably rapidly spun up a new dedicated cluster within Australia specifically for Sportsbet, which removed another potential barrier to being ready for the AFL season. Ably’s SAML integration also enabled Sportsbet to plug into their existing SSO system in record time.

Sportsbet + Ably: The results

The ‘Bet With Mates’ chat feature has proven a hit with fans, contributing to the organic growth of the overall ‘Bet with Mates’ platform. It has also proven sticky – customers who use ‘Bet With Mates’ Chat use it regularly.

Sportsbet put this success down to Ably’s unwavering reliability when it comes to performance and message delivery. They also reported that autoscaling has performed flawlessly without any incidents of concern in over a year, even on high traffic days. Peak figures for these high traffic periods have reached around 4.5 million published messages a day.

Reflecting on the success of the project and relationship with Ably, Andy commented:

“By choosing to partner with Ably, we were able to deliver a high quality outcome in a frankly impressive timeframe, and free up our delivery teams earlier to focus on other initiatives. It’s a testament to the strength of Ably’s offering how much of our time with them is spent discussing other potential use cases rather than the current implementation.”

Ably: The definitive realtime experience platform. Built for scale.

Sportsbet is one of the thousands of companies that depend on Ably to power realtime experiences for billions of people - including live updates, chat, collaboration, notifications and fan engagement. Reliably, securely and at serious scale.

Why choose Ably?

99.999% uptime SLA: We guarantee 5x9s of uptime, but consistently do better. We've had 100% uptime for 5+ years.
No scale ceiling: Ably handles massive amounts of data throughput and concurrent connections without SREs breaking into sweat.
Strong data integrity: Guaranteed data ordering, delivery, and exactly-once semantics. Even under unreliable network conditions.
Almost-infinite elasticity: Bursty connection traffic? Ably seamlessly and automatically absorbs millions of concurrent connections arriving at once.
Composable realtime: Our range of application building blocks and integrations enable developers to create the live experiences users and businesses demand. From live chat to data broadcast, and collaborative UXs to notifications, our SDKs unlock innovation - with no infrastructure to build.
Customer-first pricing, affordable at scale: Ably's pricing offers per-minute billing, consumption-based pricing, and volume-based discounts to keep you ROI positive, as you scale.

For more information, read our docs, or sign up for free!

How Mentimeter deliver reliable live experiences at scale

Ably Blog — Thu, 19 Dec 2024 16:51:00 +0000

There are no second chances when it comes to live events: Mentimeter's solutions have to work perfectly every time. Their audience engagement features must be accessible via mobile devices without user sign-in. They must also be fast. And they need to scale effortlessly to cope with huge spikes in demand; a single event can drive connections from zero to 70,000+ participants in a matter of seconds.

Mentimeter's engineers designed their systems to cope with those exacting demands, but as the business experienced rapid growth, its realtime infrastructure provider struggled to keep pace. The platform's performance at scale started to suffer.

Eventually, a tipping point came when a spike of a relatively small number of concurrent connections – ~35,000 – caused part of their realtime system provider's network to crash. Luckily, Mentimeter had fallback solutions in place for realtime communication so their services continued to operate, but with a degraded user experience for several hours. They knew it was time to find an alternative solution.

Selecting the right solution for growth-ready realtime

Johan Bengtsson, Mentimeter CTO, swiftly dismissed the possibility of building realtime infrastructure in-house due to the time and cost to build, along with significant ongoing operating costs and engineering burden. His team set out to identify a new partner that could meet Mentimeter's exacting demands while supporting all Mentimeter's current and future use cases.

Any new realtime provider needed seamless integration with Mentimeter's React front end, Ruby and Node.js backend, and AWS Kinesis for analytics. Performance, scalability, and reliability were essential requirements, particularly as Mentimeter had ambitions to raise its concurrent connections limit and cater for events of up to 150,000 participants or more. What's more, a new provider needed to provide Bengtsson with scope to innovate in bringing new features to the platform.

Why Mentimeter chose Ably
Mentimeter identified Ably as the ideal solution based on a market comparison exercise and a recommendation from existing Ably customer Split.io.

According to Bengtsson: "Ably is very transparent about engineering for reliability and scalability with its Four Pillars of Dependability. Along with a solid five-nines SLA and clear pricing, that gave me a lot of confidence. I also felt that Ably wanted to be more than a supplier. There was a real sense that it would be a true partner that would work closely with us, listening and responding to our needs and supporting our innovation roadmap."

Bengtsson was able to get a proof of concept up and running within two hours. Even though Mentimeter's developer team was stretched at the time, migration to Ably was complete inside a month, thanks to expert and fast support and a wealth of documentation.

Mentimeter + Ably: The results

The decision to move to Ably has been crucial for Mentimeter to deliver a consistent, high-quality customer experience. Bengtsson and his team are confident they can rely on its services to help them deliver business value through improved communication and audience engagement.

First and foremost, Ably's meticulous design for elastic scale and high availability has solved the scalability and reliability issues that had become an issue previously.

Bengtsson explains: "Ably has been super reliable, a part of our stack I know I can depend on. It copes easily with big loads generated by multiple presentations happening simultaneously across the globe. We put the reliability of presentation experience front and center. If Ably had issues, we'd know about them, but now we don't have to worry about stability, even when we get huge traffic spikes. Ably is a key partner for us, one that we refer to as 'Enably' because it allows us to innovate at pace to elevate our core proposition. In fact, we're so confident now, we're looking to triple our concurrent connections limit to 150,000."

Mentimeter has used Ably to innovate as it seeks to enhance and extend the value its platform delivers to customers. For instance, the Mentimeter team builds upon the Ably presence feature to ensure customers collaborating upon interactive presentations can see who is online and carrying out edits. Before adopting Ably, this was functionality the team was going to build in-house.

Bengtsson said: "We're delighted with the range of capabilities Ably gives us, while the support and documentation have helped to spark ideas at our end. The limitations with leaderboard scaling were hurting us in terms of our ability to innovate the core product. With Ably, those limits are a thing of the past."

Finally, moving to Ably has helped Mentimeter find efficiencies, spend less time on maintenance and realtime incidents, and allow its engineers to develop the core platform.

"It's part of our ethos that if there is a service providing the capabilities we need, we'll use it rather than trying to build and maintain ourselves," Bengtsson explained. "With Ably in place, our engineers can focus on what they are good at. They don't have to worry about realtime any more, which makes for a happier, more productive team."

Ably: The definitive realtime experience platform. Built for scale.

Mentimeter is one of the thousands of companies that depend on Ably to power realtime experiences for billions of people - including live updates, chat, collaboration, notifications and fan engagement. Reliably, securely and at serious scale.

Why choose Ably?

99.999% uptime SLA: We guarantee 5x9s of uptime, but consistently do better. We've had 100% uptime for 5+ years.
No scale ceiling: Ably handles massive amounts of data throughput and concurrent connections without SREs breaking into sweat.
Strong data integrity: Guaranteed data ordering, delivery, and exactly-once semantics. Even under unreliable network conditions.
Almost-infinite elasticity: Bursty connection traffic? Ably seamlessly and automatically absorbs millions of concurrent connections arriving at once.
Composable realtime: Our range of application building blocks and integrations enable developers to create the live experiences users and businesses demand. From live chat to data broadcast, and collaborative UXs to notifications, our SDKs unlock innovation - with no infrastructure to build.
Customer-first pricing, affordable at scale: Ably's pricing offers per-minute billing, consumption-based pricing, and volume-based discounts to keep you ROI positive, as you scale.

For more information, read our docs, or sign up for free!

Scaling PubSub with WebSockets and Redis

Steven Lindsay — Thu, 19 Dec 2024 14:24:24 +0000

There is increasing demand for realtime data delivery as users expect faster experiences and instantaneous transactions. This means not only is lower latency required per message, but providers must also handle far greater capacity in a more globally-distributed way.
When building realtime applications, the right tools for the job address these requirements at scale. WebSocket has become the dominant protocol for realtime web applications, and has widespread browser support. It offers a persistent, fully-duplexed connection to achieve low latency requirements. Publish/Subscribe (pub/sub) has been around even longer, and enables providers to scale data transmission systems dynamically, something that is crucial in providing these new capacity needs. While Pub/Sub itself is just an architectural pattern, tools like Redis have built out pub/sub functionality to make it easier to develop and deploy a scalable pub/sub system.

This article details how to build a simple pub/sub service with these components (WebSockets and Redis). It also discusses the particular technical challenges of ensuring a smooth realtime service at scale, the architectural and maintenance considerations needed to keep it running, and how a realtime service provider could simplify these problems and your code.

First, let’s briefly define the fundamental components at play.

What is pub/sub?

The core of pub/sub is messaging. Messages are discrete packets of data sent between systems. On top of this, channels (or topics) allows you to apply filtering semantics to messages; publishers (clients that create messages); and subscribers (clients that consume messages). A client can publish and consume messages from the same channel or many channels.

Because of this, pub/sub can be used to support a number of different patterns:

One-to-one: Two clients that both need to subscribe and publish messages to the same channel. This is common in support chat use cases.
One-to-many: Many clients may receive information from a centralized source, as with dashboard notifications.
Many-to-one: Many clients publish messages to one channel; for example, in a centralized logging system, all logs of a certain tag are routed to one repository.
Many-to-many: Multiple clients send messages to all in the members of the group - Such as a group chat or presence feature.

The pub/sub pattern allows clients to send and receive messages directly, without a direct connection to recipients with a message broker. It promotes an async and decoupled design, which lends itself to building scalable, distributed systems that can handle concurrent traffic for millions of connected clients. However, in practice, building a pub/sub system like this requires a reliable and performant data store.

What is Redis?

Redis is a highly performant in-memory key-value store, though it can also persist data to non-volatile storage (disk). It is capable of processing millions of operations per second, so makes it ideal for systems with high throughput. Its feature set includes pub/sub, allowing clients to publish to a channel which is then broadcast out to all subscribers of that channel in realtime.

What are WebSockets?

WebSockets are a protocol for bidirectional communication, and are a great choice where low latency and persistent connectivity are required. In contrast to something like HTTP long polling, which repeatedly queries the server for updates, WebSockets make a continuous connection between the client and server, so that messages flow in both directions as and when they occur. Because of this, WebsSockets can be ideal for applications like chat systems, live notifications, or collaborative tools. Having a persistent connection removes the need to poll a server for updates, this in turn reduces latency, a key metric in realtime applications.

They are commonly used for communication between some web client (browser) and some backend server. The persistent connection is formed over TCP.

A tutorial: A simple pub/sub service

Now, let’s see how a simple realtime pub/sub service comes together with Redis and WebSockets, where multiple clients can subscribe to a channel and receive channel messages.

A typical architecture consists of a WebSocket server for handling client connections, backed by Redis as the Pub/Sub layer for distributing new messages. A load balancer like NGINX, or AWS ALB is used to handle incoming WebSocket connections and route them across multiple server instances; this is key to distributing load on our service.

Autoscaling can be used to dynamically adjust our servers to match demand, maintaining performance while reducing cost during quiet periods. To do this, we would need our WebSocket servers to remain stateless.

Breakdown of components

WebSocket server - Clients connect to this server to receive updates from a channel. The server subscribes to Redis channels and forwards messages it receives through the pub/sub subscription. We can build this with something like Gorilla-WebSocket or Express-ws.
Redis as a pub/sub layer - The distribution service between publishers (backend services) and subscribers (our WebSocket servers). For simplicity, we will run a single instance of Redis. In production this will likely need to be clustered to avoid becoming a bottleneck.
Load balancer (NGINX) - Distributes incoming WebSocket connections across multiple WebSocket server instances. It could do this naively in a round-robin fashion, or use some other method like IP hashing, though care should be taken to avoid creating hotspots.
Autoscaler - It’s highly likely we will experience changing traffic, so an autoscaler will adjust our WebSocket servers according to available resources. We can implement this with Kubernetes, which can monitor metrics and respond accordingly.
Monitoring and logging - Monitoring tools like Prometheus and Grafana track system performance, latency, and message delivery success rates. This will allow us to see how our system is operating and catch any errors that occur. It’s a good idea to configure some alerting on these metrics too.

We will write the server in TypeScript (TS) using Node as our runtime environment. We won’t cover setup and usage of auto scaling, but more information on this topic can be found here.

1. Set up NGINX

Install NGINX and get it running. On Mac or Linux operating systems, we could do this with a package manager like brew.

brew install nginx && brew services start nginx

We also need to configure it to route connections, forward requests to the server, and ensure NGINX handles the WebSocket requests correctly, something like this:

# Define the upstream WebSocket server
   upstream websocket_backend {
       server 127.0.0.1:8080; # Your WebSocket server address
   }
location /ws {
           # Proxy WebSocket traffic to the backend
           proxy_pass http://websocket_backend;
           # Required for WebSocket
           proxy_http_version 1.1;
           proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "Upgrade";
           # Pass the original Host header
           proxy_set_header Host $host;
           # Timeouts for WebSocket
           proxy_read_timeout 300s;
           proxy_send_timeout 300s;
       }

2. Set up Redis

Install Redis and get it running.


brew install redis && brew services start redis

We should now have a local instance available at localhost. Note: this creates an instance of Redis with no authentication, and is not an advisable setup for production.

3. Set up the server

Now, we will need to write the server code to handle client requests, integrate with our Redis instance, and allow WebSocket connections. TypeScript has a vast range of packages that we can use, such as WS, which provides WebSocket client and server capabilities. WS implements the WebSocket protocol and abstracts away a lot of the details like handshakes, framing and connection management, so we get a nice high-level API.


import WebSocket, { WebSocketServer } from 'ws';

We will also need the ioredis package, which provides a fully-featured client to manage our connections to Redis.


import Redis from 'ioredis';

Let’s create a basic application that will do the following;

Accept client requests and upgrade them to WebSockets
Provide simple handling of the WebSocket and Channel lifecycle
Store the new connections so we can send messages as needed
Connect to Redis and subscribe to a specified channel
When a new message is received on the channel, iterate through all subscribed clients and send the message along.
Remove unused redis connections

This looks like:


// Maps to track channel subscriptions
const redisClients: Map<string, Redis> = new Map(); // Redis client per channel


// Maps to track WebSocket connections
const channelSubscribers: Map<string, Set<WebSocket>> = new Map(); // WebSocket clients per channel


// WebSocket Server
const wss = new WebSocketServer({ port: 3000 });
console.log('WebSocket server is listening on ws://localhost:3000');


wss.on('connection', (ws: WebSocket) => {
   console.log('New WebSocket connection established');


   ws.on('message', (message: string) => {
       try {
           const data = JSON.parse(message);


           // Validate the message format
           if (data.action === 'subscribe' && typeof data.channel === 'string') {
               const channel = data.channel;


               // Ensure a Redis client exists for this channel
               if (!redisClients.has(channel)) {
                   const redisClient = new Redis();
                   redisClients.set(channel, redisClient);


                   // Attempt to subscribe to the Redis channel
                   redisClient.subscribe(channel, (err) => {
                       if (err) {
                           console.error(`Failed to subscribe to Redis channel ${channel}:`, err);
                           ws.send(JSON.stringify({ error: `Failed to subscribe to channel ${channel}` }));
                       } else {
                           console.log(`Subscribed to Redis channel: ${channel}`);
                       }
                   });


                   // Handle incoming messages from the Redis channel
                   redisClient.on('message', (chan, message) => {
                       if (channelSubscribers.has(chan)) {
                           channelSubscribers.get(chan)?.forEach((client) => {
                               if (client.readyState === WebSocket.OPEN) {
                                   client.send(JSON.stringify({ channel: chan, message }));
                               }
                           });
                       }
                   });
               }


               // Add this WebSocket client to the channel's subscribers
               if (!channelSubscribers.has(channel)) {
                   channelSubscribers.set(channel, new Set());
               }
               channelSubscribers.get(channel)?.add(ws);
               console.log(`WebSocket subscribed to channel: ${channel}`);
           } else {
               ws.send(JSON.stringify({ error: 'Invalid action or channel name' }));
           }
       } catch (err) {
           console.error('Error processing WebSocket message:', err);
           ws.send(JSON.stringify({ error: 'Invalid message format' }));
       }
   });


   ws.on('close', () => {
       console.log('WebSocket connection closed');


       // Remove this WebSocket from all subscribed channels
       channelSubscribers.forEach((subscribers, channel) => {
           subscribers.delete(ws);


           // If no more subscribers for the channel, clean up Redis client
           if (subscribers.size === 0) {
               redisClients.get(channel)?.quit();
               redisClients.delete(channel);
               channelSubscribers.delete(channel);
               console.log(`No more subscribers; unsubscribed and cleaned up Redis client for channel: ${channel}`);
           }
       });
   });


   ws.on('error', (err) => {
       console.error('WebSocket error:', err);
   });
});


}

4. Send some messages

Produce some messages on the channel so that the server can begin sending the data on to connected clients. Here, as an example, we'll query for telemetry data of the International Space Station (ISS) each second, then publish this to our channel.

import axios from 'axios';
import Redis from 'ioredis';

const telemetryUrl = 'http://api.open-notify.org/iss-now.json'; 

const redisChannel = 'iss.telemetry'; // Redis channel name
const publishInterval = 1000; // Interval in milliseconds

// Create a Redis client
const redis = new Redis();

// Function to query URL and publish to Redis
async function queryAndPublish() {
   try {
       console.log(`Fetching data from ${telemetryUrl}...`);

       // Query the URL
       const response = await axios.get(telemetryUrl);

       // Get the response payload
       const payload = response.data;

       // Publish the payload to the Redis channel
       const payloadString = JSON.stringify(payload);
       await redis.publish(redisChannel, payloadString);

       console.log(`Published data to Redis channel "${redisChannel}":`, payloadString);
   } catch (error) {
       console.error('Error publishing data:', error);
   } finally {
       // Schedule the next query and publish task
       setTimeout(queryAndPublish, publishInterval);
   }
}

// Start the query and publish loop
queryAndPublish()

// Handle graceful shutdown
process.on('SIGINT', () => {
   console.log('Shutting down...');
   redis.disconnect();
   process.exit();
});

5. Create the client

Construct a simple client that will point to our NGINX server and establish a WebSocket connection. It should begin receiving messages published to the channel. We have kept things simple here, but a production solution would need to handle things like heartbeats, retry mechanics, failover, authentication and more.

import WebSocket from 'ws';

const serverUrl = 'ws://localhost/ws';
const channelName = 'iss.telemetry';

// Create a new WebSocket connection
const ws = new WebSocket(serverUrl);

// Handle the connection open event
ws.on('open', () => {
   console.log('Connected to WebSocket server');

   // Send a subscription request for the specified channel
   const subscriptionMessage = JSON.stringify({ action: 'subscribe', channel: channelName });
   ws.send(subscriptionMessage);
   console.log(`Subscribed to channel: ${channelName}`);
});

// Handle incoming messages
ws.on('message', (message) => {
   console.log(`Message received on channel ${channelName}:`, message.toString());
});

// Handle connection close event
ws.on('close', () => {
   console.log('WebSocket connection closed');
});

// Handle errors
ws.on('error', (error) => {
   console.error('WebSocket error:', error);
});

After starting up the server, publisher, and client, we should see data being printed out to the standard output (stdout).

Challenges of a Redis + WebSockets build at scale

The code we’ve written is a basic implementation, but at scale in production, we will have to account for additional complexities. There are some issues with using Redis at scale, including:

Message persistence

In Redis pub/sub, messages are not persisted and are lost if no subscriber is listening (fire-and-forget). For some realtime use cases, like chat, this is likely not acceptable.

Horizontal scaling

A single instance of Redis is insufficient at scale, and as there is a limit to how far it can be vertically scaled, so a horizontally-scaled infrastructure is going to be your best option.

Redis can be deployed in a clustered mode for high availability and scalability. This essentially turns your single Redis instances into a network of interconnected redis nodes, then data can be distributed across these nodes. It allows for potentially much higher throughput and greater numbers of subscribers. Each node would take a portion of the data, this can be managed automatically, to distribute the workload (through sharding). But this comes with its own complexities, including:

Needing active management to ensure even data distribution and cluster health - not specific to Redis, but general distributed systems issues, including:
- Failover mechanisms when a node goes down
- Manual node provisioning in case there are no replica sets
- Robust monitoring to catch network issues
- Increased networking overheads for cross-node communication
Node discovery, leader election and split brain (where multiple leader/primary nodes become disconnected and we need to work out which nodes we will consider to still be active). Redis cluster uses a gossip protocol to share cluster state and quorum based voting for leader election and to help prevent split brain.
- There should be an odd number of nodes in a redis cluster setup, so if a network partition occurs, a quorum can still be reached.
Lack of strong consistency guarantees - redis cluster prioritises throughput and works on eventual consistency, this means you can face data loss in some cases. You could use something like redisraft (experimental) that implements the RAFT consensus algorithm, but this writes to only a single leader at a time, creating a bottle neck for write-heavy workloads. (And RAFT does not account for byzantine failures, though these are rare in practice.)

Disaster recovery and geo distribution

For realtime use cases at scale, a globally-distributed system is non-negotiable. For example, what happens if our single datacenter fails? Or what if customers on the other side of the world require the lowest latency possible? Forcing a request between geographic regions would incur unacceptable latency penalties. Both problems would require us to geolocate our cluster in multiple regions.

A Redis cluster can be globally distributed - in the case of AWS, this would mean running in multiple availability zones. But this introduces more management and maintenance complexities on top of those we’ve already accumulated for horizontal scaling, like the need to ensure correct sharding, manage reconciliation of concurrent writes across regions, route traffic in the case of a net split, and rebalance to avoid hot nodes.

Challenges of WebSockets as a protocol

WebSockets on their own have potential issues to consider in the context of building out a realtime service:

Implementation

WebSocket libraries provide the foundations to build a WebSocket service, but they aren’t a solution we can use straight out of the box. For example, how do you handle routing requests when your server is becoming overloaded? What monitoring do you put in place? What scaling policies do you have in place, and what capacity do we have to deal with large traffic spikes? Home brewed implementations often fall short for large-scale, realtime apps, a gap that usually requires a huge amount of additional development to fill.

Authentication

WebSockets also complicate authentication, as they lack the standardized mechanisms found in HTTP protocols, like HTTP headers. Also, many client-side WebSocket libraries don’t provide much support for authentication, so developers have to implement custom solutions themselves. These gaps can complicate the process of securing WebSocket connections, so it’s paramount to select an approach here that gives a good balance of security with usability.

Basic Authentication, passing API keys as URL parameters, is a simple but insecure method and should not be used for client-side applications. A more secure and flexible alternative is Token-Based Authentication like JSON Web Tokens (JWT).

How do we manage these keys though, and how do we implement token revocation? How do we ensure a token is refreshed before a client loses access to resources?

Handling failover

WebSocket connections can encounter issues that often require fallback mechanisms. Secure ports like 443 are preferred for reliability and security, but fallback transports such as XHR polling or SockJS are needed for when networks have stricter restrictions. It’s likely we would need to implement this too if you’re looking to implement your own WebSocket solution.

Power consumption and optimization

WebSocket connections are persistent, so thought must be taken when considering devices like mobiles, as the drain on battery life must be well managed. Heartbeat mechanisms like Ping/Pong frames (aka heartbeats) can help maintain connection status but may increase power consumption. We could use push notifications, waking up apps without maintaining an active connection, but they don’t provide the same kind of reliability or ordering guarantees of WebSocket streams.

Handling discontinuity

A simpler way: dedicated services

As we can see, developing and maintaining WebSockets means managing complexities like authentication, reconnections, and fallback mechanisms if we want to provide a robust realtime experience.

The considerations needed across Redis and WebSockets for an at-scale realtime service need an entire team dedicated to them. If this sounds like a headache, that’s because it is! We should avoid doing this unless we absolutely have to - whole companies have been set up to solve these problems for us.

Ably provides a globally distributed pub/sub service, with high availability and a scalable system able to meet any demand while maintaining low latencies that are crucial for realtime experiences.

There is a way we can achieve the same results with Ably while alleviating all of the issues discussed in previous sections:

WebSocket management - Ably provides pre-built WebSocket support in multiple SDKs, with SLAs that guarantee availability and reliability, and infrastructure currently handling over 1.4 billion connections daily.
Connection reliability - Ably’s SDKs are designed to maintain consistent connectivity under all circumstances. They have automatic failover, selecting the best transport mechanism and ensuring connections remain stable even during network issues. If a temporary disconnection occurs, the SDKs automatically resume connections and retrieve any missed messages. Additionally, message persistence is supported through REST APIs, allowing retrieval of messages missed due to connection failures or extended offline periods.
Scalability and capacity - Ably uses AWS load balancers to distribute WebSocket traffic efficiently across stateless servers, enabling limitless scaling to handle connections. With autoscaling policies in place, the infrastructure dynamically adjusts to meet changes in demand while maintaining a 50% capacity buffer to handle sudden spikes. Ably’s high-availability SLA guarantees 99.999% uptime, relying on redundant architecture and globally distributed routing centers.
Global distribution - Ably operates core routing datacenters in seven regions, with data persistence across two availability zones per region. This setup ensures message survivability, achieving a 99.999999% guarantee by accounting for the probability of availability zone failures and enabling quick replication. Connections are routed to the closest datacenter to minimize latency, achieving a P99 latency of less than 50ms globally.
Data distribution - Ably uses consistent hashing for channel processing, but also has dynamically scaling resources and will distribute the load as needed. This includes things like connection shedding.
Operations and maintenance - Since Ably is a fully-managed service, it has dedicated monitoring, alerting and support engineers with expertise in operating a distributed system at scale.
SDKs - 25+ SDKs for different environments.

In essence, Ably gives a comprehensive, managed Pub/Sub solution that handles the difficulties of building a distributed system with high availability, reliability, and survivability guarantees. This means developers can focus on their core application without the overhead of managing complex infrastructure like this.

Ably in practice: a code example

Let’s try using Ably’s ably-js SDK.

We can condense all of the code in the previous example into a few lines, for subscribing to our data channel:

import * as Ably from 'ably';

let client: Ably.Realtime;

async function subscribeClient() {
 client = new Ably.Realtime('xxx.yyy');

 const channel = client.channels.get('iss.data');
 try {
   await channel.subscribe((message) => {
     console.log(`Received message: ${message.data}`);
   });
 } catch (error) {
   console.error('Error subscribing:', error);
 }
}

// Handler for SIGINT
function exitHandler() {
 if (client) {
   client.close();
   console.log('The client connection with the Ably server is closed');
 }
 console.log('Process interrupted');
 process.exit();
}

// Handle SIGINT event
process.on('SIGINT', exitHandler);


(async () => {
 await subscribeClient();
})();

And publishing to our data channel:

import * as Ably from 'ably';
import axios from 'axios';

let client: Ably.Realtime;

async function publishData() {
 client = new Ably.Realtime('xxx.yyy');
 const channel = client.channels.get('iss.data');

 try {
   // Query the endpoint
   const response = await axios.get('http://api.open-notify.org/iss-now.json');

   // Get the response
   const payload = response.data;

   // Publish the payload to the Ably channel
   const payloadString = JSON.stringify(payload);
   await channel.publish({ data: payloadString });
 } catch (error) {
   console.error('Error publishing:', error);
 }
}

// Handler for SIGINT
function exitHandler() {
 if (client) {
   client.close();
   console.log('The client connection with the Ably server is closed');
 }
 console.log('Process interrupted');
 process.exit();
}

// Handle SIGINT event
process.on('SIGINT', exitHandler);

(async () => {
 await publishData();
})();

And in just a few lines of code, we have removed the need to handle the infrastructural complexities we mentioned earlier. If you’re interested in trying Ably, sign up for a free account today.

How Genius Sports slashed costs and lowered latencies for last-mile data delivery

Ably Blog — Wed, 18 Dec 2024 10:18:46 +0000

Genius Sports provides its customers with live sports and betting data across 240,000 events worldwide – each generating 10,000+ realtime messages.

Its value proposition is built on reliable, live data and on-premises infrastructure, but rapid growth brought challenges when it came to maintaining live data performance at low latency and global scale.

Genius Sports had been forced to locate ever larger, on-premises RabbitMQ clusters physically close to customers to deliver the reliable, predictable realtime performance at low latency that in-play betting demands. If data arrives late, players can bet on events that have already happened, costing Genius Sports’ customers money and damaging its reputation.

As a result, hardware capital and support costs were increasing exponentially, while infrastructure teams were constantly firefighting hardware issues.

Wondering how they overcame this challenge?

Genius Sports chose to migrate to Ably to benefit from our elastically scalable realtime infrastructure for the edge. Using Ably enabled Genius Sports to essentially hand off the heavy-lifting associated with live data processing and delivery, and gave developers the insight and flexibility they needed to innovate the core service and to offer customers more granular control of the data they subscribe to.

We spoke to Gary Williams, IT Infrastructure Team Lead at Genius Sports in our webinar - and you can now watch it on demand!

The key takeaways were:

Shifting to the cloud was key: Genius Sports chose Ably to provide realtime messaging infrastructure as a service in an early implementation of an ongoing shift to a cloud-first strategy – safe in the knowledge that the migration from RabbitMQ would require no significant refactoring of its wider architecture, coding or developer work.
The migration was straightforward: According to Gary Williams, "Overall, even with a super-cautious approach we had Ably live on our production system in less than two months. For a system handling business critical data, that is quite an achievement."
The move has freed up time and budget: The migration to Ably has eliminated annual hardware costs and freed up around 30 hours per month in RabbitMQ maintenance that Genius Sports’ developers and engineers can now refocus on service optimisation and innovation.

Chat API pricing: Comparing MAU and per-minute consumption models

Carolina Carriazo — Tue, 10 Dec 2024 15:48:47 +0000

Pricing is critical to deciding which chat API you will use - however, it can often feel like there are limited options. Whether you are looking to gradually scale a chat app or anticipate large and sudden spikes in traffic, pricing models can make or break the bank depending on your usage - and most vendors will expect you to accept the one or two industry standards.

Chat API providers predictably fall into a handful of pricing model categories. We’ll explore them in this article by explaining them, comparing them, and ultimately concluding which is best for each particular use case.

Chat APIs: Common pricing models

Chat API pricing models are designed to align with different usage patterns - like a steady user base and usage, or periodic spikes - but they also introduce trade-offs depending on an application’s scale and messaging demands. These models are generally categorized as forms of consumption-based pricing, where costs are tied to how the service is used. Let’s look at the most common pricing models in use today:

Monthly active users (MAU)

The Monthly Active Users (MAU) model is one of the most widely used pricing models in the industry. Providers like CometChat, Sendbird, Twilio, and Stream charge based on the number of unique active users per month.

You pay for each user who interacts with the chat API within a given month, regardless of the number of messages they send or receive. While this can simplify billing, it comes with the tradeoff of assuming the “typical usage” of a monthly active user. For example, an individual MAU may actually use much less active connection time or send much fewer messages than what is assumed for an average MAU. Simply put, this method is not granular.

This model is predictable for applications with small and steady user bases, since, if you’re not expecting much user volatility, it’s easy to estimate costs. But any volatility in workloads, like experiencing a brief viral period and dipping back down, can result in overpaying for peak costs (peak MAUs) in a monthly period.

For chat services operating at scale, the monthly amount spent on peak MAUs often grossly exceeds the bill for actual usage; it wastes allocated resources and money.

There is a pricing model designed to tackle these pricing issues at scale, however - and we use it at Ably.

Per-minute consumption

A per-minute consumption model goes beyond traditional consumption-based pricing by billing customers based on their actual usage of service resources—connection time, channels, and messages. This approach directly addresses the inefficiencies inherent in MAU pricing models. This isn’t a common model in the industry, but we’ve adopted it here at Ably to meet the usage needs of our customers at scale.

Per-minute consumption measures actual usage in fine-grained units, such as:

Connection minutes: The total time devices are connected.
Channel minutes: The time channels remain active.
Message events: Each message sent or received by users.

By tracking usage at this granular level, it ensures customers only pay for what they consume, without overpaying for resources they don’t use. Traffic spikes don’t necessarily lead to hugely increased costs either - the pricing is distributed across these dimensions, smoothing the overall impact. For example, livestreaming events, which may have a huge number of messages at their peak but a low number of channels, would see a more modest increase in cost than if they were billed by user count. Instead of penalizing a single metric, this approach provides greater predictability and reflects resource utilization more holistically.

Per-minute consumption also incentivizes resource optimization, such as reducing idle connections or batching messages, which can further mitigate cost surges during spikes. (Batching comes in handy when many:many chat interactions lead to an exponential increase in delivered messages, which we're implementing soon at Ably on the server side).

Popular pricing models compared

Deciding whether an MAU, message throughput, or per-minute consumption pricing model works for you depends on your use case - but if you are looking to scale a chat application to any considerable degree, as a general rule, per-minute consumption will be the best option.

MAU pricing assumes a “typical user” for billing purposes. This involves bundling resources such as connection time, message throughput, and storage into a fixed monthly fee per active user, which doesn’t accurately reflect the actual usage of the user.

Now imagine a customer operating a live event platform. They’re running a live event for two hours in the month that peaks at 50,000 users. What would the monthly prices look like between an MAU model and a per-minute consumption model?

Let’s say that the MAU model assumes that each “average” user will send 1k messages per month. While the bill comes in based on user count only, built into the cost per user is an assumption on how much one would use (in this case, a total of 50 million messages for 50k users). The MAU model then bills the whole month based on the peak of 50,000 users.

With per-minute consumption, costs reflect the actual connection time and messages used - we’ll estimate generously:

Connection time: 50,000 users × 240 minutes (accounting for pre- and post-event activity) = 12 million connection minutes.
Message volume: 50,000 users sending an average of 200 messages = 10 million messages.
Channels and channel time: let’s say 5 channels x 240 minutes = 1,200 channel minutes.

Even without specific prices to hand, we can see that billing for a typical user is inefficient in this scenario. Per-minute billing focuses on ensuring fairness and transparency for highly volatile traffic situations like these (for more information on this, Matt O’Riordan, Ably’s CEO, talks about pricing model issues in his blog post).

What does this mean in practice?

This table breaks it down:

Model	Examples	Best suited to	Challenges
Monthly Active Users (MAU)	Stream, Sendbird, Twilio	Apps with steady or low user activity	Paying for peak costs during volatile usage periods
Per-minute consumption	Ably	Apps with scalable, high-volume messaging	Requires tracking of usage metrics

Ably’s per-minute consumption model

If the per-minute consumption model we discussed above sounds promising to you, here’s some more information on how this works specifically with Ably.

At Ably, we’ve developed a pricing model designed to align more closely with the needs of realtime chat applications. Unlike traditional MAU or throughput-based models, Ably offers per-minute pricing that scales predictably and transparently with your application.

Here’s how Ably stands out:

Flexibility: Pay only for what you use, with no penalties for growing user bases or unexpected spikes in message throughput.
Scalability: Ably’s infrastructure supports billions of messages daily, with costs optimized for applications of any scale.
Transparency: Ably’s pricing eliminates the hidden costs often associated with rigid MAU or throughput models, giving you full visibility into your expenses.

Ably’s platform is built on a globally-distributed infrastructure designed for high-performance, scalable, and dependable messaging. With support for exactly-once delivery, message ordering, and <50ms global average latency, Ably ensures a seamless chat experience for users anywhere in the world.

Our Chat SDK in private beta offers fully-fledged chat features, like chat rooms at any scale; typing indicators; read receipts; presence tracking; and more. And of course, our per-minute pricing means that your consumption is as cost-effective as possible.