DEV Community: Ably Blog

The missing transport layer in user-facing AI applications

Ably Blog — Mon, 13 Apr 2026 14:17:39 +0000

Most AI applications start the same way: wire up an LLM, stream tokens to the browser, ship. That works for simple request-response. It breaks when sessions outlast a connection, when users switch devices, or when an agent needs to hand off to a human.

The cracks appear in the delivery layer, not the model. Every serious production team discovers this independently and builds their own workaround. Those workarounds don't hold once users start hitting them in production.

Here's what breaks, and what the transport layer needs to handle.

The shift that creates the problem

Simple AI applications are synchronous. User sends a message, model returns a response, done. A dropped connection restarts cleanly.

Agentic applications aren't like that. They run in a loop: perceive the user's intent, reason with the model, act by calling tools or sub-agents, and observe the result. Then they go around again until the task is done.

A research agent might loop a dozen times over several minutes, calling APIs and querying databases. The user is present throughout, watching, waiting, potentially needing to redirect. The connection might drop mid-loop, the user might switch devices, or they realize mid-stream the agent is heading the wrong way.

That's a different problem, and one HTTP streaming wasn't designed to solve. The backend surviving and the session surviving are two different things. What's missing is a layer that treats the conversation as durable state: persisting across connections, devices, and participants.

Durable execution makes the backend crash-proof. Durable sessions makes what the user actually sees crash-proof. Most teams building agentic products need both.

What breaks in production

Tokens disappear and reconnects corrupt state. HTTP streaming delivers tokens once. A dropped connection loses them. Most workarounds handle full page reloads but not tab switches or mobile backgrounding.

Worse, naive reconnect implementations replay the same output and produce duplicates: fragments, repeated tokens, or an interface in an indeterminate state. The Vercel AI SDK makes the tradeoff explicit: its resume and stop features are incompatible. You can resume a dropped stream or cancel it, but not both. A full breakdown of what resumable streaming requires at the infrastructure level is here.

Users can't see what the agent is doing. The agent is running tool calls, checking backend systems, orchestrating sub-agents. From the user's perspective it's a spinner and silence. Users abandon tasks they can't see progressing.

There's no standard mechanism for surfacing intermediate results as first-class events on the session channel.

There's no way to interrupt. Once generation starts, the user is locked out. Interruption requires bi-directional communication on the same channel simultaneously, user input arriving while agent output is still streaming, without breaking state. One company disabled user input entirely during agent responses because the backend couldn't distinguish an intentional cancel from a dropped connection.

The agent keeps working after the user has left. No signal tells the agent the user closed the tab. Compute and token costs accumulate.

Presence is a live membership set showing who is active in the session. Agents use it to pause expensive operations when nobody is there and resume when they return.

Multiple agents collide. When two specialist agents are working on the same request, every intermediate update routes through the orchestrator. The orchestrator becomes a bottleneck: when it's relaying progress it doesn't care about, the architecture starts to fight itself. The multi-agent coordination post goes deeper on how this plays out with concurrent specialist agents.

Agents fail silently. Most infrastructure has no agent health mechanism at the transport level. When an agent crashes, a presence disconnect fires immediately, rather than waiting for a timeout inferred from a dead stream. Build on the wrong signal and recovery logic breaks under real failure conditions.

Human handovers lose context. When an agent escalates, most implementations open a different interface, summarize what happened, and hope the transfer works. The user explains their problem again. A unified channel where agents and humans can both participate addresses this: the human arrives with full history and picks up mid-thread.

There are no transport-level diagnostics. Model-level tooling shows what the model decided to do. Nothing shows what happened between the agent and the user's screen: whether a message arrived, whether a reconnection worked, whether delivery stalled. Debugging a failed session means stitching together server logs that rarely reconstruct what actually happened.

What the transport layer needs to handle

Resumable streaming. Output persists in the channel, not the connection. When a client reconnects, it rejoins from its last received position with no gaps and no duplicates. Mutable messages handle retry corruption: republish to the same message ID and the client sees clean updated state, not a second copy. Vercel built a pluggable ChatTransport interface specifically to support this pattern; TanStack AI shipped a ConnectionAdapter for the same reason. The ecosystem has diagnosed the problem and built the plug-in points.

Multi-device continuity. Session state lives on the channel, not any individual client. Any device subscribing gets the same history and live updates. The session follows the user, not the connection.

23 of 26 AI platforms evaluated in recent market research have no multi-device session continuity, including ChatGPT.

Bi-directional communication on a shared channel. User input and agent output flow on the same channel simultaneously. A redirect from the user arrives as an explicit signal while the agent is mid-stream, not as an ambiguous TCP side effect. The backend can now distinguish an intentional cancel from a dropped connection.

Progress as structured events. Agent reasoning steps, tool call progress, and intermediate results should be first-class events on the channel, subscribable independently of the main response stream. Specialized agents publish progress directly. The orchestrator stops relaying events it doesn't care about.

Presence. A live membership set for users, agents, and human operators. Agents make real decisions based on it: pause when the user is gone, resume when they return. Crash detection is a presence event: when an agent disconnects, the event fires immediately.

Session-level diagnostics. Channel history serves as both the live diagnostic feed and the persistent audit record: structured, timestamped, and identity-attributed. This covers the delivery layer between agent and user, separate from model-level observability, and both surfaces matter in production.

The underlying principle

Each of these problems is tractable in isolation. Solving all of them together, without a dedicated infrastructure layer, is where engineering budget quietly disappears. None of it has anything to do with the AI product itself.

The workaround that seemed to hold breaks as soon as teams need cancellation, multi-device continuity, or human handover without a context break. The result is a growing layer of glue code that keeps teams away from the features they're actually trying to ship.

The category forming around this problem, durable sessions, is the session-layer equivalent of what durable execution did for backend workflows. The infrastructure requirement is the same: a layer built for the failure modes that actually occur, not workarounds patched onto infrastructure designed for something else.

Where Ably AI Transport fits

Ably AI Transport is a drop-in durable session layer that absorbs this complexity. Developers publish to a session. The infrastructure handles resumable streaming, multi-device continuity, presence, shared state, and bi-directional communication. No changes required to your model calls or agent orchestration.

Docs go deeper →

Why your AI response restarts on page refresh (and what it takes to prevent it)

Ably Blog — Mon, 13 Apr 2026 13:58:52 +0000

Your AI assistant is mid-sentence explaining a complex debugging strategy. The user refreshes the page. The response starts over from the beginning, or worse, vanishes entirely.

This isn't a model problem. It's a delivery problem.

What breaks

Most AI applications stream LLM responses over HTTP using Server-Sent Events or fetch streams. The connection delivers tokens in order until the response completes. If the user refreshes, closes the tab, or loses network connectivity, the stream ends. When they reconnect, there's no mechanism to resume from where they left off.

The application has two options: start the entire response over (wasting tokens and user time) or lose everything that was streamed before the disconnection (losing context the user already read).

Neither option works in production. Users refresh pages. Networks drop. Browsers crash. Mobile apps background. These aren't edge cases.

Why naive approaches fail

Client-side buffering: You can cache tokens in memory or localStorage, but this only handles intentional refreshes on the same device. It doesn't help with network interruptions, crashes, or users switching devices mid-conversation.

Response regeneration: Re-requesting the full response from the LLM costs tokens, adds latency, and often produces different output. The user sees the response change on reload, breaking continuity.

Stateless HTTP streaming: Standard SSE and fetch streams have no concept of session recovery. When the connection closes, the client has no way to tell the server "resume from token 847."

How resumable streaming actually works

The system needs three components:

Session identity: Each AI response gets a unique session ID that persists across connections. When the client reconnects, it presents this ID to resume the same logical response.

Offset tracking: The server tracks which tokens have been delivered. The client tracks which tokens it has received and rendered. On reconnect, the client requests "start from token N."

Ordered delivery with history: The transport layer guarantees token ordering and maintains a replayable history. When a client reconnects with an offset, the server resumes delivery from that point without re-invoking the LLM.

Tradeoffs

Building this yourself means managing session state, handling offset synchronisation across multiple connections, and ensuring tokens arrive in order even if network packets don't. You'll need persistent storage for token history and logic to handle race conditions when users reconnect from multiple tabs.

A concrete example

User asks an AI assistant to explain a codebase. The LLM streams 2,000 tokens over 30 seconds. At token 1,247, the user's network drops for eight seconds. Without resumability, the user sees a frozen response, then either loses everything or watches it restart.

With resumable streaming:

The client detects the disconnection and stores offset 1,247
Network recovers, client reconnects with session ID and offset
Server resumes delivery from token 1,248
User sees the response continue exactly where it stopped

The user never knows there was an interruption.

Multi-device continuity

Resumable streaming also enables conversation continuity across devices. The user starts a question on their laptop, switches to their phone, and sees the AI response pick up mid-stream. Same session ID, same offset tracking, different client.

This matters for AI workflows that span locations: research started at a desk, continued on a commute, finished in a meeting room. Without transport-level session management, each device restart loses context.

Why this matters for AI reliability

Unreliable delivery creates unreliable AI experiences. Users learn not to trust that responses will complete. They avoid asking complex questions because they might lose the answer. They stop using AI features on mobile networks.

Fixing this isn't about better models or smarter prompts. It's about ensuring delivery is as dependable as the intelligence behind it.

Next steps

If you're building AI features where responses take more than a few seconds, or where users might switch devices or encounter network issues, you need resumable streaming. You can build session management and offset tracking yourself, or use infrastructure like Ably AI Transport that handles it for you.

Either way, design for reconnection from day one. Your users will refresh. Your network will drop. Production isn't a stable connection.

Resume tokens and last-event IDs for LLM streaming: How they work & what they cost to build

Ably Blog — Mon, 13 Apr 2026 13:32:33 +0000

When an AI response reaches token 150 and the connection drops, most implementations have one answer: start over. The user re-prompts, you pay for the same tokens twice, and the experience breaks.

Resume tokens and last-event IDs are the mechanism that prevents this. They make streams addressable – every message gets an identifier, clients track their position, and reconnections pick up from exactly where they left off. The concept is straightforward. The production scope is not: storage design, deduplication, gap detection, distributed routing, and multi-device continuity all follow from the same first decision.

How resume tokens actually work

Resumable streaming has four moving parts.

Message identifiers. Every token or message gets a sequential ID when published – monotonically increasing, so each new message has a higher ID than the previous one.

Client state. The client tracks the ID of the last message it successfully received. In a browser, that's typically held in memory or local storage. On mobile, it needs to survive app backgrounding.

Reconnection protocol. When the connection drops, the client presents the last ID it saw. The server responds with everything that arrived after that ID, then transitions to live streaming.

Catchup delivery. The client receives missed messages in order before live tokens resume. The seam should be invisible.

The stream itself becomes the source of truth. The client doesn't reconstruct what it missed – the stream delivers it.

What SSE's Last-Event-ID header gives you

Server-Sent Events implements this natively. When an SSE connection drops, the browser automatically includes a Last-Event-ID header on reconnection. The server sees which event the client last received and resumes from there.

The browser handles reconnect logic. Application code doesn't change between initial connection and reconnection. For the happy path – stable connection, single device, short responses – SSE with Last-Event-ID works well.

The problems start at the boundary of what SSE can do.

SSE is unidirectional and HTTP-only. It has no native history beyond what you implement server-side. It doesn't handle bidirectional messaging, so live steering – users redirecting the AI mid-response – requires a separate channel. On distributed infrastructure, a reconnecting client may reach a different server instance that has no record of the original session. SSE handles the reconnect handshake. Everything else – distributed state, per-instance routing, multi-device history – is still your problem. For use cases that need bidirectional messaging, WebSockets vs SSE covers the tradeoffs in detail.

Building resume into WebSockets

WebSockets don't include resume semantics. When a WebSocket closes, the connection is gone. Reconnecting creates a new socket with no knowledge of the previous one.

Building resume on WebSockets means building all of it yourself:

Session IDs generated at stream start, stored server-side, presented by the client on reconnection. Message IDs assigned sequentially. Server logic to look up a session, find the position, replay history, then transition to live. Buffer management to decide how long to keep messages for sessions that haven't reconnected yet. Cleanup logic to expire stale sessions without cutting off legitimate reconnects.

Each piece is straightforward in isolation. The edge cases are where the weeks go.

The storage problem teams underestimate

Token-level storage is where most implementations hit an unexpected wall.

A 500-word response generates roughly 625 tokens. If you store each token as a separate record, loading one response means retrieving 625 records. A conversation with 20 exchanges is 12,500 records. Multiply across thousands of concurrent users and history retrieval becomes the performance bottleneck.

This matters because history retrieval is on the critical path for multi-device continuity. When a user switches from laptop to phone, the speed of catchup determines whether the experience feels continuous or broken.

The more practical model is to treat each AI response as a single logical message and append tokens to it rather than publishing them individually. Clients joining mid-stream receive the full message so far, then get new tokens as they arrive. One record per response instead of hundreds.

Duplicates and gaps: the two failure modes that break trust

Duplicates happen when the connection drops after the client receives a message but before the acknowledgement reaches the server. On reconnect, the server doesn't know whether to replay that message. Without deduplication logic, the client renders the same token twice.

The fix is using message IDs as deduplication keys on the client – straightforward in principle, but it needs to survive page reloads and work across tabs.

Gaps happen when sequential IDs arrive out of order or not at all. If a client receives message 153 after 150, messages 151 and 152 are missing. Without gap detection, the client silently renders an incomplete response. With it, you need logic to request missing messages, decide what to do if they can't be retrieved, and handle the state when the client gives up waiting.

Both failure modes are rare enough to be invisible in testing. Both surface under real network conditions: mobile handoffs, flaky WiFi, corporate proxy timeouts. The first time you see them is usually a support ticket.

What distributed deployment adds

A single-server implementation can tie session state to process memory and mostly work. As soon as you run multiple instances – which you will, for reliability and scale – a routing problem appears.

A client that connected to instance A reconnects to instance B. Instance B has no record of the session. Your options: route all reconnections back to the originating instance (a pinning strategy that creates hotspots and defeats the purpose of multiple instances), or store session state in shared infrastructure that all instances can read.

Shared session storage means Redis or equivalent: network round-trips on reconnect, cache invalidation logic, and failure handling when the cache is unavailable. This is solvable. It's also not in the first implementation.

The multi-device gap

Multi-device continuity is where connection-oriented design hits a wall.

When state lives in the connection – or in server memory tied to that connection – device switching loses context. The phone doesn't know what the laptop received. Without a shared source of truth for message history that any device can query, each reconnect from a new device is a new session.

True multi-device continuity requires decoupling state from connections entirely. The conversation lives in a channel or persistent store. Devices subscribe and catch up rather than resuming a connection.

This is a different architectural model than resuming an HTTP stream. For most teams, that realisation arrives after the first implementation is already in production.

When resumable streaming matters most

Not every streaming application needs this. For short-lived, single-session interactions on stable connections, standard HTTP streaming is fine.

Resume becomes critical under specific conditions:

Mobile clients handle network handoffs between WiFi and cellular constantly. Each one is a potential disconnection.

Long responses – anything over 30 seconds – have a high probability of encountering a transient failure.

Multi-device usage means the conversation needs to live in a channel, not a connection.

Multi-agent systems, where several agents publish updates to a shared channel. A reconnecting client needs to catch up on everything all agents published, not just the primary response thread.

The alternative is forcing users to restart on every interruption. That breaks trust fast, and the cost compounds on longer or more complex tasks where restarting is most painful.

What you're actually signing up for to build this

Teams that have shipped resumable streaming in production describe a consistent arc: the first implementation takes a week, the edge cases take a month, and cross-device reliability is still not fully solved six months later.

The full scope of a production-grade build: session management, message storage with efficient retrieval by ID range, client-side deduplication, gap detection, distributed routing, cache invalidation, buffer expiry, and monitoring to surface issues you can't reproduce locally.

Good transport infrastructure handles duplicates and gaps automatically. Application logic shouldn't need to check for either – that's the infrastructure's job.

Build vs infrastructure

Building resumable streaming yourself is a reasonable choice if you have a stable team, time to maintain it, and no multi-device or distributed requirements.

It's a harder choice than the SSE documentation makes it look. One team described spending several weeks on custom session management and still not fully solving cross-device reliability. The problems weren't obvious in the design phase – they appeared under mobile network conditions, under load, and when users did things the system wasn't built to handle.

The alternative is transport infrastructure that implements resume as part of the platform. You keep control of your LLM, prompts, and application logic. Session continuity, offset management, ordered delivery, and multi-device state become infrastructure concerns rather than application concerns.

Both paths are defensible. The costs of building are real and most of them are invisible until the first deploy.

Streaming responses between AI agents and clients? Ably AI Transport includes resumable token streaming, automatic replay, and channel-based delivery with guaranteed ordering. Docs go deeper.

Appends for AI apps: Stream into a single message with Ably AI Transport

Ably Blog — Thu, 26 Feb 2026 12:05:30 +0000

Streaming tokens is easy. Resuming cleanly is not. A user refreshes mid-response, another client joins late, a mobile connection drops for 10 seconds, and suddenly your "one answer" is 600 tiny messages that your UI has to stitch back together. Message history turns into fragments. You start building a side store just to reconstruct "the response so far".

This is not a model problem. It's a delivery problem

That's why we developed message appends for Ably AI Transport. Appends let you stream AI output tokens into a single message as they are produced, so you get progressive rendering for live subscribers and a clean, compact response in history.

The failure mode we're fixing

The usual implementation is to stream each token as a single message, which is simple and works perfectly on a stable connection. In production, clients disconnect and resume mid-stream: refreshes, mobile dropouts, backgrounded tabs, and late joins.

Once you have real reconnects and refreshes, you inherit work you did not plan for: ordering, dedupe, buffering, "latest wins" logic, and replay rules that make history and realtime agree. You can build it, but it is the kind of work that quietly eats weeks of engineering time.

With appends you can avoid that by changing the shape of the data. Instead of hundreds of token messages, you have one response message whose content grows over time.

The pattern: create once, append many

In Ably AI Transport, you publish an initial response message and capture its server-assigned serial. That serial is what you append to.

It's a small detail that ends up doing a lot of work for you:

const result = await channel.publish({ name: 'response', data: '' });
const { serials: [msgSerial] } = result;

Now, as your model yields tokens, you append each fragment to that same message:

if (event.type === 'token') {
  channel.appendMessage({ serial: msgSerial, data: event.text });
}

What changes for clients

Subscribers still see progressive output, but they see it as actions on the same message serial. A response starts with a create, tokens arrive as appends, and occasionally clients may receive a full-state update to resynchronise (for example after a reconnection).

Most UIs end up implementing this shape anyway. With appends, it becomes boring and predictable:

switch (message.action) {
case 'message.append': renderAppend(message.serial, message.data); break;
case 'message.update': renderReplace(message.serial, message.data); break;
}

The important difference is that history and realtime stop disagreeing, without your client code doing any extra work. You render progressively for live users, and you still treat the response as one message for storage, retrieval, and rewind.

Reconnects and refresh stop being special cases

Short disconnects are one thing. Refresh is the painful case, because local state is gone and to stream each token as a single message forces you into replaying fragments and hoping the client reconstructs the same response.

With message-per-response, hydration is straightforward because there is always a current accumulated version of the response message. Clients joining late or reloading can fetch the latest state as a single message and continue.

Rewind and history become useful again because you are rewinding meaningful messages, not token confetti:

const channel = realtime.channels.get('ai:chat', {
  params: { rewind: '2m' }
});

Token rates without token-rate pain

Models can emit tokens far faster than most realtime setups want to publish. If you publish a message per token, rate limits become your problem and your agent has to handle batching in your code.

Appends are designed for high-frequency workloads and include automatic rollups. Subscribers still receive progressive updates, but Ably can roll up rapid appends under the hood so you do not have to build your own throttling layer.

If you need to tune the tradeoff between smoothness and message rate, you can adjust appendRollupWindow. Smaller windows feel more responsive but consume more message-rate capacity. Larger windows batch more aggressively but arrive in bigger chunks.

Enabling appends

Appends require the "Message annotations, updates, appends, and deletes" channel rule for the namespace you're using. Enabling it also means messages are persisted, which affects usage and billing.

Why this is a better default for AI output

If you are shipping agentic AI apps, you eventually need three things at the same time:

streaming UX
history that's usable
recovery that does not depend on luck

Appends are how you get there without building your own "message reconstruction" subsystem. If you want the deeper mechanics (including the message-per-response pattern and rollup tuning), the AI Transport docs are the best place to start.

Realtime steering: interrupt, barge-in, redirect, and guide the AI

Ably Blog — Mon, 09 Feb 2026 09:58:06 +0000

Start typing, change your mind, redirect the AI mid-response. It just works. That is the promise of realtime steering. Users expect to interrupt an answer, correct its direction, or inject new instructions on the fly without losing context or restarting the session. It feels simple, but delivering it requires low-latency control signals, reliable cancellation, and shared conversational state that survives disconnects and device switches. This post explores why expectations have shifted, why today's stacks struggle with these patterns, and what your infrastructure needs to support proper realtime steering.

What's changing

AI tools are moving beyond static, one-turn interactions. Users expect to interact dynamically, especially in chat. But most AI systems today force users to wait while the assistant responds in full, even if it's off-track or no longer relevant. That's not how human conversations work.

Expectations are shifting toward something more natural. Users want to jump in mid-stream, adjust the AI's course, or stop it altogether. These patterns (barge-in, redirect, steer) are becoming table stakes for responsive, agentic assistants.

What users want, and why this enhances the experience

Users want to stay in control of the conversation. If the AI starts drifting, they want to say "stop" or "try a different angle" and get an immediate course correction. They want to guide the assistant's direction without breaking the flow or starting over.

This improves trust, keeps sessions on-topic, and avoids wasted time. It also brings AI interactions closer to how real collaboration works: iterative, reactive, fast.

Users now expect a few technical behaviours as part of that experience:

Responses can be interrupted in real time
New instructions are applied mid-stream without reset
The AI keeps context and adjusts without losing the thread

Why realtime steering is proving hard to build

Most AI systems treat generation as a one-way stream. Once the model starts producing tokens, the system just plays them out to the client. If the user wants to interrupt or change direction, the only real option is to cancel and resend a new prompt - often from scratch. That's because most systems today cannot support mid-stream redirection because their underlying communication model does not allow it.

Stateless HTTP cannot carry steering signals

Traditional request–response models push output in one direction only. Once a long-running generation begins, there is no reliable way to send control signals back to the server. Cancelling or redirecting usually means tearing down the stream and starting again.

Browser-held state breaks immediately

Most apps keep the state of an active generation in the browser. If the user refreshes or switches device, the in-flight response loses continuity. Any client-side steering logic tied to that state vanishes too, which forces a full reset.

Backend models often run without shared conversational state

If the orchestration layer is not tracking what the AI is currently doing, it cannot apply corrections cleanly. The model receives a brand-new prompt instead of a context-preserving instruction layered onto an active task.

The default stack was never designed for low-latency control loops

Steering requires coordinated signalling between UI, transport, orchestration, and model inference. That means ordering guarantees, durable state, and fast propagation of control messages. Without these, the AI continues generating tokens after a user says stop, causing confusion and wasted compute.

Steering mid-stream looks like a simple UX gesture. It is not. It is a distributed-systems problem sitting under a conversational interface.

Why you need a drop-in AI transport layer for steering

Delivering realtime control requires more than token streaming. It requires a transport layer that keeps context alive, supports low-latency bidirectional messaging, and ensures that user instructions and model output remain synchronised.

Bi-directional, low-latency messaging

Client-side signals such as "stop" or "try this instead" must reach the backend quickly and reliably. WebSockets or similar long-lived connections make this possible by enabling client-to-server control while the AI continues to stream output.

Reliable interrupt and cancellation primitives

Stopping generation must be instant and clean. The transport must carry cancellation events with ordering guarantees so the backend halts inference exactly where intended, without corrupting state.

Session continuity

The system needs persistent session identity so instructions and outputs are tied to the same conversational thread. Redirection should extend the session, not rebuild it.

Presence and focus tracking

If users have multiple tabs or devices open, the system needs to know where instructions are coming from. Steering messages must route to the correct active session without collisions.

Realtime steering relies on a transport layer designed for conversational control, not just message delivery.

How the experience maps to the transport layer

User experience desired	Required transport layer features	Underlying technical implementation
Interrupt and redirect responses in real time	Bi-directional messaging	WebSocket-based channels enabling client-to-server signals during output
Cancel generation cleanly	Interrupt primitives	Server-side control hooks to stop model inference and close stream pipelines
Preserve continuity after steering	Session continuity	Persistent session or conversation IDs with context caching
Update response direction on the fly	Dynamic state sync	Shared state model where new input is merged into active conversational context
Steer across devices	Identity-aware multiplexing	Fan-out model updates across all user sessions in sync

Realtime steering for AI you can ship today

You don't need a new architecture to support real-time steering, cancellation, or recovery. You need a transport layer that can keep the session alive, deliver messages in order, and preserve state across disconnects. Ably AI Transport provides those foundations out of the box, so you can build controllable, resilient AI interactions without rebuilding your entire stack.

Sign-up for a free account and try today.

Why orchestrators become a bottleneck in multi-agent AI published

Ably Blog — Tue, 03 Feb 2026 12:42:23 +0000

Complex user tasks often need multiple AI agents working together, not just a single assistant. That's what agent collaboration enables. Each agent has its own specialism - planning, fetching, checking, summarising - and they work in tandem to get the job done. The experience feels intelligent and joined-up, not monolithic or linear. But making that work means more than prompt chaining or orchestration logic. It requires shared state, reliable coordination, and user-visible progress as agents branch out and converge again. This post explores what users now expect, why traditional infrastructure falls short, and how to support truly collaborative AI systems.

What's changing?

The shift from simple question-response to collaborative AI experiences goes beyond continuity or conversation. It's about delegation. Users are starting to expect AI systems that can take a complex request and break it down behind the scenes. That means not one big model doing everything, but a network of agents, each focused on a part of the task, coordinating to deliver a coherent outcome. We've seen this in tools like travel planners, research assistants, and document generators. You don't just want answers, you want progress, structure, and coordination you can see. The AI system shouldn't just feel like a chat thread, it should feel like a team quietly getting on with things while keeping you informed.

What users want, and why this enhances the experience

When users interact with a system powered by multiple agents, they want to feel the benefits of parallelism without the overhead of managing complexity. If one agent is fetching flight data, another handling hotel options, and a third reviewing visa requirements, the user doesn't care about the internal plumbing. They care that their travel plan is evolving visibly and coherently. They want to see that agents are working, understand what's happening in realtime, and be able to intervene or revise things if needed.

Crucially, users expect the state of their task to reflect reality, not just the conversation. If they change a hotel selection manually, the system should adapt. If an agent crashes or stalls, the UI should show it. The value isn't just in faster results, it's in reliability, transparency, and the sense that multiple agents are genuinely collaborating, with each other and with the user - toward a shared goal.

To deliver this, agent systems need to stay in sync. State needs to be shared across agents and user sessions. Progress needs to be surfaced incrementally, not hidden behind a final answer. And context must be preserved so agents don't overwrite or duplicate each other's work. That's what turns a bunch of isolated model calls into a coordinated assistant.

Why this is proving challenging

Multi-agent systems can work today, but the default pattern most tools push you toward is an orchestration-first user experience. Even when multiple agents are running behind the scenes, their activity is typically funnelled through a single orchestrator that becomes the only "voice" the user can see. That hides useful progress, creates a single bottleneck for updates, and limits how fluid the experience can feel.

That's because traditional LLM interfaces assume a single stream of input and a single stream of output. Orchestration frameworks may invoke multiple agents in parallel, but the UI still tends to expose a linear, synchronous workflow: the orchestrator collects results, then reports back. If the user changes direction mid-process, or if an agent needs to react immediately to something in shared state, you're often forced back into "wait for the orchestrator" loops.

The underlying infrastructure assumptions reinforce this. HTTP request/response cycles work well when one component is responsible for coordinating everything, but they make it awkward for multiple agents to maintain an ongoing, direct connection to the user and to shared context. Token streaming helps, but it usually represents one agent's output to one user - not concurrent updates from a group of agents reacting in real time to a changing state.

Ultimately, the challenge isn't that orchestration fails. It's that it constrains app developers. Most systems don't give you fine-grained control over which agent communicates what, when, and how, or an easy way to reflect multi-agent activity directly in the user experience. To build confidence and responsiveness, clients need to know which agents are active, what they're doing, and how that activity relates to the shared, realtime session context - without everything having to be mediated by a heavyweight orchestrator.

Why you need a drop-in AI transport layer

To make multi-agent collaboration work in practice, you need infrastructure that handles concurrency, coordination, and visibility - not just messaging.

The transport layer must support persistent, multiplexed communication where multiple agents can publish updates independently while still participating in the same user session. That gives app developers fine-grained control over the user experience: which agents speak to the user, when they speak, and how progress is presented. Orchestrators can still exist, but they don't have to mediate every user-facing update.

State synchronisation is non-negotiable

Structured data, like a list of selected hotels or the current trip itinerary, should live in a realtime session store that agents and UIs can both read from and write to. This creates a single source of truth, even when updates happen asynchronously, across devices, or outside the chat interface

Presence adds another layer of confidence

When users see which agents are online and working, it sets expectations and builds trust. If an agent goes offline, the system should detect it, not leave the user guessing. This becomes even more important as these systems scale up in production environments where reliability is critical.

Interruption handling rounds it out

Users will change their minds mid-task. Your system needs to respond without the orchestrator agent tearing down and restarting everything. That means listening for user input while processing, canceling or rerouting tasks, and updating the shared state cleanly so individual agents can pick up where they left off or switch strategies on the fly.

How the experience maps to the transport layer

User experience desired	Required transport layer features	Underlying technical implementation
Visible, concurrent agent progress	Multiplexed pub/sub channels	Multiple agents publish progress updates to a shared realtime channel the UI subscribes to
Shared, up-to-date task state	Structured state synchronisation	Use of shared shared session state with clear schemas to reflect selections, status, and choices
Seamless agent-to-agent coordination	Out-of-band messaging support	Internal HTTP APIs or RPC protocols between agents, decoupled from user-facing updates
Awareness of system activity and health	Presence tracking	Agents register presence on connection and broadcast availability or error states
Graceful handling of mid-task changes	Event-driven state updates and recovery	Listen to user changes in shared state and cancel or adjust in-flight work accordingly

Making it work today

Multi-agent collaboration is already happening in planning tools, research systems, and internal automation workflows. The models are not the limiting factor. The hard part is the infrastructure that keeps agents in sync, shares state reliably, and exposes progress to users in real time.

Ably AI Transport gives you the infrastructure needed to support this pattern. Realtime channels, shared state objects, presence, and resilient connections provide the foundations for agents that coordinate reliably and surface their work as it happens. No rebuilds, no custom multiplexing, no home-grown state machinery.

Sign-up for a free developer account and try it out.

Multi-agent AI systems need infrastructure that can keep up

Ably Blog — Fri, 30 Jan 2026 10:49:09 +0000

An Ably AI Transport demo

When you're building agentic AI applications with multiple agents working together, the infrastructure challenges show up fast. Agents need to coordinate, users need visibility into what's happening, and the whole system needs to stay responsive even as tasks branch out across specialised workers.

We built a multi-agent travel planning system to understand these problems better. What we learned applies well beyond holiday booking.

The coordination problem

The demo uses four agents: one orchestrator and three specialists (flights, hotels, activities). When a user asks to plan a trip, the orchestrator delegates sub-tasks to the specialists. Each specialist queries data sources, evaluates options, and reports back. The orchestrator synthesises everything and presents choices to the user.

This mirrors how most teams are actually building agentic systems. You don't build one massive agent that tries to do everything. You build focused agents, give them specific tools, and coordinate between them.

The infrastructure question is: how do you keep everyone (the agents and the user) synchronized as work happens?

Why streaming alone isn't enough

Token streaming solves part of this. The orchestrator can stream its responses back to the user so they're not waiting for complete answers. That's table stakes now for any AI interface.

But streaming tokens from the orchestrator is only part of the problem. Users want visibility into the behaviour of each specialised agent – through their own token streams, structured updates like pagination progress, or the current reasoning of an agent working through a task.

Prompt: Plan a weekend trip to a nearby city

In our AI Transport demo, we also use Ably LiveObjects to publish progress updates from each specialist agent. The user sees which agent is active (tracked via presence), what it's querying, and how much data it's processing. These aren't logs or debug output. They're structured state updates that drive the UI. The agent even decides how to represent its progress to the user, taking raw database query parameters and turning them into natural language descriptions through a separate model call.

This requires infrastructure that can handle multiple publishers updating different parts of the shared state concurrently. The flight agent publishes its progress. The hotel agent publishes its progress. The orchestrator streams tokens (and it doesn't need to care about intermediate progress updates from the specialized agents). All on the same channel, all staying in sync.

Agent searches for flights and hotels based of user's criteria

State that reflects reality, not just conversation

Chat history creates a limited view of what's actually happening. If a user changes their mind, deletes a selection, or modifies something outside the conversation thread, the agent needs to know about it.

We use Ably LiveObjects to maintain the user's current selections (flights, hotels, activities) and agent status. This creates a source of truth that exists independently of the conversation. The orchestrator can query this state directly through a tool call, even if nothing in the chat history explains the change.

The interesting bit: agents can subscribe to changes in this data, so they see updates live. While you could store this in a database and have agents query it via tool calls, the ability to subscribe means agents can react to user context in real time (what the user is doing in the app, data they're manipulating, configuration changes they're making).

When the user asks "what's my current itinerary?", the agent doesn't rely on conversation history. It checks the actual state. If the user deleted their flight selection, the agent sees that immediately.

This separation matters more as systems get complex. The conversation is one interface to the system. The actual state (what's selected, what's in progress, what's completed) needs to exist independently. Agents, users, and other parts of your system all need reliable access to current state, not a reconstruction from message history.

Agent offers hotel options while remembering flight choice

Synchronising different types of state

Not all state is created equal, and your infrastructure needs to handle different patterns:

Structured, bounded state works well with LiveObjects. Progress indicators (percentage complete, items processed), agent status (online, processing, completed), user selections, and configuration settings all have predictable size limits. Clients can subscribe to changes and re-render UI efficiently. Agents can read current state without parsing through message history.

Unbounded state like full conversation history, audit trails, or complete reasoning chains still belongs in messages on a channel. You're appending to a growing log rather than updating bounded data.

Bidirectional state synchronization enables richer interactions. You can sync agent state to users (progress updates, ETAs, task lists), let users configure controls for agents (settings, preferences, constraints), and give agents visibility into user context (where they are in the app, what they're doing, what data they're viewing). Each of these can use structured data patterns for efficient synchronization.

The key is knowing which pattern fits which data, and having infrastructure that supports both.

Decoupling internal coordination from user-facing updates

The agents in our demo communicate with each other over HTTP using agent-to-agent protocols. That's appropriate for internal coordination. It's synchronous, it's request-response, it follows established patterns.

The user-facing updates go over Ably AI Transport. That's where you need state synchronization and the ability for multiple publishers to update different parts of the UI concurrently.

This decoupling matters. Each agent can independently decide how to surface its progress updates and state to the user, while the user maintains a single shared view over updates from all agents.

We also let specialist agents write directly to LiveObjects, bypassing the orchestrator. When the flight agent has progress to report, it writes it. The user sees it. The orchestrator never touches that data (it only needs the final result). This avoids additional coordination and keeps the architecture simpler.

Handling interruptions

Users change their minds. They interrupt. They refine requests mid-task. Your infrastructure needs to support this without rebuilding everything from scratch.

In the demo, you can barge in and interrupt the agent while it's working. The system detects the new input, cancels the in-flight task, updates the state, and kicks off a new search. The UI shows the cancellation, the new request, and the new progress, all without breaking the conversation.

This works because state updates are events on a channel. The agents listen for new user input even while they're processing. When they see it, they can decide whether to cancel current work, adapt it, or complete it first. The infrastructure doesn't dictate this logic (it enables it).

Agent then helps user select activities to do on trip

What presence actually tells you

Before any interaction starts, the UI shows which agents are online. This comes from Presence. Each agent enters presence when it starts up and updates it as its status changes.

Presence serves multiple purposes. Agents can see the online status of users and take action if a user goes offline (canceling tasks or queuing notifications – essential from a cost optimization perspective). In multi-user applications, users can see who else is online in the conversation. And for your operations team, it's observability built into the architecture. This answers a basic question for users: is this system actually working right now?

The enterprise patterns that emerge

This travel demo is deliberately simple, but the patterns map directly to enterprise use cases:

Research workflows where multiple agents pull from different data sources (financial databases, customer records, market data) and coordinate findings. Users need to see progress across all of them, not wait for a final answer.

Document generation where one agent structures the outline, others fill in sections, another handles compliance checks. The state (which sections are complete, which are being reviewed, what's been approved) needs to stay synchronized as different agents work in parallel.

Customer support routing where classification agents determine issue type, specialist agents handle resolution, and orchestration agents manage escalation. Status updates need to flow to support reps, customers, and dashboards in real time.

The common thread: multiple agents, concurrent work, shared state, and humans who need visibility and control. The infrastructure that makes a travel planner responsive and reliable is the same infrastructure that makes these systems work.

Labelled screenshot of AI Travel Agent's moving parts

What this requires from infrastructure

You need a reliable transport layer that allows concurrent agents and clients to communicate in realtime. This isn't just about pub/sub – it's about robust infrastructure, high availability, and guaranteed delivery.

You need state synchronisation that works for both structured data and message logs. Having access to both patterns depending on your needs is critical–bounded state objects for UI updates and configuration, unbounded message streams for conversation history and audit trails.

You need presence so you know what's actually online and available. You need connection recovery so users don't lose context when networks flicker.

Most importantly, you need this to work at the edge – in browsers and mobile apps, not just between backend services. That's where your users are. That's where responsiveness matters. The transport layer needs to be robust enough to handle the reality of client connectivity: spotty networks, mobile handoffs, browser tabs backgrounded and resumed.

The hard part of building multi-agent systems isn't the LLMs. The models are getting better every month. The hard part is the coordination, the state management, the visibility, and the reliability as these systems get more complex.

This is why we built AI Transport. We saw teams struggling with these exact problems: cobbling together WebSocket libraries, building their own state synchronization, dealing with reconnection logic, and watching their systems break under the messiness of real client connectivity. AI Transport gives you the infrastructure layer these systems need, built on Ably's proven reliability at scale, so you can focus on your agents instead of your transport layer.

Building agentic AI experiences? You can ship it now

This demo was built with Ably AI Transport. It's achievable today. You don't need to rebuild your stack to make it happen.

Ably AI Transport provides all you need to support persistent, identity-aware, streaming AI experiences across multiple clients. If you're working on agentic products and want to get this right, improve the AI UX, we'd love to talk.

Anticipatory customer experience: How realtime infrastructure transforms CX

Ably Blog — Wed, 28 Jan 2026 10:27:17 +0000

We're entering a new era of anticipatory customer experience – one that's not just reactive, not just responsive, but truly predictive. In this new model, systems don't wait for friction to appear; they recognise signals early and step in before the user ever feels a slowdown or moment of uncertainty. The bar has shifted: customers now expect brands to predict their needs and act before friction even surfaces. It's a fundamental rewiring of the relationship between companies and the people they serve.

This shift toward predictive customer experiences isn't hypothetical. Anticipatory experiences are happening now, powered by realtime data infrastructure that moves companies from playing catch-up to staying ahead. Think of it as the Age of Anticipation – where realtime signals, reliability, and adaptability form the core of modern CX design.

Anticipatory CX isn't magic, it's just realtime infrastructure done right.

So, if you're building next-generation CX or AI-powered agentic systems, this article outlines the architectural groundwork required to make anticipation real.

What is anticipatory customer experience?

Anticipatory customer experience uses realtime data infrastructure to predict and address customer needs before friction occurs. Unlike reactive support that waits for problems, anticipatory CX leverages continuous data streams, event-driven patterns, and predictive signals to intervene proactively, turning unknowns into reassurance.

Why realtime infrastructure matters for CX: Realtime infrastructure enables the continuous flow of customer signals needed for prediction. Without it, systems rely on stale, batch-processed data that kills foresight. Companies like Doxy.me and HubSpot use realtime platforms to anticipate confusion, delays, and churn risk before customers experience frustration.

From reactive to anticipatory: Why realtime data infrastructure powers predictive CX

Anticipation starts with having the right information at the right moment. But prediction requires fresh, realtime signals flowing continuously through your systems.

The healthcare sector illustrates this shift perfectly. Doxy.me, a telehealth platform trusted by hundreds of thousands of providers, faced a critical challenge: how do you anticipate patient confusion before it derails a virtual appointment? Their answer was "teleconsent" – a feature where healthcare providers walk patients through consent forms collaboratively, in real time.

As the patient reads, fills in fields, and types responses, the provider sees every change as it happens. No refresh required. No lag. No wondering if the patient is stuck on question three. The system detects hesitation patterns and enables providers to intervene before confusion becomes abandonment. This is anticipatory CX in action – predicting friction points and addressing them before they escalate.

But building this required infrastructure that could handle the continuous flow of patient interactions without introducing the very friction it was meant to eliminate. "The more that I can get my team to focus on healthcare business logic and less to focus on infrastructural data synchronisation, the better," explains Heath Morrison from Doxy.me. "Anything that provides higher level APIs to get us more in that space – and not be specialised in the stuff you guys should specialise in – is appealing and valuable to us."

By rebuilding their realtime stack on reliable infrastructure, Doxy.me achieved a 65% cost reduction while transforming their system from a liability into a core strength. Read the full Doxy.me case study →

Retailers are doing similar work, spotting churn risk in realtime and intervening with targeted offers or support before the customer clicks away. Financial services companies are shifting from asking "what happened?" to "what's about to happen?" These aren't reactive fixes. They're anticipatory moves that change outcomes – but only when the underlying data infrastructure can keep pace.

Realtime infrastructure like Ably's makes this possible – it's the unseen layer that ensures systems receive the continuous stream of signals they need to predict accurately, without lag or data loss.

Industries using anticipatory CX

Healthcare: Telehealth platforms use realtime infrastructure to anticipate patient needs, showing "doctor joining now" before patients wonder if something's wrong.

Financial Services: Banks predict fraud patterns and alert customers to unusual activity before money moves.

Retail: E-commerce platforms spot abandonment signals and intervene with targeted offers before checkout failure.

Logistics: Delivery services flag delays and update ETAs before customers start refreshing tracking pages.

Building trust through realtime customer engagement: The infrastructure foundation

Trust is built in moments of uncertainty. And anticipation? It turns unknowns into reassurance.

Think about the last time you booked a rideshare or waited for a delivery. The difference between a company that leaves you guessing and one that proactively updates you – "your driver is two minutes away," "slight delay, new ETA: 3:47pm" – is the difference between anxiety and confidence. Realtime anticipation doesn't just inform, it reassures.

Telehealth platforms have figured this out. When patients see "doctor joining now" before they've even begun to wonder if something's wrong, it changes the entire experience. Logistics companies that flag delays before customers start refreshing tracking pages are doing the same thing: reducing friction before it becomes frustration.

But there's a flip side: when realtime systems fail, trust erodes faster than it built up. A phantom notification, a delayed update, an inaccurate prediction – these aren't just technical hiccups. They're credibility problems. Reliability isn't a nice-to-have, it's the foundation. When customers cite Ably's five-plus years without a global outage, they're not celebrating uptime for its own sake. They're describing the baseline that makes anticipation possible at scale. View Ably's live uptime status.

Ably exists to be that foundation. The reason trust can scale across millions of interactions, without companies needing to worry about the underlying infrastructure failing at the worst moment.

Core technologies behind anticipatory CX

Core technology	Benefit
Realtime pub/sub messaging	WebSocket-based event distribution for instant signal propagation
Event-driven architecture	Composable, adaptive systems that respond to customer signals
Predictive analytics	AI-powered interpretation of continuous data streams
Continuous data streams	Sub-6.5ms message delivery latency without polling
Fault-tolerant infrastructure	99.999% uptime requirements for maintaining trust

Future-proofing customer experience: Event-driven architecture for anticipatory CX

To anticipate effectively, your CX stack needs to evolve as fast as your customers' expectations do. Rigid, monolithic architectures can't keep up with new signals, emerging channels, or changing customer behaviors. The future belongs to composable, event-driven systems.

Doxy.me's evolution illustrates this perfectly. They built their realtime features organically – using PubNub to handle presence detection and state synchronisation, all ephemeral data that disappeared after each session. But as they planned their next phase, they hit a wall: they needed persistence. The ability to decouple patient workflows from video calls, support richer collaboration, maintain state across sessions, and plug in new capabilities without rebuilding their entire stack. They prototyped with Convex and loved the developer experience, but needed production-grade infrastructure that could slot into their Node/TypeScript/Postgres/AWS environment.

Event-driven architectures make this kind of evolution possible. You can layer in predictive capabilities, plug in new communication channels, or add analytics tools – all without tearing everything down and starting over. One enterprise CX leader described it this way: "We used to dread adding new functionality. Now we think in terms of what events we need to listen for and what actions we want to trigger. It has completely changed our velocity."

Ably enables this kind of interoperability – CRMs, chat systems, analytics tools, customer-facing applications all publishing and subscribing to customer events in real time. WebSockets and pub/sub patterns ensure consistent, low-latency communication across every channel, without developers having to reinvent transport logic for each integration. It's the connective tissue that makes anticipatory systems work at scale.

But more moving parts do mean more complexity. Companies need governance frameworks and resilience planning to ensure their adaptive architectures don't become fragile ones. The ones succeeding here aren't necessarily the ones with the newest tech – they're the ones who've built systems that can absorb change without breaking.

The Age of Anticipation is composable. Adaptive, event-driven architecture is what makes foresight scalable.

How to implement anticipatory CX

1. Establish realtime data infrastructure – Replace polling with streaming architecture for continuous signal flow

2. Implement event-driven pub/sub patterns – Enable loosely coupled systems that respond to customer signals

3. Build predictive models using continuous data – Layer AI/ML on top of realtime streams for pattern recognition

4. Create proactive intervention workflows – Design automated responses to predictive signals (offers, alerts, support)

5. Monitor reliability metrics rigorously – Track latency, uptime, message integrity to maintain trust at scale

What makes this different

Most CX discussions focus on speed (faster responses, quicker resolutions.) But anticipation goes deeper. It's about infrastructure that doesn't just move data quickly, but does so reliably enough to build trust and flexibly enough to adapt as expectations evolve. Explore Ably's four pillars of dependability.

Realtime infrastructure is the hidden enabler. It's what makes customer care feel effortless, predictive, and ultimately, more human. Not because it replaces human judgment, but because it removes the friction that gets in the way of delivering exceptional care.

The companies winning in the Age of Anticipation aren't the ones with the flashiest technology demos. They're the ones who've built the unglamorous, reliable, adaptive infrastructure that makes anticipation possible at scale. They've realised that foresight isn't magic – it's architecture.

Business impact of anticipatory customer experience

52% of consumers stopped using brands after bad experiences – making proactive, anticipatory CX non-negotiable (PwC 2025 Customer Experience Survey)
65% cost reduction achieved by Doxy.me through realtime infrastructure that prevents issues versus fixing them reactively
61% of CX leaders deliver proactive communications using AI, while only 6% of laggards do, creating a significant competitive gap (Cisco research)
5+ years without a global outage – Ably's proven track record demonstrates the reliability required for maintaining trust at enterprise scale
40% of companies plan to increase investment in predictive instant experiences in 2025, signalling industry-wide shift to anticipatory models

Three pillars of anticipatory CX

1. Realtime data streams – Fresh, continuous signals flowing through your systems without latency or data loss

2. Reliability at scale – Infrastructure trusted to maintain consistency across millions of interactions, measured in years of uptime

3. Adaptive architecture – Event-driven systems that evolve with customer expectations without requiring rebuilds

Ready to build anticipatory experiences?

Ably's realtime platform delivers the continuous data streams and event-driven patterns your systems need to anticipate customer needs, with the reliability required to maintain trust at scale.

Six-plus years of 100% uptime. Sub-6.5ms message delivery latency. Built-in message integrity guarantees.

How we built an AI-first culture at Ably

Ably Blog — Tue, 27 Jan 2026 10:44:50 +0000

Most companies talk about being "AI-first." At Ably, we decided to actually become one. We build realtime infrastructure for AI applications. To do that credibly, we need to live and breathe AI ourselves – not just in our product, but in how we work every day.

A year ago, we began a company-wide push for AI adoption. This post breaks down how we did it: the pillars, the tooling, the MCP advantage, the early mistakes, the wins across engineering, marketing, sales, and finance, and the cultural momentum that turned a mandate into a mindset.

Building an AI-first company culture

When Jamie Newcomb, Ably's Head of Product Operations, began championing internal AI adoption, the approach was straightforward: everyone at Ably should explore how AI could make them more effective. No exceptions.

"We might want to tone down the language," Jamie admits with a laugh, "but it really is mandated. Everyone at Ably should be using AI to see how they can make themselves more effective. But it's not just about doing things faster. It's about doing things you couldn't do before. The goal is to shift the mindset, where people stop asking 'can AI help with this?' and start assuming it can, then push further: what's now possible that wasn't?"

Today, that mandate has evolved into something far more organic. A company-wide culture where AI isn't just accepted, it's expected.

For a company processing 2 trillion operations monthly, this isn't about following trends, it's about credibility. It's about walking the walk. To build AI Transport that developers can trust for agentic workloads, we need firsthand experience of how AI performs in real operational environments, both the advantages and the pitfalls.

Three pillars of successful AI adoption

Ably's approach to AI rests on three interconnected pillars:

Internal AI adoption and enablement: Integrating AI into workflows and processes across every team to enhance capabilities and drive productivity improvements. The goal isn't just providing tools, it's automating repetitive, time-consuming tasks so people can focus on strategic thinking and creative problem-solving.
AI developer experience: Using AI to make Ably's platform more discoverable and easier to use for developers. This means AI-enhanced documentation, intelligent tooling, and optimized SDK experiences, empowering developers to build real-time products faster with the help of LLMs. The goal is to position Ably as essential infrastructure for real-time user experiences powered by AI.
AI product enhancement: Making proactive, explicit efforts to understand AI use cases where Ably delivers value, determining what we need to enable those use cases, and ensuring those capabilities are part of our roadmap. This pillar is about building infrastructure informed by real customer needs, both known and yet to be discovered.

"My main role is about process efficiency in product engineering," Jamie explains. "And that naturally extended to AI adoption. We believe there are significant productivity improvements we can make if everyone adopts AI thoughtfully across the company."

These pillars aren't separate initiatives, they're a unified strategy. Internal productivity adoption teaches us what works in practice. Developer experience ensures we're making Ably discoverable and easy to use for the growing number of developers building with AI. And AI product enhancement ensures we're building infrastructure informed by real customer needs, not just theory. This article focuses primarily on the first pillar, but the three are deeply connected. What we learn from using AI internally shapes how we build for developers using AI externally.

The MCP advantage

Perhaps the most significant internal development has been Ably's adoption of the Model Context Protocol (MCP), built over the summer of 2025.

"The Ably MCP connects all our internal tools together," Jamie explains. "It lets people access data across systems via AI assistants. Building this and seeing it genuinely change how people work has been incredibly rewarding."

What started as an experiment to see what was possible has grown into a company-wide platform that's now critical to daily workflows, integrating 15+ services through over 140 tools. Engineers can check CI build status and debug workflow failures without leaving their conversation. Product managers search across Jira issues, GitHub PRs, and Slack threads in a single query. Sales teams pull Gong call transcripts and HubSpot contact history to prepare for customer meetings. The breadth is significant: GitHub, Jira, Confluence, Slack, HubSpot, Gong, Jellyfish, Metabase, PagerDuty, GSuite and more, all accessible through natural conversation.

Before MCP, every AI interaction started from zero, engineers manually explaining Ably's infrastructure, marketers pasting in brand guidelines, constant context-switching that made AI feel like more work rather than less.

Now when an Ably employee opens Claude, they're not starting from scratch. Through MCP, they have immediate access to:

Shared context and prompt library
Company knowledge and documentation
Ably's tone of voice guidelines and style guides
Live data from internal tools and systems

Instead of "Here's what Ably is, here's our tone of voice, now help me write this email," it becomes simply: "Help me write this customer email about latency improvements." The AI already knows.

Scaling to 140+ tools created its own challenge: context limits. Ably solved this with a tool registry that lets the AI discover only what it needs for each task, keeping interactions lean and responsive.

"That context library is really important," Jamie emphasises. "The prompts for critical workflows (like our ICP matching) are all version controlled. When something needs adjusting, it's not about AI being wrong. It's about iterating on what you're asking the AI to do."

The platform continues to evolve based on team feedback. When engineers noticed they were dropping out the terminal to check GitHub Actions builds, new workflow tools were shipped within hours. Claude Code is used heavily to maintain and extend the MCP itself, with Claude's Agent SDK integrated throughout the development workflow. Using AI to build AI tooling is a big part of why the velocity is so high. That responsiveness, treating internal AI tooling as a living product rather than a one-off project, reflects how deeply AI has become embedded in Ably's operating culture.

Jamie spoke at length with Jellyfish on how Ably moved beyond data retrieval to unlock real analysis and insights through MCP, and you can read the full article here.

AI tool selection

When Ably first encouraged company-wide AI adoption, the approach was deliberately open-ended. People experimented with ChatGPT, Claude, and workflow orchestration tools like N8N, Zapier, and Relay.

"We've settled on Claude for our primary AI, particularly Claude Code for engineers, but people have the freedom to use whatever works best for them," Jamie says. "If someone has a strong case for a different tool, that's fine. We're not prescriptive about it."

Everyone at Ably has access to Claude for day-to-day work, whether that's drafting documents, thinking through problems, or exploring ideas. For workflow automation, Relay emerged as the orchestration layer, handling the multi-stage pipelines that power lead enrichment, ICP scoring, and sales alerts. The combination of Claude for reasoning and Relay for orchestration has become Ably's default stack, though teams remain free to experiment.

This flexibility matters, especially given Ably's positioning around AI Transport. "We can't just say 'use Claude' when we're building infrastructure that works with any LLM provider," Jamie notes. "We need to show that our approach works regardless of which AI you're using."

Results by team

Engineering

All engineers now use Claude Code for agentic coding, but the workflows vary based on the task.

For narrow, well-defined tickets, Claude can often one-shot a solution. Engineers point it at the relevant files, describe what they want, and use test-driven development as a guardrail. Claude writes the test first, sees it fail, writes the implementation, and confirms the test passes. For larger tasks, the approach is more iterative: Claude generates a plan as a markdown file, the engineer reviews and refines it, then kicks off implementation in a fresh context with the plan as input.

Discovery is another common use case. Engineers ask Claude questions about the codebase, "where does X get used?", "how does a message get from acceptance to being broadcast out to clients?", using it as a way to navigate complex systems without reading through thousands of lines of code.

The Ably MCP bridges the gap between documentation and code. Engineers pull context from Confluence docs, have Claude synthesise summaries, and feed those into coding sessions, turning scattered documentation into usable implementation context. Some are experimenting with Claude Code running asynchronously in the browser, queuing up tasks from a phone and reviewing the work later.

Beyond individual workflows, Claude is integrated into the development pipeline itself. Claude's Agent SDK is connected to GitHub to generate implementation context, review PRs, and fix CI issues before code reaches production. When a PR goes up, AI reviews it for obvious issues first, then engineers review it as they would any other colleague's work.

One principle remains constant: a single human author owns every PR, regardless of how much was AI-generated. The practice of engineering judgment, knowing what to accept, what to push back on, and what to rewrite, is still the job.

Marketing

The Marketing team wanted to spend more time shaping the narrative and shipping campaigns, not doing repetitive admin. There are always multiple activities in flight, each needing planning, research, execution, reporting, and analysis. That's where AI has been a huge productivity lever: the team has adopted it to streamline the "admin" layer so they can increase both output and quality without adding headcount.

Today, the team uses a small stack of AI tools across the lifecycle. They analyse Gong calls to accelerate market research and tighten messaging and positioning. They use Claude to pull and synthesise data from multiple sources to scope and validate content opportunities faster. They also automate lead validation and categorisation for sales follow-up, enriching contact and company data so the first human touch starts from context, not guesswork. And they map the customer journey with attribution, using AI to connect what prospects do pre-signup to intent signals, so they can prioritise the right audiences and double down on what's actually working.

For example lead qualification that used to take hours, is now a multi-stage AI pipeline that runs automatically on every signup. The system researches companies across 6+ data sources (Crunchbase, LinkedIn, SEC filings, PitchBook), extracts structured data, scores against 8 ICP criteria, classifies personas, and routes alerts to Slack with tier assignments and recommended actions – all before anyone on the team sees the lead.

"Marketing used to spend considerable time on this," Jamie recalls. "Now the first time they see a lead, it already has a confidence-scored ICP assessment, enriched company data, and suggested next steps."

Sales

New lead assignment uses multi-signal analysis (employee count, funding raised, revenue for public companies) to automatically route accounts to Commercial, Enterprise, or Strategic segments. For qualified leads, AI generates personalised email sequences based on the ICP analysis, tailoring messaging to the prospect's industry, technical challenges, and relevant customer references.

For existing customers, AI monitors self-service accounts against usage limits, surfacing expansion opportunities when customers approach thresholds and flagging critical capacity alerts that need immediate outreach. Relay handles the orchestration across all workflows.

Finance

Finance operations at Ably are treated like a tech product, using AI as a force multiplier to engineer away repetitive work.

The team systematically verifies contracts, builds smarter revenue models, and automates reconciliation work. A recent hackathon project eliminates thousands of monthly clicks in the Stripe-to-Xero process, the kind of repetitive work that most finance teams wouldn't know where to start automating.

They use Ably's MCP to retrieve data from Xero, then create and update sheets directly through Claude, turning what would be manual exports and data entry into conversational requests. It's a small example of how the platform extends beyond engineering and into every corner of the business.

The cultural shift

Creating an AI-first culture isn't just about providing tools – it's about enablement, support, and honest assessment of where you are versus where you're headed.

We run AI drop-in sessions every Friday where team members can bring questions, share what they've built, or explore new ideas. An internal Slack channel serves as a continuous stream of AI experiments, wins, and collaborative problem-solving.

"When Charlotte [Delivery Manager] and I approach teams, we don't even talk about AI initially," Jamie reveals. "We ask: what are your repetitive processes? Once teams understand their processes, then you can start the AI conversation."

"Anyone can build something now," Jamie says. "The barrier to solving a problem has basically been removed because people can use AI to build the solution themselves."

The result is what Jamie calls the "wow moment": when someone successfully builds their first AI-powered solution, a ceiling lifts. "Once people have that moment, they just keep building."

What's next

But Jamie is candid about where Ably still has room to grow. "To be completely honest, we haven't hit our potential yet," he admits. "We've made real progress, but there's still more impact to unlock from AI across how we work, our processes, and how we achieve our product outcomes. And when we think we've got there, there'll still be more room to grow."

The vision for what's next is clear: continuing to integrate AI deeper into how Ably works. The foundations are in place: agentic coding, AI-assisted PR reviews, automated workflows across teams. But the true potential lies in making these the default across every function, not just the teams that adopted early.

"The biggest gains come from how people think, not just what tools they use," Jamie explains. "When people stop asking 'can AI help with this?' and start assuming it can, that's where the real impact comes from."

The most significant outcome isn't any specific tool or workflow, it's that cultural shift in action. "We don't have a problem at Ably where people are on the fence about whether AI can help them," Jamie reflects. "We've shown that it can. Now it's about enablement and encouraging people to identify problems they can solve themselves."

The same infrastructure philosophy that powers our internal AI adoption powers our AI Transport product. Read how Ably enables reliable, scalable realtime experiences for conversational AI here.

Gen‑2 AI UX: Conversations that stay in sync across every device

Ably Blog — Tue, 27 Jan 2026 10:15:10 +0000

Start a conversation on your laptop, finish it on your phone. The context just follows you. That's what cross-device AI sync delivers. No reloading history, no reintroducing yourself, just one continuous thread across every screen. It builds trust, reduces friction, and makes the assistant feel like a single, persistent presence. This post unpacks why users expect it, what makes it technically tricky, and what your system needs to get it right.

AI conversations must survive the device switch

Modern users have grown to expect realtime, seamless experiences from their apps and AI tools. They want instantaneous responses, continuous interactions, and no interruptions as they move between devices. This expectation extends to AI-powered experiences: if you start a conversation with an AI assistant on your laptop, you should be able to pick it up on your phone or another tab without missing a beat. Equally, if you have initiated a long-running asynchronous task you want to be notified once it's completed, no matter which device you're using at the time.

What users want, and why this enhances the experience

Users want continuous, identity-aware AI conversations that follow them across devices. In practice, this means an AI chat session linked to their identity rather than a single tab or device. The conversation should feel like a single thread they can return to at any time.

That continuity builds trust. The AI isn't "forgetting" just because you switched devices. A remembered history signals reliability and intent, helping users feel the assistant is genuinely useful. Multi-turn conversations flow naturally, and users avoid repeating themselves or reconstructing context.

This matters even more once AI systems move beyond simple chat. When an LLM is running long-lived, asynchronous work such as multi-step research, tool calls, or background analysis, users expect to see progress and results wherever they happen to be at the time. You might start a task on your desktop, step away while the model works, and then pick up your tablet to see the output appear as soon as it's ready.

Real-world usage makes multi-device continuity unavoidable. These moments must be frictionless and reinforce the sense that the AI is persistent, reliable, and working on your behalf rather than being tied to a fragile client session.

Why this is proving challenging

HTTP is fundamentally stateless. Each request stands alone, meaning conversation context has to be manually preserved and restored for both the client and the server. This makes cross-device AI session continuity complex.

Having clients poll for updates is inefficient and adds latency. Long-polling or server-sent events help, but only partially. They don't enable simultaneous bi-directional, low latency messaging, which is what smooth AI conversations require.

Handling reconnections, preserving message order, and managing updates across multiple active clients requires considerable infrastructure. Doing this reliably, at scale, across networks and devices, is beyond what the typical product team can or should build from scratch.

Why you need a drop-in AI transport

Given the challenges described, building a robust system for realtime synchronisation from scratch can significantly drain engineering resources and slow product velocity. This is where a drop-in AI transport layer becomes essential.

Persistent, bi-directional messaging

To support conversations that stay in sync across devices, such a layer must offer persistent, bi-directional messaging using protocols like WebSockets for AI streaming. This allows for continuous, low-latency communication in both directions, enabling the AI to push updates and the client to send input without waiting for discrete request/response cycles.

Identity aware fan-out

Equally important is identity-aware fan-out. The transport system needs to recognize all active sessions associated with a single user and ensure that every message or state update is sent to all of those endpoints. That means when a user sends a message on one device, every other device they're signed in on should immediately reflect the change.

Ordering and session recovery

The system also needs to preserve message ordering and support reliable session recovery. If the connection drops momentarily, say from a device switch or network disruption, the user shouldn't lose messages or see them out of order. A well-designed transport layer offers mechanisms to replay missed events and keep message sequences intact, ensuring consistency in the conversation.

Presence tracking

This enables the backend to know which devices are currently online and active. It helps coordinate updates, prevents redundant notifications, and can be used to power features like realtime indicators for typing or collaborative editing across devices.

Streaming support

To maintain a high-quality conversational UX, the transport layer must support LLM token streaming. This includes delivering partial, realtime updates from the AI model as it generates responses. That stream must arrive quickly, in order, and appear simultaneously on any active device.

Ably's infrastructure supports all of these capabilities as part of its realtime platform. It eliminates the need to custom-build low-level transport solutions, allowing engineering teams to focus on building intelligent, agentic features instead of protocol logic.

How the desired experience maps to the transport layer

The table below breaks down what users expect from cross-device AI conversations, what your transport layer must support to deliver those experiences, and the technical mechanics that make it all work.

User Experience Desired	Required Transport Layer Features	Underlying Technical Requirements
Identity-aware sync across devices	Identity-aware fan-out	Map user identity to all active sessions and ensure message fan-out across them.
State replication across sessions	Maintain consistent shared state for conversation history and updates across devices.
Realtime continuation of streaming outputs and input states	Durable stream relay	Emit model outputs as streaming tokens; buffer streams server-side to continue on reconnect or switch.
No loss of flow during context switches	Reliable message ordering	Guarantee delivery order of messages across devices, preserving conversational context.
Session recovery on reconnect	Rehydrate sessions with missed messages after disconnects or page refreshes.
A persistent mental model of the AI assistant	Live session multiplexing	Allow multiple client connections per user and route interactions through a unified session view.
Realtime delta propagation	Transmit message edits or UI state updates as granular deltas to all active endpoints.
Multi-device sync	Mirror updates across devices in real time, including UI elements and message scroll state.
Session handoff and presence awareness	Detect presence state and manage smooth transition of active sessions between devices.

Cross device AI? You can ship it now.

Seamless cross-device conversations aren't futuristic - they're achievable today. You don't need to rebuild your stack to make it happen.

Ably AI Transport provides all you need to support persistent, identity-aware, streaming AI experiences across multiple clients. If you're working on Gen‑2 AI products and want to get this right, we'd love to talk.

The new Ably dashboard: realtime visibility in your hands

Ably Blog — Thu, 22 Jan 2026 17:54:35 +0000

We've rebuilt the Ably dashboard to give developers clear, realtime visibility into how their applications behave.

This isn't just a cosmetic refresh. It's a shift from a configuration-first dashboard to a live observability surface. One that lets you see channels, connections, messages, and errors as they happen, debug issues instantly, and understand usage without stitching together logs and tooling.

Why we rebuilt the dashboard

The previous dashboard did a solid job in helping you manage your apps, API keys, and configurations. But it didn't show what was actually happening inside your realtime system. When something broke or behaved unexpectedly, you were left piecing together clues from SDK logs, APIs, and external tools. There wasn't a single place to answer operational questions like who's connected right now, what's happening on a particular channel, or whether a pattern of errors is brand new or recurring.

The new dashboard brings realtime observability directly into the browser. No setup, no extra tooling, no context switching - just a live window into Ably.

What's new (and why it matters)

At the heart of this release are four capability upgrades that change how your team operates realtime systems on Ably. Each one is useful on its own; together, they make it far easier to understand behavior in your apps, debug faster, and understand what's driving usage.

Channel and connection inspectors

The new inspectors provide realtime visibility into how your system behaves as data flows through Ably.

The channel inspector shows you who's attached to the channel, which messages are being sent, and what's happening in the presence set. You can also see which rules and integrations are active on that channel. Alongside that live activity, it surfaces realtime metrics like message rates, occupancy, and connection counts, so you can see performance as it changes - not after the fact.

The connection inspector enables you to see who's connected to your app. You can choose a specific connection and see which channels it's attached to and view live statistics such as the rate of messages being published by that connection. Additionally, you can see information such as the geographical location of the connection and the SDK it's connected to Ably with.Combined, the inspectors put realtime visibility into who's interacting with your app, and which channels they're interacting with. You now have the ability to debug any issues far more easily, such as 'why is this channel still active?'

Log search

Live streams are great for what's happening right now, but many investigations start with a timestamp. Log search lets you query historical platform events so you can understand past behavior, trace what happened and understand why it occurred. You can then compare today's traffic with last week's. It's ideal for debugging anomalies and spotting patterns, especially when you're trying to answer whether a problem is new or recurring.

Reports and analytics

The new reports section gives you aggregated visibility into how your apps are used over time - message volumes, connection and channel durations - so you can understand where consumption is coming from and what's driving traffic. This is particularly helpful when needing to explain usage internally, plan scaling work, or map realtime costs back to product features.

This is the foundation for deeper analytics arriving in future releases, including more granular breakdowns by product, SDK and device, plus finer‑grained views by app, channel, and namespace.

Web CLI

We've also introduced a new web CLI. A browser-based command line that lets you run Ably commands instantly. You can publish and subscribe to messages, enter presence, and manage your app configuration without any local setup. It complements the redesigned dashboard to give you a fast way to interact with Ably from anywhere. The web CLI is a powerful tool that's invaluable for exploring Ably features without requiring you to write any code. It is especially useful during support calls where you need to quickly reproduce a certain behavior or send a specific set of messages.

Logs tail and live telemetry

Each inspector includes a live, realtime log stream scoped to the resource you're viewing. If you're inspecting a channel, you see the events relevant to that channel; if you're inspecting a connection, you see the events relevant to that connection. This means you can trace behaviour as it happens, correlate spikes in live metrics with specific platform events, and debug instantly rather than collecting evidence after the incident.

A more modern, product-first dashboard

Alongside these new capabilities, we've modernized the dashboard itself. Navigation is cleaner and faster, with dedicated sections for each Ably product. The result is a more intuitive experience that helps teams get to the right tools quicker; whether they're debugging, trying out a new product, testing new features, or monitoring live traffic.

What's coming next

This release is a major step toward making the dashboard a full observability layer for Ably. We're working towards live logs: a global realtime stream of platform events across all resources to complement the existing log search functionality - so you can see what is happening live, and what has happened previously. We're also continuing to expand the reports section to provide richer visualization of your usage, performance, and reliability, across all your apps.

In parallel, we're continuing to modernize the remaining areas of the dashboard so that all your resources benefit from enhanced observability and analytics.

The evolution of realtime AI: The transport layer needed for stateful, steerable AI UX

Ably Blog — Wed, 21 Jan 2026 17:16:19 +0000

When we launched Ably in 2016, we set out to solve a fundamental problem: delivering reliable, low-latency real-time experiences at scale. So we set out to build a globally distributed system that didn't force developers to choose between latency, integrity, and reliability – trade-offs that had defined the realtime infrastructure space for years.

Fast forward to today, and we're reaching 2 billion devices monthly, processing 2 trillion operations for customers who demand rock-solid infrastructure for their mission-critical features. But over the past year, as AI has transformed from a backend optimisation tool into a front-and-centre user experience, we've been asking ourselves a critical question: What's Ably's role in the AI ecosystem?

From "nice-to-have" to essential infrastructure

A year ago, if you'd asked us about Ably's AI story, we would have told you that yes, customers were using us. Companies like HubSpot and Intercom were leveraging Ably for token streaming and realtime AI features. But honestly? The value proposition felt incremental. Traditional LLM interactions followed a simple request-response pattern: send a query, stream back tokens, done. HTTP streaming handled this reasonably well, and while Ably offered benefits, there wasn't a smoking gun reason to use us specifically for AI.

That's changed dramatically.

The shift to Gen 2 AI experiences

What we're calling "Gen 2" AI experiences are fundamentally different from what came before. Instead of simply querying a model's training data, today's AI agents reason, search the web, call APIs, interact with tools via MCP (Model Context Protocol), and orchestrate complex multi-step workflows. Just look at how Perplexity searches, or how ChatGPT now breaks down complex requests into observable reasoning steps.

This shift introduces an entirely new set of challenges:

Modern AI UX problems

Async by default: When an AI agent needs 30 seconds or a minute to complete a task (not 3 seconds) user behaviour changes. They switch tabs, check their phone, or start other work. A simple HTTP request suddenly needs to handle disconnections, reconnections, and state recovery.

Continuous feedback is mandatory: Users need to know what's happening. "Searching the web... Analysing documents... Calling your CRM..." This isn't a nice-to-have anymore. Without feedback, users assume the system has failed.

Multi-threading conversations: Imagine asking a support agent about your order status. While they're checking, you ask another question. Now you have two parallel operations that need coordination. The agent needs to know what else is in flight and potentially prioritise or sequence responses intelligently.

Cross-device continuity: Users want to be able to set tasks running and then pick-up later from where they left off. They may start a deep research query on their laptop, close it, and then want to check progress on their phone an hour later. The entire conversation state needs to seamlessly transfer.

The transport layer modern AI needs

Our vision for addressing these challenges centres on what we're calling the Ably AI Transport. A drop-in solution that handles the complexity of making the AI UX resilient and multi-device - so developers can focus on building great agent experiences, not wrestling with networking problems.

We focus on everything between your AI agents and end-user devices, leaving orchestration, LLM selection, and business logic where they belong – in your control.

Layer 1: The foundation

At the base level, the AI Transport provides what you'd expect from Ably: bulletproof reliability, multi-device synchronisation, and automatic resume capabilities. But the real shift is architectural. Instead of your agent responding directly to requests, it returns a conversation ID. Devices subscribe to that conversation, and from that point forward, the agent pushes updates through Ably.

This simple change unlocks powerful capabilities:

Decoupling: Agents and devices can disconnect and reconnect independently without losing continuity
Bidirectional control: Need to stop an agent mid-task or ask a follow-up question? There's a direct communication channel that doesn't require complex routing infrastructure
State recovery: Reconnecting devices don't replay every token. They get current state and resume from there

Layer 2: Richer orchestration

The next layer introduces shared state on channels using live shared state. This enables sophisticated coordination:

Agent presence and status: Devices know if agents are active, thinking, or have crashed. Agents can broadcast their current focus ("Analyzing Q4 data...") as state rather than events.
Multi-agent coordination: When multiple agents work simultaneously - say, one handling a technical query while another processes a billing question - they can see each other's state and coordinate without stepping on each other's work.
Context-aware prioritisation: Agents can see if a user is actively waiting versus having backgrounded their session, enabling smarter resource allocation.
Client-side tool calls: In co-pilot scenarios, agents can query the client directly about user context "Is the user currently editing this field?" without roundtripping through backend systems.

Layer 3: Enterprise-grade observability

Because everything flows through Ably, you gain comprehensive visibility into the last mile of your AI experience. Stream observability into your existing systems, integrate with Kafka for audit trails, and leverage enterprise features like SSO and SOC 2 compliance that come standard with Ably's infrastructure.

A new way of building

What excites us most is the idea of providing a stateful conversation layer that removes infrastructure concerns from the developer's plate. Think of it as abstract storage for conversation state, combined with the realtime capabilities developers need for modern AI UX.

The developers building these experiences don't want to solve networking problems. They want to focus on prompts, orchestration, RAG pipelines, and agent logic. The transport layer shouldn't be where they spend their time – but it will become critical as user expectations evolve to match what ChatGPT, Perplexity, and Claude are demonstrating daily.

Framework-agnostic by design

One pattern we've noticed: as engineering teams mature in their AI journey, they tend to move away from monolithic frameworks and build custom orchestration logic. This makes sense – these systems become core to their business differentiation.

That's why the Ably AI Transport is deliberately framework-agnostic. Yes, we're building drop-in integrations with OpenAI's agent framework, LangChain, LangGraph, Vercel AI SDK, ag-ui and others to make getting started trivially easy. But the architecture doesn't lock you in. Swap out your orchestration layer, change LLM providers, rebuild your agent logic – your transport layer and device communication remain consistent.

The road ahead

If you're building AI experiences and wrestling with questions like "How do I handle interruptions?", "What happens when users switch devices mid-conversation?", or "How do I coordinate multiple parallel agent tasks?" – we'd love to talk. We're convinced there's a better way to build these experiences, and it starts with not having to rebuild the real-time infrastructure layer from scratch.

The plumbing shouldn't be your problem. Building great AI experiences should be.