You build an AI chat application.
A user sends:
"Explain how distributed systems work."
Your server calls an LLM API and starts streaming the answer:
LLM
│
├── "Distributed"
├── "systems"
├── "are"
├── ...
│
▼
Browser
Everything looks great.
Then the user closes the browser tab.
The response disappears.
But what about the LLM request?
Is it still running?
If your server doesn't explicitly propagate cancellation, the answer may be yes.
And that's not just a correctness problem. For an AI application, it can become a cost problem.
A user can abandon a generation after 500 tokens, while your backend continues paying for the remaining thousands of tokens.
In this article, we'll build the cancellation path for a TypeScript LLM server:
Browser
│
│ client disconnect
▼
Hono / Node.js
│
│ AbortSignal
▼
fetch()
│
│ cancellation
▼
LLM API
Along the way, we'll look at:
- Why a browser closing a tab doesn't automatically cancel your LLM request
- How
AbortControllerandAbortSignalwork - How to combine client disconnects with server-side timeouts
- How cancellation interacts with streaming responses
- When you should not cancel a request
- Why this is fundamentally a server-side TypeScript problem
The Problem: The User Has Left, But Your LLM Hasn't
Consider the simplest possible server:
app.post('/api/chat', async (c) => {
const response = await fetch(LLM_API_URL, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
model: '...',
messages: [
{
role: 'user',
content: 'Explain distributed systems',
},
],
stream: true,
}),
});
return new Response(response.body, {
headers: {
'Content-Type': 'text/event-stream',
},
});
});
This works.
The browser sends a request:
Browser
│
│ POST /api/chat
▼
Server
│
│ fetch()
▼
LLM API
The LLM starts generating tokens, and the server streams them back to the browser.
But now:
Browser
│
X
│
│ tab closed
The browser is gone.
What happens to the fetch() call to the LLM?
Nothing automatically tells your application to cancel it.
The server and the upstream LLM request are separate operations.
Your server needs to explicitly connect their lifecycles.
First: Understand the Two Connections
There are actually two HTTP connections here.
Connection #1
Browser ────────────────► Server
│
│ Connection #2
▼
LLM API
The browser controls Connection #1.
Your server controls Connection #2.
When the browser closes the tab:
Browser Server LLM API
│ │ │
│──── HTTP request ─────►│──── HTTP request ───►│
│ │ │
X │ │
│ │ │
│ connection closed │ │
│ │ │
│ │
│──── still connected ─►│
The LLM API doesn't magically know that the browser disappeared.
Your server has to propagate the cancellation:
Browser
X
│
▼
Server
│
│ abort()
▼
LLM API
This is where AbortController comes in.
AbortController: The Cancellation Primitive
The Web Platform already gives us a standard cancellation mechanism:
const controller = new AbortController();
const response = await fetch(url, {
signal: controller.signal,
});
// Later:
controller.abort();
The important part is:
signal: controller.signal
The AbortSignal is passed into the operation.
When:
controller.abort();
is called, APIs that support the signal can terminate the operation.
Node.js supports AbortSignal throughout its asynchronous APIs, including streams and HTTP-related operations.
This gives us a useful mental model:
AbortController
│
│ signal
▼
┌─────────────┐
│ async work │
└─────────────┘
│
│ abort()
▼
cancelled
The controller doesn't need to know what it's cancelling.
It simply broadcasts:
"Stop."
Any operation that received its signal can react accordingly.
Detecting a Client Disconnect
Now we need to answer the first question:
How does the server know that the browser has disconnected?
With Hono, the request exposes the underlying request signal:
c.req.raw.signal
This signal is aborted when the client connection is terminated.
So we can connect it directly to the LLM request:
app.post('/api/chat', async (c) => {
const response = await fetch(LLM_API_URL, {
method: 'POST',
signal: c.req.raw.signal,
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
model: '...',
messages: [
{
role: 'user',
content: 'Explain distributed systems',
},
],
stream: true,
}),
});
return new Response(response.body, {
headers: {
'Content-Type': 'text/event-stream',
},
});
});
Now the lifecycle looks like this:
Browser
│
│ request
▼
Hono
│
│ c.req.raw.signal
▼
fetch()
│
▼
LLM API
If the browser disconnects:
Browser
X
│
▼
c.req.raw.signal
│
│ aborted
▼
fetch()
│
│ cancelled
▼
LLM API
This is the critical connection that many first versions of AI applications miss.
But Client Disconnect Isn't the Only Reason to Cancel
There is another failure mode.
What if the LLM API simply takes too long?
You don't want a request hanging forever.
So we have two independent cancellation conditions:
┌─────────────────┐
│ Client closes │
│ browser │
└────────┬────────┘
│
▼
CANCEL
▲
│
┌────────┴────────┐
│ Server timeout │
│ 30 seconds │
└─────────────────┘
We want:
Cancel if either condition occurs.
Modern JavaScript gives us exactly that:
AbortSignal.any()
AbortSignal.any() creates a signal that aborts when any of the supplied signals aborts. It is available in Node.js 20+ and later versions.
So:
const timeoutSignal = AbortSignal.timeout(30_000);
const signal = AbortSignal.any([
timeoutSignal,
c.req.raw.signal,
]);
Now one signal represents both conditions.
The Complete Version
Putting it together:
app.post('/api/chat', async (c) => {
const signal = AbortSignal.any([
AbortSignal.timeout(30_000),
c.req.raw.signal,
]);
try {
const response = await fetch(LLM_API_URL, {
method: 'POST',
signal,
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
model: '...',
messages: [
{
role: 'user',
content: 'Explain distributed systems',
},
],
stream: true,
}),
});
return new Response(response.body, {
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
},
});
} catch (error) {
if (signal.aborted) {
console.log('LLM request cancelled');
}
throw error;
}
});
There are now two ways the request can terminate:
┌── browser closes
│
│
AbortSignal.any ──┤
│
│
└── 30s timeout
│
▼
abort
│
▼
fetch()
│
▼
LLM API
This is much better than manually maintaining separate timers and disconnect handlers.
Why This Matters Even More for Streaming
Cancellation becomes particularly important when you're streaming LLM output.
Without streaming:
Browser ── request ──► Server ──► LLM
20 seconds
Browser ◄────────────── complete response
With streaming:
Browser ◄── token ── token ── token ── token ── ...
The request may stay alive for tens of seconds or even minutes.
That creates a much larger cancellation window.
The architecture is essentially a stream pipeline:
LLM API
│
│ chunks
▼
SSE parser
│
│ events
▼
TransformStream
│
│ events
▼
HTTP response
│
▼
Browser
The important thing is that the stream is not a special "AI" mechanism.
It's just a stream-processing pipeline.
Node.js and the Web Streams API provide cancellation mechanisms through AbortSignal, and stream operations can be terminated when their signal is aborted.
This is why understanding server-side streams is so useful when building AI applications.
Chunks Are Not Messages
There's another subtle problem with streaming.
Suppose the LLM sends:
data: Hello\n\n
data: world\n\n
You might imagine that your HTTP client receives exactly those chunks.
It doesn't have to.
The network might give you:
data: Hel
then:
lo\n\ndata: wor
then:
ld\n\n
A TCP chunk is not necessarily an application-level message.
So your streaming pipeline needs to buffer incomplete data:
Network chunks
│
▼
┌──────────────┐
│ Buffer │
└──────┬───────┘
│
▼
Complete SSE events
│
▼
Application
A TransformStream is a natural fit:
function createLineTransform(): TransformStream<string, string> {
let buffer = '';
return new TransformStream({
transform(chunk, controller) {
buffer += chunk;
const lines = buffer.split('\n');
// The last fragment may be incomplete.
buffer = lines.pop() ?? '';
for (const line of lines) {
if (line.trim()) {
controller.enqueue(line);
}
}
},
flush(controller) {
if (buffer.trim()) {
controller.enqueue(buffer);
}
},
});
}
The pattern is:
upstream
↓
chunks
↓
buffer
↓
complete messages
↓
business logic
This same pattern appears when processing large files, SSE responses, and LLM streaming responses. The key abstraction is incremental processing, not AI.
What About Backpressure?
There's one more reason to treat this as a stream pipeline.
What if the producer is faster than the consumer?
LLM
│
│ very fast
▼
TransformStream
│
│ very fast
▼
Network
│
│ slow
▼
Browser
If data were allowed to accumulate indefinitely, memory usage could grow.
Streams solve this with backpressure.
Conceptually:
LLM
│
▼
Transform
│
▼
Network
│
▼
Browser
▲
│
└──── backpressure
When the downstream consumer cannot keep up, the stream machinery can stop pushing data upstream until capacity becomes available.
This is one of the major reasons streams are preferable to accumulating the entire response in memory.
And it is the same reason the following two approaches are fundamentally different:
// Wait for everything
const result = await response.text();
versus:
// Process incrementally
for await (const chunk of stream) {
process(chunk);
}
For AI applications, incremental processing is what makes token-by-token responses possible.
The Cost Problem
Now return to our original question.
Suppose:
User starts generation
│
▼
LLM generates 5,000 tokens
│
│
├── User closes tab after 500 tokens
│
▼
Server continues generating
The exact financial impact depends on the model, provider, request, caching, and billing model.
But the engineering principle is simple:
If an operation no longer has a consumer, you should explicitly decide whether the operation should continue.
For an interactive chat response, continuing is usually wasteful.
The user isn't going to read tokens that have nowhere to go.
Cancellation gives you a way to release the work:
500 tokens generated
│
▼
client disconnect
│
▼
AbortSignal
│
▼
cancel upstream request
The benefit isn't only token cost.
You also release:
- an HTTP connection
- stream buffers
- server-side resources
- concurrency capacity
- provider-side generation work, where the upstream API honors cancellation
But Don't Cancel Everything
Here's the important architectural distinction.
Client disconnect does not always mean "cancel the task."
Consider four operations.
1. Interactive chat
User asks question
↓
LLM generates response
↓
User closes tab
Cancel it.
The result has no value if the user has abandoned it.
2. File indexing
User uploads PDF
↓
Server starts indexing
↓
User closes browser
Don't necessarily cancel it.
The indexing operation is part of a persistent workflow.
The user's browser is merely observing the operation.
3. Database write
User submits form
↓
Server writes database
↓
Browser disconnects
Usually, you want the database operation to complete.
The database write is a business operation, not a streaming response.
4. Long-running Agent
Consider an Agent run:
User
│
▼
POST /agent/run
│
▼
Agent
│
├── search
├── browse
├── call tools
├── generate
└── ...
This might take several minutes.
Binding the entire Agent lifecycle to an HTTP connection is usually the wrong architecture.
Instead:
POST /agent/run
│
▼
taskId
│
▼
background worker
│
├── tool calls
├── LLM calls
└── state
The frontend can then subscribe to the task:
Browser ───────► task status
◄──────── SSE / WebSocket / polling
Now closing the browser doesn't necessarily destroy the Agent run.
This distinction is important:
Cancellation is a business decision, not merely a technical decision.
A Better Mental Model
Instead of thinking:
"The browser disconnected, so cancel everything."
Think:
"What is the lifecycle of this operation?"
There are two fundamentally different types of work:
Operation
│
┌───────────┴───────────┐
│ │
Connection-bound Task-bound
│ │
▼ ▼
Chat streaming Agent run
autocomplete file indexing
live response database write
│ │
▼ ▼
disconnect → cancel disconnect → continue
This distinction becomes increasingly important as an AI application grows.
One More Problem: Errors
Cancellation is not necessarily a normal application error.
For example:
try {
await fetch(url, { signal });
} catch (error) {
if (signal.aborted) {
// Expected cancellation
return;
}
throw error;
}
Compare that with:
429 Rate Limit
502 Upstream Failure
500 Internal Error
AbortError
These represent different things.
A production server should distinguish:
- expected operational failures
- external service failures
- programmer errors
- intentional cancellation
A useful error hierarchy might look like:
AppError
├── ValidationError
├── UnauthorizedError
├── RateLimitError
├── ExternalServiceError
└── ...
Then a global error handler can convert known application failures into consistent API responses while unexpected programmer errors are logged separately.
This kind of centralized error handling is especially important on servers because an unhandled error can affect many users rather than just one browser tab.
The Final Architecture
Putting everything together:
Browser
│
│ POST /chat
▼
┌───────────────┐
│ Hono │
│ │
│ Zod validate │
└───────┬───────┘
│
▼
┌───────────────────┐
│ AbortSignal │
│ │
│ client disconnect │
│ OR │
│ 30s timeout │
└─────────┬─────────┘
│
▼
┌────────────┐
│ fetch() │
└─────┬──────┘
│
│ streaming
▼
┌────────────┐
│ LLM API │
└─────┬──────┘
│
│ chunks
▼
┌────────────────┐
│ TransformStream│
│ │
│ parse / buffer │
└───────┬────────┘
│
│ SSE
▼
Browser
There are several independent pieces here:
Type safety
Zod → validated input → typed route
Streaming
LLM → chunks → TransformStream → SSE
Cancellation
disconnect ─┐
├─→ AbortSignal → fetch()
timeout ────┘
Error handling
LLM / application errors
↓
error hierarchy
↓
global handler
↓
consistent API
None of these mechanisms is specifically an "AI framework."
They're server-side engineering primitives.
And that's exactly why they matter.
The Bigger Lesson
When you first build an LLM application, it's tempting to think about the model first:
Which model?
Which prompt?
Which agent framework?
Which vector database?
But once the application has real users, many of the expensive problems are much less glamorous:
What happens when the user disconnects?
What happens when the model takes 2 minutes?
What happens when the provider returns 429?
What happens when the browser can't consume the stream fast enough?
What happens when the request times out?
What happens when an Agent outlives the HTTP connection?
These are server engineering problems.
And TypeScript gives you excellent primitives for solving them:
Promises
Streams
TransformStream
AbortController
AbortSignal
async iterators
typed errors
Zod
Hono
Once you understand these primitives, an LLM streaming server stops looking like mysterious AI infrastructure.
It's just a carefully designed asynchronous pipeline.
And that is the important shift:
AI engineering is still software engineering.
The model may be probabilistic.
The server shouldn't be.
Further Reading
If you want to go deeper into the underlying primitives, the Node.js documentation covers AbortSignal, stream cancellation, and Web Streams in detail.
This article is also based on the server-side TypeScript patterns covered in Chapter 3 of 《AI Engineering with TypeScript — A Comprehensive Guide to Building AI Agents》at Leanpub, particularly the sections on Streams, cancellation, and type-safe API design. The chapter explicitly treats LLM streaming as an instance of the same streaming model used elsewhere in Node.js rather than as a separate AI-specific abstraction.
If you're building AI applications with TypeScript, these are the foundations worth understanding before adding more sophisticated agent frameworks.
Top comments (1)
ei love your article it happen a couple too much time when controling multi agents love the content thanks you for the knowledge and your time