A user closes the browser tab.
Your Node.js server notices the disconnect.
You propagate the AbortSignal to the upstream LLM request:
const signal = AbortSignal.any([
timeoutSignal,
c.req.raw.signal,
]);
await fetch(LLM_URL, { signal });
The fetch() is aborted.
So the model stopped generating.
Right?
Not necessarily.
This is an important distinction that is easy to miss when building streaming AI applications:
HTTP cancellation
≠
inference cancellation
≠
billing cancellation
Aborting a request proves something about your HTTP client. It does not, by itself, prove what happened inside the provider.
That boundary matters.
The Cancellation Chain
Consider the complete path:
Browser
│
│ disconnect
▼
Your Server
│
│ AbortSignal
▼
HTTP Client
│
│ connection closes
▼
LLM Provider
│
├── request cancellation received?
│
▼
Inference system
│
├── generation stopped?
│
▼
Usage / billing system
│
└── fewer tokens billed?
There are several independent claims here.
Claim 1: The browser disconnected
Your server can observe this.
Claim 2: Your server aborted the upstream request
You can observe this too:
signal.addEventListener('abort', () => {
console.log('upstream request aborted');
});
Claim 3: The provider stopped inference
This is a provider-side semantic.
Claim 4: Billing stopped at the same point
This is a usage-accounting semantic.
The first two are things your application controls.
The last two are things you have to verify with the provider.
AbortSignal Only Controls What Holds the Signal
The standard cancellation pattern is still absolutely worth implementing.
const controller = new AbortController();
const response = await fetch(url, {
signal: controller.signal,
});
// Later:
controller.abort();
Your server can also combine independent cancellation conditions:
const signal = AbortSignal.any([
AbortSignal.timeout(30_000),
c.req.raw.signal,
]);
This is exactly the right pattern for a connection-bound streaming response.
The important qualification is what the signal actually guarantees.
It tells the operation holding the signal:
Stop this operation.
It does not give your process magical control over the provider's internal scheduler.
AbortSignal propagates cancellation across async operations, and AbortSignal.any() lets a client disconnect and a server timeout share one signal.
But once the signal crosses the HTTP boundary, you need to know what the remote API promises.
A Better Diagram
Instead of:
Browser
X
│
▼
Server
│
X
▼
LLM
I prefer this:
Cancellation boundary
│
▼
Browser ───────► Your Server ───────► Provider
│ │ │
│ │ │
closed fetch aborted ????
│
├── inference stopped?
│
└── billing stopped?
Every arrow represents a different contract.
This is a distributed-systems problem disguised as a JavaScript API.
Why "The Connection Closed" Is Not Enough
Suppose your server logs:
12:00:00 request started
12:00:03 client disconnected
12:00:03 AbortSignal aborted
12:00:03 fetch rejected with AbortError
That proves your server stopped waiting for the upstream response.
It does not prove:
12:00:03 inference stopped
12:00:03 generation stopped
12:00:03 billing stopped
The provider might:
- immediately stop generation;
- stop generation asynchronously;
- finish a small amount of already-buffered work;
- continue the generation despite the client disconnect;
- stop generation but still account for tokens already generated.
Only the provider's semantics and usage data can tell you which one happened.
The Experiment
This article comes with a small experiment suite. The code is at aiengineering-book/llm-cancellation-experiment.
The important thing about the experiment is that it separates two questions.
Experiment A: Does my server cancel the HTTP operation?
This is purely a client/server question.
We start a long streaming request and abort it after a fixed number of milliseconds.
Expected result:
client:
request started
received N chunks
AbortController.abort()
fetch/stream terminates
This proves the local cancellation path.
Experiment B: Did the provider stop generating?
That requires provider-side evidence.
The experiment therefore records:
request ID
abort timestamp
last received chunk
provider completion/status
usage record
Do not infer the last two from the first two.
A Local Provider That Makes the Boundary Visible
The experiment repository includes a deliberately simple mock LLM provider.
It supports two modes:
HONOR_ABORT
provider stops when the client disconnects
IGNORE_ABORT
provider keeps generating even after the client disconnects
This is not pretending to be a real LLM provider.
It is a controlled experiment showing why the HTTP-layer observation cannot prove the provider-layer behavior.
Run:
pnpm install
pnpm experiment:honor
and:
pnpm experiment:ignore
The two runs use exactly the same client code.
Only the provider-side behavior changes.
What You Should See
In the HONOR_ABORT run, the provider will report something like:
provider:
generated: 21
billed: 21
stopped: client_disconnect
In the IGNORE_ABORT run:
provider:
generated: 100
billed: 100
stopped: completed
The exact numbers will vary because the experiment uses timing.
The important observation is:
Client observation:
"I aborted the request."
Provider observation:
"What happened to inference?"
Those are different measurements.
A Real Provider Result: DeepSeek
The mock provider shows that the two outcomes are possible. A real provider shows which one you have.
The experiment was run against DeepSeek's streaming Chat Completions endpoint (deepseek-flash, thinking disabled). DeepSeek was chosen because it exposes an account balance endpoint, GET /user/balance, which can serve as an external meter. The test needs two properties from that meter: it must be independent of the stream your client reads, and it must be fine-grained enough to tell the two hypotheses apart.
The method:
- Run one request to completion as a control, and record its
usageand the balance change. This gives a rough cost per token. - Run several requests with a large
max_tokens(12,000). Abort each one a fixed time after its first content chunk, so every abort lands mid-generation. Do this at two different abort points. - Wait for the balance to stop changing, then compare the drop with two predictions: the provider stopped near the abort, or the provider ran each request to
max_tokens.
Two abort points matter because a single one cannot show that billing follows the abort. If the cost tracks how long the stream stayed open, the later abort should cost proportionally more than the earlier one.
What was measured
The control request finished all 8,000 tokens and dropped the balance by about 0.03-0.04 CNY. That is about 4.4 millionths of a CNY per output token, which is consistent with the provider's published off-peak output price at typical exchange rates. The control also produced 41,892 characters, or about 5.2 characters per token, which is how the received tokens below are estimated from the characters the client saw.
Aborted mid-stream, 8 requests each, max_tokens = 12,000
3 s after 15 s after
first chunk first chunk
received by client (tokens) ~3,000 ~14,600
predicted cost if stopped ~0.013 CNY ~0.064 CNY
predicted cost if run to max ~0.42 CNY ~0.42 CNY
observed balance drop 0.01 CNY 0.06 CNY
Aborting five times later let the client receive about five times as many tokens, and the cost rose from 0.01 to 0.06 CNY. Both observations sit close to the "stopped near the abort" prediction and far below the 0.42 CNY that running every request to max_tokens would have cost.
An earlier run of 5 aborted requests, only 2 of which were mid-stream, showed the same pattern: the total balance drop for the whole run was 0.04 CNY, almost all of it explained by the 8,000-token control.
For this model and these settings, a client disconnect did not lead to billing out to max_tokens. Billing followed the point at which the client aborted.
What this does and does not establish
The result is a statement about billing, measured from outside. It is a proxy for inference: a provider that kept generating but did not charge would look identical. In practice that is an unusual arrangement, but the experiment cannot distinguish it.
Some limits apply to the numbers themselves:
- Resolution: the balance has two decimals of CNY. The 3-second batch cost about 0.01 CNY, which is at the edge of what the meter can show, so that point alone is weak evidence. The 15-second batch is clearer, and the comparison with 0.42 CNY holds for both. A precise per-request token count is not available this way.
- Cost per token: it comes from a single 8,000-token control run measured at 0.01 CNY resolution, so it is good to about 15%. That is enough to tell 0.06 from 0.42, not enough to tell 13,600 billed tokens from 14,600.
- Lag: the balance moved about 9-13 minutes after the requests ended. A check made one minute later would have reported no cost at all. Wait before reading it.
-
No usage and no stop signal in the stream: an aborted stream never delivers the final chunk that carries
usage, and nothing else on the stream says "stopped at token N". The client cannot learn the billed token count from the stream it aborted. In this test the stop point could only be inferred from billing, and only roughly. Other providers may expose more, such as an explicit cancel event with a final status, but this experiment does not cover them. -
Overshoot: if a provider finishes work already in flight before it notices the disconnect, the extra tokens would be a small number past the abort point. That is far below what a two-decimal balance can resolve, so this measurement can rule out running to
max_tokens, but not a few tokens of overshoot. - Scope: one model, streaming only, thinking disabled, off-peak pricing, a single day. Thinking mode, non-streaming requests, and aborts before the first chunk were not measured separately. Non-streaming calls are harder to test this way, because there is no first chunk to anchor the abort on.
One more observation concerns the abort timing. With a fixed 5-second abort in the first run, 3 of 5 requests had received no content at all when the abort fired. The time to first token can exceed an aggressive abort window, so a cancellation test that starts its timer at the request start may measure the queue rather than generation. The final experiments start the timer at the first content chunk.
This is one provider's behavior on one day, written down with its method so it can be repeated. It is not a general statement about LLM APIs.
To repeat it:
git clone https://github.com/aiengineering-book/llm-cancellation-experiment.git
cd llm-cancellation-experiment
pnpm install
cp .env.example .env # set LLM_API_KEY
pnpm experiment:billing
ABORT_AFTER_MS sets the abort point, counted from the first content chunk. Run it once with 3000 and once with 15000. The run spends real money, up to a few cents, and takes 15-30 minutes because it waits for the balance to settle.
Accounting for a Cancelled Request
The measurement says what one provider did. It does not make the cost of a cancelled request known, so record it as a range instead of a number:
- Lower bound: the tokens your client received before the abort.
-
Upper bound:
max_tokens, plus the prompt, because you cannot see where the provider stopped.
The DeepSeek result puts those requests near the lower end. Another provider, or a self-hosted server, can sit anywhere in the range. Log the last usage frame you actually received, and reconcile against the provider's reported usage later, instead of estimating cost from the abort time.
Side Effects Do Not Cancel
An HTTP abort stops your wait on the stream. It does nothing for work that has already started somewhere else. If the model emitted a tool call and your code began executing it, the abort does not reach the work that tool kicked off downstream.
So the cancellation chain has one more boundary after the provider: the tools. A tool that sends an email or writes a record needs to check for cancellation itself, and be safe to run twice. The next article in this series covers that, along with what to do when the work should survive the connection entirely.
Why This Matters for Real Providers
With a real provider, replace the mock provider with the provider's streaming API. The result above used DeepSeek, so the response ID here comes from the id field on each streaming chunk. Other providers expose it differently.
Record the provider request/response ID as early as possible.
For example, OpenAI's Responses streaming API emits a response.created event containing a unique response ID, and its response object exposes status and usage fields.
Your instrumentation should therefore capture something like:
requestId = resp_...
startedAt = ...
firstTokenAt = ...
abortAt = ...
lastChunkAt = ...
Then, after the request has terminated, obtain the provider-side record or usage information through whatever mechanism that provider exposes.
The question you want to answer is not:
"Did my fetch reject?"
It is:
"For request
resp_..., what was the final provider-side status and usage?"
Measure, Don't Assume
A controlled test should look like this:
Control
│
▼
Long generation
│
▼
record usage
versus:
Cancellation
│
▼
Long generation
│
abort at T
│
▼
record usage
Then compare:
Control Cancelled
-----------------------------------------
Request ID resp_A resp_B
Abort time — 5.0s
Final status completed ?
Output tokens 8,421 ?
Billed tokens 8,421 ?
You need provider-side evidence for the final two columns.
If the provider does not expose enough information to establish this, the honest conclusion is:
We verified HTTP cancellation, but we could not independently verify provider-side inference termination or billing behavior.
That is a much stronger engineering conclusion than guessing.
Cancellation Is Not the Same as "Delete the Work"
There is another subtle point.
Even if a provider supports cancellation, cancellation is not necessarily instantaneous.
Imagine:
t = 0s request starts
t = 5s client disconnects
t = 5s server aborts fetch
t = 5.1s provider receives disconnect
t = 5.2s inference scheduler stops generation
There is a propagation delay.
In a distributed system, you should expect this.
The correct model is:
abort requested
│
▼
signal propagated
│
▼
provider receives cancellation
│
▼
provider stops work
Each transition takes time.
And Billing Has Its Own Semantics
Even if inference stops at 5.2 seconds, billing may not be defined as:
"Charge exactly the number of tokens generated before 5.2 seconds."
A provider might have:
- token accounting rules;
- minimum billable units;
- cached input accounting;
- reasoning-token accounting;
- provider-specific cancellation rules;
- delayed usage aggregation.
Therefore:
inference stopped
and:
billing stopped
are still separate claims.
For example, OpenAI's current pricing is token-based and distinguishes input, cached input, and output pricing across models.
That makes it especially important to use the provider's actual usage record rather than estimating cost from the number of chunks your server happened to receive.
A Useful Production Instrumentation Pattern
Treat the LLM request as an observable operation.
type LlmRequestMetrics = {
requestId?: string;
startedAt: number;
firstChunkAt?: number;
lastChunkAt?: number;
abortedAt?: number;
completedAt?: number;
clientDisconnected: boolean;
abortReason?: 'client_disconnect' | 'timeout' | 'application';
receivedChunks: number;
receivedBytes: number;
providerStatus?: string;
inputTokens?: number;
outputTokens?: number;
totalTokens?: number;
};
Then your logs can tell the whole story:
llm.request.started
request_id=resp_123
llm.request.first_chunk
request_id=resp_123
http.client_disconnected
request_id=resp_123
llm.request.aborted
request_id=resp_123
reason=client_disconnect
llm.provider.completed
request_id=resp_123
status=cancelled
output_tokens=1234
Now you can actually investigate the behavior.
Without the request ID, timestamps, and usage data, you are mostly guessing.
This Also Changes How You Think About "Saving Money"
Suppose you have:
100,000 generations / day
and:
10% of users abandon generation early
It is tempting to say:
"Cancellation saves us 10%."
That conclusion is unjustified.
You need to know:
abandonment rate
×
average unused generation
×
provider cancellation semantics
×
billing semantics
Only then can you estimate the actual savings.
A useful metric is:
wasted_output_tokens
rather than simply:
abandoned_requests
For example:
Request A
generated: 10,000
user saw: 500
potentially abandoned: 9,500
versus:
Request B
generated: 1,000
user saw: 500
potentially abandoned: 500
Both are one abandoned request.
Their cost profiles are completely different.
The Three Questions Every AI Backend Should Ask
When implementing cancellation, ask these separately.
1. Did my application cancel?
You can test this locally.
signal.aborted === true
and:
catch (error) {
// AbortError / cancellation
}
2. Did the provider cancel?
Check:
- provider request ID;
- final response status;
- provider logs;
- provider API;
- provider documentation.
3. Did billing reflect the cancellation?
Check:
- usage records;
- token counts;
- billing records;
- controlled experiments.
Do not collapse all three into:
AbortController.abort()
A Particularly Important Example: Realtime APIs
Some APIs expose an explicit application-level cancellation operation.
For example, OpenAI's Realtime API has an explicit response.cancel client event, and the server reports the response as cancelled.
That's conceptually different from merely closing an HTTP connection.
You have:
HTTP disconnect
versus:
application-level cancel command
If a provider offers the latter, it gives you a much stronger semantic contract.
This illustrates the larger principle:
The stronger the cancellation guarantee you need, the more explicit the provider-side cancellation API should be.
The Distributed Systems Lesson
This isn't really about AbortController.
AbortController is just where the problem becomes visible in TypeScript.
The deeper issue is distributed cancellation.
Your application is a chain:
Browser
│
▼
API server
│
▼
LLM gateway
│
▼
Provider
│
▼
Inference scheduler
│
▼
Usage / billing
Cancellation must cross boundaries.
At every boundary, there is a question:
Does this component understand cancellation, and what does cancellation guarantee here?
That is a distributed-systems question.
The Correct Mental Model
Don't think:
abort()
↓
everything stops
Think:
abort()
↓
my operation stops
↓
cancellation signal crosses boundary
↓
remote system may honor it
↓
remote work may stop
↓
usage may change
↓
billing may change
Every arrow needs a contract.
Some arrows are controlled by you.
Some are controlled by the provider.
Final Takeaway
AbortSignal is still essential for a production LLM server.
If a user closes a streaming chat response, you should normally propagate that cancellation to the upstream request when the generation is connection-bound. Your server-side architecture should also distinguish these ephemeral responses from persisted work and long-running Agent tasks.
But don't make a stronger claim than your evidence supports.
This:
fetch() aborted
proves:
My application stopped waiting for that HTTP operation.
It does not automatically prove:
The provider stopped inference.
And it certainly does not automatically prove:
The provider stopped billing.
Those are three different facts.
Instrument the request. Capture the provider request ID. Run a controlled cancellation experiment. Compare provider-side usage.
Then you know.
The Rule
Cancellation is a mechanism.
Inference termination is a provider behavior.
Billing is an accounting result.
Don't assume one implies the others.
Measure the boundary.
Further Reading
This article is a follow-up to Your LLM App Is Wasting Money: What Happens When Users Close the Tab?, which covers how a client disconnect reaches your server and how to propagate it upstream with AbortSignal.
The server-side patterns it builds on, including streams, cancellation, and the difference between ephemeral responses and persisted work, are covered in Chapter 3 of 《AI Engineering with TypeScript — A Comprehensive Guide to Building AI Agents》 at Leanpub. The book's companion code is at github.com/aiengineering-book/code.
Top comments (10)
This cancellation chain is exactly the trap people fall into when they assume
abort()is a kill switch. The part I'd add from running agents in production: even when the provider does honor cancellation, the stop point is non-deterministic, so your own accounting has to treat a cancelled request as "cost unknown, upper-bounded by max_tokens" rather than zero. We log the lastusageframe we actually received on the stream and reconcile against the provider's reported usage later, because the two disagree often enough to matter on the monthly bill. The other thing worth separating out is side effects already in flight — if the model emitted a tool call before you aborted, the HTTP cancel does nothing for the work that tool kicked off downstream. We ended up making tool execution idempotent and cancellation-aware for that reason. Did you find any providers that actually surface a reliable "stopped at token N" signal, or is it all inference from the usage object?Thanks, these are good production points.
In my DeepSeek test, no: there was no stop signal at all. An aborted stream delivered no usage frame either, so I could only infer the stop point from billing, and only coarsely. I also looked at OpenAI and Anthropic's docs. OpenAI's Responses API has an explicit cancel endpoint, but only for background: true responses. Its API reference shows a cancelled response with usage: null. For synchronous requests the guide just says to terminate the connection. Anthropic's message_delta usage is cumulative, so you only have what arrived before you disconnected.
So I'd keep three things apart: the provider acknowledging a cancel, generation actually stopping, and final usage. Treating a cancelled request's cost as unknown, bounded by max_tokens plus the prompt, until you can reconcile against reported usage, seems right to me.
And agreed on tool calls. Cancelling the LLM request doesn't undo work a tool already started, so tools need to be idempotent and cancellation-aware.
Cancellation needs a capability model, not a UI promise. A useful status surface distinguishes “client detached,” “queued work cancelled,” “tool call cancelled,” and “side effect already committed,” with a correlation ID across each boundary. Otherwise Abort becomes a misleading word precisely when the workflow has external authority.
Agreed. The next article lists those four outcomes explicitly, with a correlation ID across boundaries. I've also added that to the Part 3 draft.
Does the abort actually kill the token stream, or just stop waiting for it?
Both, but at different layers, and only one of them is guaranteed.
On your side, the abort closes the connection and your client stops reading, so the stream ends for your application. That part is certain.
Whether the provider then stops generating depends on the provider. I measured one: DeepSeek, streaming chat requests aborted mid-generation. Billing followed the abort point and stayed far below what running to max_tokens would have cost. That suggests it stopped generating near the abort rather than finishing in the background, but it's indirect evidence. I could only see the bill, not the inference, and a few tokens of overshoot would be below what my meter could resolve.
Other providers can behave differently. OpenAI's background mode, for example, keeps running after the connection drops unless you call its cancel endpoint. So "I aborted" is safe to assume for your own process, and anything about the provider is worth measuring.
The bill as a check makes sense to me. For background mode I will treat the cancel call as part of the abort, not an extra.
The gap between HTTP connection drop and scheduler eviction is where the surprises happen. On hosted inference, a client disconnect sends an RST_STREAM to the reverse proxy, but the worker batch scheduler might keep running the sequence until the next iteration boundary or chunk flush. In production we see cancelled calls log almost full generation tokens because the provider finished the forward passes before the gateway tear-down reached the worker pool. If you run your own vLLM or Triton instances you can tune sequence preemption directly, but with third-party APIs you have to assume any in-flight batch completes and charges accordingly.
That matches the conservative stance for third-party APIs. My result is one provider, and a few tokens of overshoot past the abort would be below what my balance-based meter can resolve. I've added that as an explicit limit.
the nasty combo is abort + regenerate. user hits regenerate, the first one might still be running on the provider side, and now you're paying for two