Your User Closed the Tab. Your Node Server Is Still Paying for the LLM Call

• Book: AI That Ships
• The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

---

Your user asks the support bot for a full migration plan. Three seconds in, they see the answer is going the wrong way and close the tab. The browser drops the connection. Your Node handler doesn't notice. It keeps pulling tokens off the model stream, keeps writing them into a socket nobody is reading, and when the model asks for a tool call, it runs the tool and sends the whole conversation back for another turn.

Nothing crashed, so nothing alerted. The only trace is on the invoice.

Node doesn't cancel any of this for you. A handler is just an async function, and it runs until it returns. If you want the work to stop when the user leaves, you have to carry that fact from the socket into every call that costs money. The tool for that is one AbortSignal, passed all the way down.

What actually keeps running

Three things outlive the user if you do nothing.

The model stream. Your server holds an open HTTP connection to the provider and keeps reading it. As long as you read, the model keeps generating until it finishes or hits max_tokens.

The tool calls. If the model asked for a search or a database lookup, your code runs it. The results go nowhere useful.

The next turns. In an agent loop, every turn re-sends the conversation so far as input. A turn you start is a turn you pay for, input and output. A turn you never start costs nothing. That last point is the one you can count on regardless of provider.

What happens to output tokens after you close the upstream connection is murkier. The OpenAI streaming guide and the Anthropic streaming guide don't spell it out.

One AI gateway vendor, TrueFoundry, publishes per-provider cancellation policies: it lists Vertex AI and Azure OpenAI as "confirmed not to bill cancelled streams," and says OpenAI "bills the full completion whether or not you keep reading." It describes the table as based on its own measurements but doesn't publish them or cite a source. A developer on the OpenAI community forum has said the opposite, though the same thread notes the input context is still charged. Treat the output side as provider-specific and check your own usage logs. Cancel anyway: the turns and tool calls you skip are savings no matter who is right.

The tab is gone. The meter on the server side keeps running until your code tells it to stop.

Detecting the disconnect in Node

This is where most handlers go wrong. The obvious listener is req.on("close"), and it stopped meaning "the socket closed" years ago. The Node.js HTTP docs record the change in v16.0.0: "The close event is now emitted when the request has been completed and not when the underlying socket is closed." For a POST, that means req fires close as soon as you finish reading the body. Abort on it and you cancel every request. The old 'aborted' event is deprecated with the note "Listen for 'close' event instead."

Listen on the response. The ServerResponse close event "indicates that the response is completed, or its underlying connection was terminated prematurely." To tell the two apart, check writableFinished, which is true only once all data has been flushed:

// disconnect.ts
import type { ServerResponse } from "node:http";

export function disconnectSignal(
  res: ServerResponse,
): AbortSignal {
  const ac = new AbortController();
  res.on("close", () => {
    if (!res.writableFinished) {
      ac.abort("client_disconnected");
    }
  });
  return ac.signal;
}

This works under Express too, since its res is an enhanced version of Node's own response object. Combine it with a hard deadline using AbortSignal.any (added in Node v20.3.0 and v18.17.0) and AbortSignal.timeout, and you have one signal that fires on either.

If you run behind nginx, it already closes the upstream connection when the client leaves: proxy_ignore_client_abort defaults to off. If someone set it to on, your Node server never sees the disconnect and none of this fires.

Passing the signal into the SDK

Both official SDKs take a signal in their per-request options. The Anthropic TypeScript SDK documents it as "An AbortSignal that can be used to cancel the request," and the OpenAI Node SDK README says the same. Here is a streaming route using the Anthropic SDK, with plain node:http so nothing hides the plumbing:

// server.ts (Node 20+)
import http from "node:http";
import type {
  IncomingMessage,
  ServerResponse,
} from "node:http";
import Anthropic from "@anthropic-ai/sdk";
import { disconnectSignal } from "./disconnect.js";

const client = new Anthropic();
const MODEL = "claude-opus-5-5";
const MAX_TOKENS = 2048;

type Stats = { input: number; outChars: number };

async function readPrompt(req: IncomingMessage) {
  let raw = "";
  for await (const chunk of req) raw += chunk;
  return (JSON.parse(raw) as { prompt: string }).prompt;
}

The handler builds one signal from the disconnect and a two-minute deadline, then hands it to the SDK:

// server.ts (continued)
async function chat(req: IncomingMessage, res: ServerResponse) {
  const signal = AbortSignal.any([
    disconnectSignal(res),
    AbortSignal.timeout(120_000),
  ]);
  const prompt = await readPrompt(req);
  res.writeHead(200, {
    "content-type": "text/event-stream",
    "cache-control": "no-cache",
  });

  const stats: Stats = { input: 0, outChars: 0 };
  try {
    const stream = await client.messages.create(
      {
        model: MODEL,
        max_tokens: MAX_TOKENS,
        stream: true,
        messages: [{ role: "user", content: prompt }],
      },
      { signal },
    );
    for await (const ev of stream) {
      if (ev.type === "message_start") {
        stats.input = ev.message.usage.input_tokens;
      } else if (
        ev.type === "content_block_delta" &&
        ev.delta.type === "text_delta"
      ) {
        stats.outChars += ev.delta.text.length;
        res.write(`data: ${JSON.stringify(ev.delta.text)}\n\n`);
      }
    }
  } catch (err) {
    if (!signal.aborted) throw err;
  }

  if (signal.aborted) {
    logAborted("/chat", stats, signal.reason);
    if (!res.writableEnded) {
      res.end("event: aborted\ndata: {}\n\n");
    }
    return;
  }
  res.end("event: done\ndata: {}\n\n");
}

The res.end in the aborted branch matters for the timeout path: the client may still be connected, and without it the SSE response stays open. On a closed socket it does nothing. Last, the server itself:

// server.ts (continued)
http
  .createServer((req, res) => {
    chat(req, res).catch((err) => {
      console.error(err);
      res.destroy();
    });
  })
  .listen(3000);

The signal.aborted check after the for await loop in chat is deliberate. In the SDK's stream iterator, an abort error is caught with the comment "If the user calls stream.controller.abort(), we should exit without throwing." So when the signal fires mid-stream, your for await loop can end quietly, as if the model had finished. Code that only handles aborts in catch never logs them. Worse, code that saves the answer after the loop stores a truncated reply as if it were complete. Check signal.aborted after the loop, every time.

The same file shows the other direction: if you break out of the loop, the finally block aborts the request for you. So "stop reading" and "cancel" are the same operation in this SDK.

message_start carries the input token count, per the streaming docs. The final output count arrives in message_delta near the end of the stream, which an aborted stream never reaches. That's why the route counts characters instead.

Tool calls and the agent loop

The single stream is the small case. The expensive case is the loop, because every turn re-sends everything. Pass the same signal to the model call, to every tool, and check it before each turn. This goes in the same file and reuses client, MODEL and MAX_TOKENS. First the tool:

// server.ts (continued)
const MAX_TURNS = 8;

const tools: Anthropic.Tool[] = [{
  name: "search_docs",
  description: "\"Search the product documentation\","
  input_schema: {
    type: "object",
    properties: { query: { type: "string" } },
    required: ["query"],
  },
}];

type AgentStats = { input: number; output: number; turns: number };

async function runTool(
  name: string,
  input: unknown,
  signal: AbortSignal,
): Promise<string> {
  if (name !== "search_docs") return `unknown tool: ${name}`;
  const { query } = input as { query: string };
  const url = new URL("https://docs.example.com/search");
  url.searchParams.set("q", query);
  const r = await fetch(url, { signal });
  return await r.text();
}

Then the loop:

// server.ts (continued)
async function runAgent(
  messages: Anthropic.MessageParam[],
  signal: AbortSignal,
  stats: AgentStats,
): Promise<Anthropic.Message | undefined> {
  for (let turn = 0; turn < MAX_TURNS; turn++) {
    if (signal.aborted) return undefined;
    const msg = await client.messages.create(
      { model: MODEL, max_tokens: MAX_TOKENS, tools, messages },
      { signal },
    );
    stats.turns += 1;
    stats.input += msg.usage.input_tokens;
    stats.output += msg.usage.output_tokens;
    if (msg.stop_reason !== "tool_use") return msg;

    const calls = msg.content.filter(
      (b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
    );
    const results = await Promise.all(
      calls.map(async (c) => ({
        type: "tool_result" as const,
        tool_use_id: c.id,
        content: await runTool(c.name, c.input, signal),
      })),
    );
    messages.push({ role: "assistant", content: msg.content });
    messages.push({ role: "user", content: results });
  }
  return undefined;
}

fetch accepts the signal natively, so an in-flight tool request dies with the user. For tools that don't take a signal (a CPU loop, a legacy client), call signal.throwIfAborted(), documented in the Node globals reference, between steps. The if (signal.aborted) at the top of each turn is the cheapest line in the file: it is the one that stops a full-context turn from starting. Mid-turn, the SDK call and fetch reject with an abort error instead, so wrap the caller the same way as the stream route: catch, check signal.aborted, and log stats.turns. The tool loop shape follows Anthropic's tool use guide.

One signal, threaded through every call that costs money. Pull it once and everything downstream stops.

Logging what you didn't spend

You can't log "tokens saved" exactly, because the tokens that were never generated have no count. You can log the parts you know and one honest upper bound. This is the logAborted that chat calls:

// server.ts (continued)
function logAborted(route: string, s: Stats, reason: unknown) {
  const outEst = Math.ceil(s.outChars / 4);
  console.log(JSON.stringify({
    event: "llm.aborted",
    route,
    reason: String(reason),
    input_tokens: s.input,
    output_tokens_est: outEst,
    output_budget_unused: Math.max(0, MAX_TOKENS - outEst),
  }));
}

The divide-by-four is OpenAI's own rule of thumb of roughly 4 characters per token for English text. It's an estimate, and other tokenizers differ. output_budget_unused is the ceiling on what you avoided, and only if your provider stops generating when the connection closes. For the agent route, log turns from AgentStats at the moment of abort, then compare against the average turn count of runs that completed. The gap, times your average per-turn input, is the part of the saving you can defend in a cost review.

Then aggregate: aborted requests per route per day, and the share of all requests they represent. If a large share of your chat requests end in client_disconnected, that's a product signal as much as a cost one: people are leaving before the answer lands.

Two edges to check before you ship

The browser side has to actually close the connection. If your UI has a Stop button, wire it to an AbortController on the fetch call, so stopping is a real disconnect and not just hiding the text:

const ac = new AbortController();
stopButton.onclick = () => ac.abort();
await fetch("/chat", {
  method: "POST",
  body: JSON.stringify({ prompt }),
  signal: ac.signal,
});

And background jobs are a different contract. OpenAI's Responses API has a background mode built so a response survives a dropped connection; you can resume the stream from a sequence_number cursor. Closing the socket is not a cancel there. The cancel endpoint is, and "only responses created with the background parameter set to true can be cancelled." If you use background mode, your abort listener should call client.responses.cancel(id) instead of relying on the disconnect.

To try the route locally, start the server, use curl with -N to send a long prompt at it, and hit Ctrl-C after a second. You should get one llm.aborted line and no event: done.

---

If this was useful

Cancellation is one of those things that never shows up in a demo and always shows up on the bill. AI That Ships is the book in the series about that gap: cost ceilings, guardrails, evals, and deploying TypeScript LLM features on Node.js so they behave when real users do unpredictable things, like leaving.

The whole AI in TypeScript series, from your first LLM call to agents in production, is at xgabriel.com/ai-in-typescript.

Comments

[Ken @ Flaming Ape]

The req.on("close") footgun is the kind of thing you only learn after a quiet night on the invoice. Wiring one AbortSignal through the model call, every tool, and the "start next turn?" check is the whole game. I've seen agent loops keep paying for turns nobody was reading anymore, because the stop lived in the UI and never made it into the loop. Also check signal.aborted after the stream ends quietly. Some SDKs swallow the abort, and you end up saving a truncated answer like it finished.