- Book: AI That Answers
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
You ship the streaming chat box. Text appears word by word, the UI feels fast, and the cost log you wired up last sprint shows a flat line of zeros. Nothing is broken on the network. The tokens were billed. Your code just looked for them in the place a non-streamed response keeps them, and a streamed response keeps them somewhere else.
Each API hides them in a different spot. OpenAI's Chat Completions sends usage only in one extra chunk at the end, and only if you ask for it. The Responses API attaches it to the response.completed event. Anthropic's Messages API splits it across two events, input on message_start and a running output count on message_delta.
This tutorial writes one small function per API. Each one streams text to a callback and returns the same typed object: input tokens, cached input tokens, output tokens, cost in dollars, and a flag that says whether the numbers are final. Every field name below was checked against the current source of the openai and @anthropic-ai/sdk packages.
Setup and the shared type
You need Node 22 or later (the openai README lists 22 as the minimum), both SDKs, and tsx to run TypeScript without a build step:
npm install openai @anthropic-ai/sdk
npm install -D tsx
export OPENAI_API_KEY=...
export ANTHROPIC_API_KEY=...
All three functions return this shape. Put it in usage.ts:
export type Price = {
inputPerMTok: number;
cachedInputPerMTok: number;
outputPerMTok: number;
};
export type StreamUsage = {
inputTokens: number; // all input, cached included
cachedInputTokens: number; // subset of inputTokens
outputTokens: number;
costUsd: number;
final: boolean; // came from the closing usage event
};
export type OnText = (text: string) => void;
export function priceUsage(
u: Omit<StreamUsage, "costUsd">,
p: Price,
): StreamUsage {
const fresh = u.inputTokens - u.cachedInputTokens;
const usd =
fresh * p.inputPerMTok +
u.cachedInputTokens * p.cachedInputPerMTok +
u.outputTokens * p.outputPerMTok;
return { ...u, costUsd: usd / 1_000_000 };
}
The final flag is the part people skip. A streamed call can end three ways: the provider sends its closing usage event, the user closes the tab, or the network drops. Only the first gives you a billing-grade number. The flag lets your logger tell "this call cost $0" apart from "this call ended before anyone told us what it cost".
OpenAI Chat Completions: the opt-in last chunk
Chat Completions does not send usage on a stream unless you set stream_options: { include_usage: true }. The SDK's own doc comment spells out what happens then: one more chunk arrives before data: [DONE], its usage field covers the whole request, its choices array is always empty, and every other chunk carries usage: null (source).
That empty choices array matters. Code that does chunk.choices[0].delta.content without optional chaining throws on the very chunk you wanted.
// openai-chat.ts
import OpenAI from "openai";
import { priceUsage, type OnText } from "./usage";
const MODEL = "gpt-5.4-mini";
const PRICE = {
inputPerMTok: 0.75,
cachedInputPerMTok: 0.075,
outputPerMTok: 4.5,
};
export async function streamChat(
prompt: string,
onText: OnText,
signal?: AbortSignal,
) {
const client = new OpenAI();
let input = 0, cached = 0, output = 0;
let final = false;
try {
const stream = await client.chat.completions.create(
{
model: MODEL,
messages: [{ role: "user", content: prompt }],
stream: true,
stream_options: { include_usage: true },
},
{ signal },
);
for await (const chunk of stream) {
const text = chunk.choices[0]?.delta?.content;
if (text) onText(text);
if (chunk.usage) {
const u = chunk.usage;
input = u.prompt_tokens;
cached = u.prompt_tokens_details?.cached_tokens ?? 0;
output = u.completion_tokens;
final = true;
}
}
} catch (err) {
if (!signal?.aborted) throw err;
}
return priceUsage(
{
inputTokens: input,
cachedInputTokens: cached,
outputTokens: output,
final,
},
PRICE,
);
}
The prices are the standard rates for gpt-5.4-mini on OpenAI's pricing page as of 2026-10-17: $0.75 per million input tokens, $0.075 cached, $4.50 output. Copy yours from the same page; they change.
OpenAI counts cached tokens inside prompt_tokens; its prompt caching guide treats ordinary input as input minus cached minus cache writes, which is why priceUsage subtracts the cached part before applying the full rate.
Same idea, three places: where each API puts the token count on a stream.
The Responses API: read response.completed
The newer Responses API streams typed events instead of chunks. Text arrives as response.output_text.delta, and the run ends with response.completed, which carries the full Response object (OpenAI streaming guide). Usage sits on event.response.usage, with input_tokens, output_tokens and a required input_tokens_details.cached_tokens in the SDK types (source).
No opt-in flag here. The catch is a second terminal event: response.incomplete fires when a response finishes early, for example after hitting max_output_tokens (streaming events reference), and it carries the same response object. If you only listen for completed, every truncated answer logs zero.
// openai-responses.ts
import OpenAI from "openai";
import { priceUsage, type OnText } from "./usage";
const MODEL = "gpt-5.4-mini";
const PRICE = {
inputPerMTok: 0.75,
cachedInputPerMTok: 0.075,
outputPerMTok: 4.5,
};
export async function streamResponses(
prompt: string,
onText: OnText,
signal?: AbortSignal,
) {
const client = new OpenAI();
let input = 0, cached = 0, output = 0;
let final = false;
try {
const stream = await client.responses.create(
{ model: MODEL, input: prompt, stream: true },
{ signal },
);
for await (const event of stream) {
if (event.type === "response.output_text.delta") {
onText(event.delta);
} else if (
event.type === "response.completed" ||
event.type === "response.incomplete"
) {
const u = event.response.usage;
if (u) {
input = u.input_tokens;
cached = u.input_tokens_details.cached_tokens;
output = u.output_tokens;
final = true;
}
}
}
} catch (err) {
if (!signal?.aborted) throw err;
}
return priceUsage(
{
inputTokens: input,
cachedInputTokens: cached,
outputTokens: output,
final,
},
PRICE,
);
}
usage is optional on the Response type, hence the if (u) guard. TypeScript narrows event inside each branch, so event.delta and event.response type-check without casts.
Anthropic: input first, output last
Anthropic's stream opens with message_start, whose message.usage holds the input count. Output arrives later on message_delta, and the streaming docs put it in a warning box: the token counts in message_delta usage are cumulative. Overwrite, never add.
The docs' own example shows why the first event is a trap. Its message_start reports "input_tokens": 25, "output_tokens": 1, and the closing message_delta reports "output_tokens": 15. Log at the start and you record one output token for a fifteen-token reply.
In the SDK's MessageDeltaUsage type, input_tokens and both cache fields are number | null and described as cumulative (source). With server tools they move: the docs' web search example starts at 2,679 input tokens on message_start and ends at 10,682 on message_delta. Take the delta's value when it is not null.
// anthropic.ts
import Anthropic from "@anthropic-ai/sdk";
import { priceUsage, type OnText } from "./usage";
const MODEL = "claude-sonnet-5-5";
const PRICE = {
inputPerMTok: 2,
cachedInputPerMTok: 0.1,
outputPerMTok: 10,
};
export async function streamClaude(
prompt: string,
onText: OnText,
signal?: AbortSignal,
) {
const client = new Anthropic();
let input = 0, read = 0, write = 0, output = 0;
let final = false;
try {
const stream = await client.messages.create(
{
model: MODEL,
max_tokens: 1024,
messages: [{ role: "user", content: prompt }],
stream: true,
},
{ signal },
);
for await (const event of stream) {
if (event.type === "message_start") {
const u = event.message.usage;
input = u.input_tokens;
read = u.cache_read_input_tokens ?? 0;
write = u.cache_creation_input_tokens ?? 0;
output = u.output_tokens;
} else if (event.type === "content_block_delta") {
if (event.delta.type === "text_delta") {
onText(event.delta.text);
}
} else if (event.type === "message_delta") {
const u = event.usage;
output = u.output_tokens;
input = u.input_tokens ?? input;
read = u.cache_read_input_tokens ?? read;
write = u.cache_creation_input_tokens ?? write;
} else if (event.type === "message_stop") {
final = true;
}
}
} catch (err) {
if (!signal?.aborted) throw err;
}
return priceUsage(
{
inputTokens: input + read + write,
cachedInputTokens: read,
outputTokens: output,
final,
},
PRICE,
);
}
Note the sum at the bottom. Anthropic's input_tokens is not the total: per the prompt caching docs, total input is cache_read_input_tokens + cache_creation_input_tokens + input_tokens. OpenAI includes cached tokens in its input count and Anthropic excludes them, so one normalised type saves you from comparing two different numbers on the same dashboard.
The prices are Claude Sonnet 5.5's rates from Anthropic's pricing page as of 2026-10-17: $2 per million input, $0.10 for cache hits, $10 output. Cache writes cost more than base input there ($2.50 per million for the 5-minute tier), and OpenAI now reports cache_write_tokens too. priceUsage bills writes at the base rate, so if you use explicit caching, add a fourth rate.
If you prefer the client.messages.stream() helper, await stream.finalMessage() gives you the accumulated message with its usage. The SDK helpers doc says that promise throws if no message exists. It doesn't say what happens on abort, so don't count on finalMessage() for billing after a cancel.
The abort case: when the last event never comes
Users close tabs. Your server cancels a request when the client disconnects. Both SDKs accept an AbortSignal in the request options (openai README, Anthropic RequestOptions), and aborting makes the for await loop throw.
The OpenAI doc comment is blunt about what you lose: "If the stream is interrupted or cancelled, you may not receive the final usage chunk…" On Anthropic you keep the input count from message_start and the last cumulative output count you saw. The real bill is at least that.
That is why each function catches only when signal.aborted is true and returns final: false instead of throwing. A real network error still throws. Here is a runner, demo.ts, that can cut a stream short on purpose:
// demo.ts
import { streamChat } from "./openai-chat";
import { streamResponses } from "./openai-responses";
import { streamClaude } from "./anthropic";
const runners = {
chat: streamChat,
responses: streamResponses,
claude: streamClaude,
};
async function main() {
type Name = keyof typeof runners;
const name = (process.argv[2] ?? "chat") as Name;
const abortAfterMs = Number(process.argv[3] ?? 0);
const ac = new AbortController();
if (abortAfterMs > 0) setTimeout(() => ac.abort(), abortAfterMs);
const usage = await runners[name](
"Explain backpressure in two short paragraphs.",
(t) => process.stdout.write(t),
ac.signal,
);
console.log("\n", usage);
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
Run npx tsx demo.ts claude for a full call, then npx tsx demo.ts claude 400 to abort after 400 ms. The second run should print final: false. I haven't published output here because your token counts depend on the model version and the prompt; run it and read your own.
An aborted stream never reaches the event that carries the bill.
What to log, and what not to
Log every call, including the ones with final: false. Dropping them hides real spend; recording them as zero lies about it. Store the flag next to the numbers and let your cost dashboard sum only final rows, with a separate count of non-final calls so a spike in aborts is visible. And normalise input to "total, cached included" before any row reaches that sum, because the two vendors disagree on what input_tokens means.
If this was useful
Token cost is the first thing that surprises people once an LLM feature leaves the prototype stage, and streaming is where it hides. AI That Answers is the first book in my AI in TypeScript series: your first LLM app in TypeScript, prompts, structured output, and keeping track of what each call costs, written for developers who already ship Node code.
The rest of the series takes the same app through RAG, tool calling, agents and production: all five books here.
Top comments (0)