What One Token Actually Costs in a Node App (and 3 Lines That Show You)

#typescript #ai #observability #node

Book: AI That Answers
The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
Me: xgabriel.com | GitHub

Your Node service logs a duration and a status code for every
outbound HTTP call. It has done that since before you added AI to
anything. So when you added the LLM call, it inherited that
instrumentation, and you can now tell me exactly how long the model
took and nothing at all about what it cost.

You find out at the end of the month, in one number, for everything.

The gap is odd because the data is right there. Every response
carries a usage object. Nobody reads it, because latency was the
thing worth measuring for the previous fifteen years of backend work
and the habit carried over.

The usage object you are throwing away

The response has it already.

const res = await client.messages.create({
  model: "claude-opus-5",
  max_tokens: 1024,
  messages,
});

console.log(res.usage);
// { input_tokens, output_tokens,
//   cache_creation_input_tokens, cache_read_input_tokens }

Four numbers, and the last two matter more than people expect. Cached
input is billed differently from fresh input — that is the entire
point of prompt caching — so an implementation that adds
input_tokens + output_tokens and multiplies by one rate will report
a number that is wrong in a direction that changes decisions.

Treat those four fields as four separate line items with four
separate rates.

Rates belong in config, not in your head

This is the part to get right before the code, because it is where
posts like this usually mislead.

Per-token prices change. They differ per model, they differ per
provider, they have changed several times in the last two years, and
there are discounts (batch, cached reads) that apply to some
categories and not others. Any figure written into a blog post is a
snapshot, and any figure written into your source is a snapshot that
someone will still be trusting in eighteen months.

So: read the current numbers off your provider's pricing page, put
them in configuration, and version them.

// rates.ts — values are per 1M tokens, in USD.
// Source: your provider's pricing page. CHECK CURRENT PRICING;
// these keys exist so the numbers live in one reviewable place.
export type Rates = {
  input: number;
  output: number;
  cacheWrite: number;
  cacheRead: number;
};

export const RATES: Record<string, Rates> = {
  "claude-opus-5": loadFromConfig("claude-opus-5"),
  "claude-sonnet-5": loadFromConfig("claude-sonnet-5"),
};

Two properties worth keeping. A missing model should throw rather
than default to zero — a silent zero turns an unknown model into a
free one, and you will not notice. And the config should carry an
effective date, so a historical cost query does not get recomputed
with today's prices.

The three lines

Given rates, cost is arithmetic.

export function costOf(model: string, u: Usage): number {
  const r = RATES[model];
  if (!r) throw new UnknownModel(model);
  return (
    (u.input_tokens * r.input +
      u.output_tokens * r.output +
      (u.cache_creation_input_tokens ?? 0) * r.cacheWrite +
      (u.cache_read_input_tokens ?? 0) * r.cacheRead) / 1_000_000
  );
}

That is the whole calculation. The interesting engineering is not
here — it is in making sure this function gets called on every path.

Wrapping the client so you cannot forget

A helper you have to remember to call is a helper that gets skipped
in the code path added under deadline. Wrap the client instead.

type Meter = (e: {
  model: string;
  usage: Usage;
  costUsd: number;
  ms: number;
}) => void;

export function metered(client: Anthropic, meter: Meter) {
  return {
    async create(params: MessageCreateParams) {
      const started = performance.now();
      const res = await client.messages.create(params);
      meter({
        model: params.model,
        usage: res.usage,
        costUsd: costOf(params.model, res.usage),
        ms: performance.now() - started,
      });
      return res;
    },
  };
}

Now every call through metered(...) is accounted for, and a new
call site cannot opt out by forgetting. Export the wrapped client
from a module and do not export the raw one.

Attaching cost to the request

A per-call number is not yet useful. What you want is cost per
request, per user, per feature — which means accumulating across
however many model calls one HTTP request makes.

AsyncLocalStorage is the Node primitive for this. It gives you
request-scoped state without threading a context object through every
function signature.

import { AsyncLocalStorage } from "node:async_hooks";

type Ledger = { costUsd: number; calls: number };
export const ledger = new AsyncLocalStorage<Ledger>();

export function costMiddleware(req, res, next) {
  const entry: Ledger = { costUsd: 0, calls: 0 };
  res.on("finish", () => {
    logger.info("request cost", {
      route: req.route?.path,
      userId: req.user?.id,
      costUsd: +entry.costUsd.toFixed(6),
      calls: entry.calls,
      status: res.statusCode,
    });
  });
  ledger.run(entry, () => next());
}

And the meter writes into whatever ledger is active:

const meter: Meter = (e) => {
  const entry = ledger.getStore();
  if (entry) {
    entry.costUsd += e.costUsd;
    entry.calls += 1;
  }
};

Six decimal places, not two. Individual calls are frequently under a
cent, and rounding to cents per call turns a real number into a
column of zeros.

What the log gives you

Once every request carries a cost, questions that were unanswerable
become one query.

Which route costs the most per call — not in total, per call. That is
where a prompt is doing more work than the feature justifies.

Which users are expensive. In a per-seat product, a small number of
users generating a large share of spend is a pricing question, and
you cannot raise it without the number.

Cost per successful outcome. Requests that end in a 4xx after three
model calls are pure loss, and they do not show up in a total-spend
chart at all.

And the ratio of cache reads to fresh input, which tells you whether
prompt caching is doing anything. A cache hit rate you assumed was
high and is not is a common and expensive surprise.

One thing to avoid

Do not estimate tokens client-side with a tokenizer to predict cost
before the call. It is tempting, and it drifts: tokenizer versions
change, system prompts get injected, tool definitions count toward
input, and cached segments are billed differently. The usage object
is what you are billed on. Measure that.

Estimating ahead of time is a legitimate thing to do for limits —
refusing to send a request that is obviously too large. It is not a
substitute for measuring what actually happened.

If this was useful

AI That Answers covers the
cost side of a first LLM app properly — what each token category
means, where caching changes the arithmetic, and how to build the
accounting in before you need it rather than after the invoice.