Tracking LLM cost per customer in Node.js with AsyncLocalStorage

Your OpenAI invoice tells you what you spent. It doesn't tell you which customer you spent it on. If you sell a flat-price plan on top of an LLM, that second number is the one that decides whether a customer is profitable.

The usual first attempt is to add a customerId parameter to every function that might end up calling the model. That works until the call sits five layers down, inside a retry helper, a summarizer, and a background job. This post shows a smaller approach:

1. Put the customer on the request's async context once, in middleware.
2. Wrap the OpenAI client once, so every call reads that context and logs its cost.
3. Store one row per call and use SQL for the questions you actually have.

Everything here is plain Node 18+ and the official openai package. No proxy, no extra service.

1. Put the customer on the async context

Node's AsyncLocalStorage carries a value through every await, timer and promise that starts inside a run() call. That's exactly the scope of "this HTTP request".

// context.ts
import { AsyncLocalStorage } from "node:async_hooks";

export interface CostContext {
  customerId: string;
  plan?: string;
  feature?: string; // "chat", "summarize", "search"... useful later
}

export const costContext = new AsyncLocalStorage<CostContext>();

export function currentCostContext(): CostContext | undefined {
  return costContext.getStore();
}

In Express (any framework with middleware works the same way):

app.use((req, res, next) => {
  const user = req.user; // whatever your auth sets
  if (!user) return next();
  costContext.run({ customerId: user.id, plan: user.plan }, next);
});

To tag a feature, nest another run() around the code that implements it:

export function withFeature<T>(feature: string, fn: () => Promise<T>): Promise<T> {
  const parent = currentCostContext();
  if (!parent) return fn();
  return costContext.run({ ...parent, feature }, fn);
}

// in a route handler
const summary = await withFeature("summarize", () => summarizeThread(thread));

Background jobs don't go through your middleware, so start a context there too, using the customer the job belongs to:

queue.process(async (job) =>
  costContext.run({ customerId: job.data.customerId, feature: "nightly-digest" }, () =>
    runDigest(job.data),
  ),
);

2. Price each call

Token counts aren't comparable across models, so convert to dollars at write time. Keep prices in one table with the date you read them, because they change.

// prices.ts: USD per 1M tokens. Check https://developers.openai.com/api/docs/pricing before relying on these.
export const PRICES: Record<string, { input: number; cachedInput: number; output: number }> = {
  "gpt-5.5":      { input: 5.0,  cachedInput: 0.5,   output: 30.0 },
  "gpt-5.4-mini": { input: 0.75, cachedInput: 0.075, output: 4.5 },
};

export function priceFor(model: string) {
  if (PRICES[model]) return PRICES[model];
  // Responses report dated snapshots like "gpt-5.4-mini-2026-03-17"; fall back to the base name.
  const base = model.replace(/-\d{4}-\d{2}-\d{2}$/, "");
  return PRICES[base];
}

export function costUsd(model: string, usage: {
  prompt_tokens: number;
  completion_tokens: number;
  prompt_tokens_details?: { cached_tokens?: number } | null;
}): number | null {
  const p = priceFor(model);
  if (!p) return null; // unknown model: log it, don't guess
  const cached = usage.prompt_tokens_details?.cached_tokens ?? 0;
  const uncached = usage.prompt_tokens - cached;
  return (uncached * p.input + cached * p.cachedInput + usage.completion_tokens * p.output) / 1_000_000;
}

Two details that cause most wrong numbers:

• Cached input tokens are included in prompt_tokens and billed at a lower rate. If you price all of prompt_tokens at the full rate, you'll over-count, sometimes by a lot for chat apps with long system prompts.
• Unknown models should produce null, not zero. A zero looks like a free call and hides the gap; a null shows up in a query.

3. Wrap the client once

// openai.ts
import OpenAI from "openai";
import { currentCostContext } from "./context";
import { costUsd } from "./prices";
import { recordUsage } from "./usage-log";

const client = new OpenAI();

export async function chat(params: OpenAI.Chat.ChatCompletionCreateParamsNonStreaming) {
  const res = await client.chat.completions.create(params);
  const ctx = currentCostContext();
  if (res.usage) {
    // Don't let logging failures break the user's request.
    recordUsage({
      customerId: ctx?.customerId ?? "unattributed",
      plan: ctx?.plan,
      feature: ctx?.feature,
      model: res.model,
      promptTokens: res.usage.prompt_tokens,
      cachedTokens: res.usage.prompt_tokens_details?.cached_tokens ?? 0,
      completionTokens: res.usage.completion_tokens,
      costUsd: costUsd(res.model, res.usage),
    }).catch((err) => console.error("usage log failed", err));
  }
  return res;
}

Note res.model, not params.model: the response tells you the model that actually ran, which is what you're billed for.

Calls with no context get logged as "unattributed" rather than dropped. Watching that bucket is the quickest way to find code paths that bypass your middleware.

Streaming

With stream: true, usage is only sent if you ask for it, and it arrives in the last chunk:

export async function* chatStream(params: OpenAI.Chat.ChatCompletionCreateParamsStreaming) {
  const ctx = currentCostContext(); // read it before the first yield
  const stream = await client.chat.completions.create({
    ...params,
    stream_options: { ...params.stream_options, include_usage: true },
  });
  let usage: OpenAI.CompletionUsage | undefined;
  let model = params.model;
  try {
    for await (const chunk of stream) {
      if (chunk.usage) usage = chunk.usage;
      if (chunk.model) model = chunk.model;
      yield chunk;
    }
  } finally {
    // runs on normal completion *and* when the consumer stops reading early
    if (usage) {
      recordUsage({
        customerId: ctx?.customerId ?? "unattributed",
        plan: ctx?.plan,
        feature: ctx?.feature,
        model,
        promptTokens: usage.prompt_tokens,
        cachedTokens: usage.prompt_tokens_details?.cached_tokens ?? 0,
        completionTokens: usage.completion_tokens,
        costUsd: costUsd(model, usage),
      }).catch((err) => console.error("usage log failed", err));
    }
  }
}

If the client disconnects before the final chunk, there's no usage to record. Decide whether that matters for you; for most apps it's a small, known gap.

4. Store one row per call

create table llm_usage (
  id               bigserial primary key,
  at               timestamptz not null default now(),
  customer_id      text not null,
  plan             text,
  feature          text,
  model            text not null,
  prompt_tokens    int not null,
  cached_tokens    int not null,
  completion_tokens int not null,
  cost_usd         numeric(12, 6) -- null = model missing from the price table
);
create index on llm_usage (customer_id, at);

// usage-log.ts
import { pool } from "./db";

export async function recordUsage(u: {
  customerId: string; plan?: string; feature?: string; model: string;
  promptTokens: number; cachedTokens: number; completionTokens: number; costUsd: number | null;
}) {
  await pool.query(
    `insert into llm_usage (customer_id, plan, feature, model, prompt_tokens, cached_tokens, completion_tokens, cost_usd)
     values ($1, $2, $3, $4, $5, $6, $7, $8)`,
    [u.customerId, u.plan ?? null, u.feature ?? null, u.model, u.promptTokens, u.cachedTokens, u.completionTokens, u.costUsd],
  );
}

Store only metadata. You don't need prompts or completions to answer cost questions, and keeping them makes this table a privacy problem.

5. The queries you'll actually run

Top customers this month:

select customer_id, plan, sum(cost_usd) as cost, count(*) as calls
from llm_usage
where at >= date_trunc('month', now())
group by customer_id, plan
order by cost desc nulls last
limit 20;

Which feature is expensive:

select feature, model, sum(cost_usd) as cost, avg(prompt_tokens)::int as avg_prompt
from llm_usage
where at >= now() - interval '7 days'
group by feature, model
order by cost desc nulls last;

A high avg_prompt on one feature usually means it's sending a whole document or conversation history when a summary would do.

Cost vs. revenue, if you keep each customer's monthly price in a subscriptions table:

select u.customer_id, s.monthly_price_usd as revenue, sum(u.cost_usd) as ai_cost,
       s.monthly_price_usd - sum(u.cost_usd) as margin
from llm_usage u
join subscriptions s on s.customer_id = u.customer_id
where u.at >= date_trunc('month', now())
group by u.customer_id, s.monthly_price_usd
order by margin asc
limit 20;

The top rows of that last query are the conversation you need to have with your pricing page.

Models missing from your price table:

select model, count(*) from llm_usage where cost_usd is null group by model;

Where this stops

This setup answers "who cost what", after the fact. It doesn't stop a customer mid-month. For that you need to read the running total before each call and refuse, or switch to a cheaper model, when it crosses a budget. That check has to be fast and safe under concurrent requests, which is a different problem from logging (I wrote about it in How to stop one customer from blowing up your OpenAI bill).

Disclosure: I maintain llmbrake, a small MIT-licensed TypeScript library that does the pricing and the before-call budget check for OpenAI-compatible chat.completions calls. You don't need it for anything in this post; the code above is self-contained.