Rajesh Kumar Kanumetta

Posted on Jul 10

Track Every LLM Token in Node.js with all-llm-token-tracker

#llm #monitoring #node #showdev

Tags (pick up to 4)

node
javascript
openai
llm

(Alternates: typescript, npm, webdev)

Cover image (optional)

Use a dashboard screenshot or a simple banner with:
"Track input/output LLM tokens in Node.js"

Article body (paste below the title)

If you build with OpenAI, Anthropic, or any LLM API, you've probably wondered:

How many tokens did that request actually use?

Token usage drives cost, performance, and scaling. But in many Node.js apps, response.usage gets logged once and forgotten.

I built all-llm-token-tracker — a small npm package that tracks input and output tokens for every LLM call, with pluggable storage and built-in provider extractors.

npm: https://www.npmjs.com/package/all-llm-token-tracker

GitHub: https://github.com/rkanumetta/all-llm-token-tracker

The problem

Most integrations look like this:

const response = await openai.chat.completions.create({ ... });
console.log(response.usage); // logged, rarely stored

That works for debugging — not for production. You usually need:

Per-call input and output token counts
Persistent storage you can query later
Summaries by model, provider, or user
Something that works in dev and production

Many existing tools bundle pricing engines, MCP servers, or heavy infra. Sometimes you just want clean token tracking.

The solution

all-llm-token-tracker does one job well:

Feature	Supported
Manual token recording	✅
Auto-wrap LLM calls	✅
OpenAI extractor	✅
Anthropic extractor	✅
Memory storage	✅
File (JSON) storage	✅
MongoDB storage	✅
Query & summarize	✅
Browser dashboard	✅
Zero required deps	✅

Requirements: Node.js 18+

License: MIT

Install

npm install all-llm-token-tracker

MongoDB storage (optional):

npm install all-llm-token-tracker mongodb

Quick start: manual recording

import { createTracker } from 'all-llm-token-tracker';

const tracker = createTracker({ storage: 'memory' });

const record = await tracker.record({
  provider: 'openai',
  model: 'gpt-4o-mini',
  inputTokens: 120,
  outputTokens: 45,
  metadata: { userId: 'user-123' },
});

console.log(record.totalTokens); // 165

Each record gets a UUID, timestamp, and optional metadata.

Auto-track LLM calls (recommended)

Wrap your existing function — usage is recorded automatically:

import {
  createTracker,
  extractOpenAiUsage,
} from 'all-llm-token-tracker';
import OpenAI from 'openai';

const tracker = createTracker({
  storage: 'file',
  file: { filePath: './.llm-usage/usage.json' },
});

const openai = new OpenAI();

const { result, record } = await tracker.track(
  () =>
    openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: 'Hello!' }],
    }),
  {
    provider: 'openai',
    model: 'gpt-4o-mini',
    extractUsage: extractOpenAiUsage,
    metadata: { feature: 'chat' },
  }
);

console.log(result.choices[0].message.content);
console.log(record.inputTokens, record.outputTokens);

What happens:

Timer starts
Your LLM function runs
Usage is extracted from the response
Record is saved to storage
You get { result, record } back

No changes to your core LLM logic — just a wrapper.

Anthropic support

import { extractAnthropicUsage } from 'all-llm-token-tracker';

await tracker.track(
  () => anthropic.messages.create({
    model: 'claude-3-5-sonnet-20241022',
    max_tokens: 1024,
    messages: [{ role: 'user', content: 'Hello!' }],
  }),
  {
    provider: 'anthropic',
    model: 'claude-3-5-sonnet-20241022',
    extractUsage: extractAnthropicUsage,
  }
);

Storage options

Memory — tests & scripts

const tracker = createTracker({ storage: 'memory' });

File — local JSON

const tracker = createTracker({
  storage: 'file',
  file: { filePath: './.llm-usage/usage.json' },
});

MongoDB — production

const tracker = createTracker({
  storage: 'mongodb',
  mongodb: {
    uri: process.env.MONGODB_URI,
    database: 'my_app',
    collection: 'llm_token_usage',
  },
});

// on shutdown
await tracker.close();

Query & summarize

const calls = await tracker.getCalls({
  provider: 'openai',
  from: '2026-07-01',
  limit: 100,
});

const summary = await tracker.getSummary();
/*
{
  totalCalls: 42,
  totalInputTokens: 12500,
  totalOutputTokens: 3200,
  totalTokens: 15700,
  averageInputTokens: 297.6,
  averageOutputTokens: 76.2,
  averageTotalTokens: 373.8
}
*/

const byModel = await tracker.getSummaryByModel();
const byProvider = await tracker.getSummaryByProvider();

Record shape

Every call is stored like this:

{
  "id": "uuid",
  "provider": "openai",
  "model": "gpt-4o-mini",
  "inputTokens": 120,
  "outputTokens": 45,
  "totalTokens": 165,
  "timestamp": "2026-07-10T06:00:00.000Z",
  "durationMs": 842,
  "metadata": { "userId": "user-123" }
}

Custom providers

import { createUsageExtractor } from 'all-llm-token-tracker';

const extractMyProvider = createUsageExtractor((response) => ({
  inputTokens: response.tokens.in,
  outputTokens: response.tokens.out,
}));

Or plug in your own storage (Postgres, Redis, etc.):

import { createTracker, type StorageAdapter, type LlmCallRecord } from 'all-llm-token-tracker';

class PostgresStorage implements StorageAdapter {
  async save(record: LlmCallRecord) { /* ... */ }
  async find(query) { /* ... */ }
  async count(query) { /* ... */ }
}

const tracker = createTracker({ storage: new PostgresStorage() });

Bonus: browser dashboard

The repo includes a simple dashboard to visualize input vs output tokens:

git clone https://github.com/rkanumetta/all-llm-token-tracker.git
cd all-llm-token-tracker
npm install
npm run example
npm run dashboard

Open http://localhost:4173

You'll see:

Total calls, input tokens, output tokens
Bar chart per call
Breakdown by model
Filterable table with metadata

Real-world use cases

Cost monitoring — track tokens per feature or user
Debugging — find which prompts burn the most tokens
A/B testing — compare models side by side
SaaS billing — feed data into your own pricing logic
Compliance — audit trail with timestamps + metadata

Why I built it this way

Small surface area — one responsibility: track tokens
Storage flexibility — file in dev, MongoDB in prod
Provider-agnostic — extend with custom extractors
TypeScript-first — full types included
Lightweight — MongoDB is optional, zero required runtime deps

Try it

npm install all-llm-token-tracker

Links:

📦 npm: https://www.npmjs.com/package/all-llm-token-tracker
🐙 GitHub: https://github.com/rkanumetta/all-llm-token-tracker
🐛 Issues: https://github.com/rkanumetta/all-llm-token-tracker/issues

If you find it useful, star the repo ⭐ and open an issue if you want support for another provider (Gemini, Mistral, etc.).

Wrapping up

LLM token tracking doesn't need to be complicated. Wrap your API calls, store consistent records, query when you need insights.

Track input. Track output. Store it. Query it. Done.

What storage backend do you use for LLM observability — file, MongoDB, or something else? Drop a comment below.

DEV.to publishing checklist

Go to dev.to/new
Paste the title
Paste the article body (from "If you build with OpenAI..." through the closing question)
Add tags: node, javascript, openai, llm
Add a cover image (dashboard screenshot helps clicks)
Set canonical URL only if you also published on Medium (avoid duplicate SEO issues)
Click Publish

Optional: shorter subtitle for DEV

Paste this as the first line under the title if DEV shows a preview/subtitle field:

A lightweight Node.js npm package to track input/output tokens for OpenAI, Anthropic, and custom LLM providers — with memory, file, and MongoDB storage plus a browser dashboard.

DEV Community