DEV Community

Agent Island
Agent Island

Posted on • Originally published at agent-island.dev

An AI Coding Cost Tracker Needs a Measurement Contract

#ai #programming #productivity #opensource

The most dangerous number in an AI coding dashboard is the one labeled cost without a definition.

A local tool can reconstruct token activity from Claude Code and Codex session records. It can apply a dated model-price table. That produces an estimate with a clear use: comparing periods and understanding the shape of local activity.

It does not produce an invoice.

Start at the file-format boundary

Claude Code and Codex do not write identical usage records. Claude can expose input, output, cache creation, and cache-read fields on assistant messages. Codex can separate model context from later token events, and cached input may be included inside total input.

Normalize those formats before calculating anything:

provider
timestamp
model
input tokens
output tokens
cache creation tokens
cache read tokens
Enter fullscreen mode Exit fullscreen mode

Once that boundary exists, the rest of the report does not need to guess what a raw provider field means.

Cached input is not ordinary input twice

For a Codex event, a safe calculation starts by separating cached input:

non-cached input = max(total input - cached input, 0)
Enter fullscreen mode Exit fullscreen mode

Then price non-cached input, output, cache creation, and cache reads with their own rates. Pricing total input and adding cache reads again creates a precise-looking overcount.

This is why a generic tokens * price function is not enough for a multi-provider tracker.

A rescan must not create new usage

Session files are read repeatedly. Apps restart. Watchers reconnect. Archived files can reappear. If the same event is counted every time it is observed, the weekly report grows while the underlying work stays unchanged.

Replay protection is part of accounting. Use stable provider event identifiers when they exist. If a format lacks one, document the fallback and its uncertainty instead of hiding it behind a polished total.

Date the price table

New model identifiers can appear before a desktop app knows their rates. API prices can also change.

A tracker should:

  • ship a dated rate snapshot;
  • keep unknown named models visible;
  • leave them unpriced rather than guessing a nearby rate;
  • preserve the rate date with historical reports.

In Agent Island, the calculated number is labeled API value. It is a counterfactual estimate under the embedded rates.

Keep four answers separate

A useful dashboard should not collapse these into one metric:

  1. Provider quota: how close the account is to a reset boundary.
  2. Local token volume: what the session records contain.
  3. Estimated API value: a dated calculation over those token categories.
  4. Actual billing: receipts, credits, subscriptions, and provider-side adjustments.

Only the fourth is a bill. The local records do not contain enough context to recreate it.

The evaluation checklist

Before trusting an AI coding cost tracker, ask:

  • Which exact local records does it read?
  • Does it separate input, output, cache creation, and cache reads?
  • How does it prevent replayed events from being counted twice?
  • What happens when a model is missing from the rate table?
  • Is the price snapshot dated?
  • Does the UI say estimate, or does it imply actual spend?
  • Is collection local, and is sharing a separate action?

The implementation can be sophisticated, but the contract should be easy to explain. If the number can be mistaken for money paid, the explanation belongs next to the number.

The full measurement contract and current Agent Island scope are in the canonical guide: AI coding cost tracker.

Top comments (0)