Build Request-Level Cost Ledgers for AI API Routes

When an AI application uses one OpenAI-compatible client across several Chinese model routes, the hard part is rarely the first request. The hard part is explaining the bill after a rollout, especially when a route has cache-hit input, uncached input, output tokens, retries, tools, and dated pricing evidence.

That explanation should not live in a spreadsheet that someone updates when finance asks a question. It should be written at request time, next to the route decision that created the spend.

This article walks through a compact request-level cost ledger. It is meant for teams evaluating routes such as DeepSeek, GLM, Qwen, Kimi, MiniMax, Doubao, StepFun, and other Chinese model families through a single OpenAI-compatible API. The examples use public pricing evidence checked on September 21, 2026 against AIWave's live pricing endpoint, but the pattern works for any gateway or direct provider setup where prices and model catalogs can change over time.

The ledger row

A useful ledger row answers five questions:

• Which model route handled the request?
• What pricing version was used when the request ran?
• How many input, cache-hit input, output, and tool-related tokens were billed?
• Which retry or fallback path changed the final cost?
• Which release or canary decision allowed the request to reach that route?

That leads to a schema like this:

type RouteCostLedgerRow = {
  request_id: string
  workflow_id: string
  route_id: string
  model: string
  provider_family: string
  pricing_checked_at: string
  pricing_source: string
  input_tokens: number
  cache_hit_input_tokens: number
  output_tokens: number
  tool_tokens: number
  retry_count: number
  fallback_from?: string
  estimated_usd: number
  final_usd?: number
  release_id: string
  policy_id: string
  decision_reason: string
}

There are two important details in that row.

First, pricing_checked_at and pricing_source are not decoration. They make the row auditable. If the route price changes next week, yesterday's decision should still point to the price evidence that existed when the request ran.

Second, estimated_usd and final_usd are separate. The estimate is what your router believed before or during dispatch. The final value is what your billing system confirms after the request settles. If they drift, that difference becomes an engineering signal rather than a support surprise.

Capture route evidence before the call

The route decision should be logged before the upstream call leaves your system. That record does not need to contain the prompt or response body. In fact, it should avoid them. A routing ledger is about operational evidence, not content retention.

function startLedgerRow(input: {
  requestId: string
  workflowId: string
  model: string
  providerFamily: string
  pricingVersion: string
  releaseId: string
  policyId: string
  reason: string
}): RouteCostLedgerRow {
  return {
    request_id: input.requestId,
    workflow_id: input.workflowId,
    route_id: `${input.providerFamily}:${input.model}`,
    model: input.model,
    provider_family: input.providerFamily,
    pricing_checked_at: input.pricingVersion,
    pricing_source: "https://aiwave.live/api/pricing",
    input_tokens: 0,
    cache_hit_input_tokens: 0,
    output_tokens: 0,
    tool_tokens: 0,
    retry_count: 0,
    estimated_usd: 0,
    release_id: input.releaseId,
    policy_id: input.policyId,
    decision_reason: input.reason,
  }
}

The decision_reason should be boring and specific:

• default-route-for-chat
• long-context-route
• cache-sensitive-agent-run
• canary-route-5-percent
• fallback-after-timeout

Avoid vague labels such as smart-router or optimized. They sound nice in a deck, but they do not help an engineer explain why a request moved.

Estimate with dated prices

At run time, your router can hold a pricing snapshot in memory. The snapshot should be refreshed on a schedule and versioned by retrieval time. A practical estimator only needs a few fields:

type RoutePrice = {
  model: string
  inputUsdPerMillion: number
  cacheHitUsdPerMillion?: number
  outputUsdPerMillion: number
  pricingCheckedAt: string
}

function estimateCostUsd(row: RouteCostLedgerRow, price: RoutePrice) {
  const input = (row.input_tokens / 1_000_000) * price.inputUsdPerMillion
  const cacheHit =
    ((row.cache_hit_input_tokens || 0) / 1_000_000) *
    (price.cacheHitUsdPerMillion ?? price.inputUsdPerMillion)
  const output = (row.output_tokens / 1_000_000) * price.outputUsdPerMillion
  return Number((input + cacheHit + output).toFixed(8))
}

Notice the cache behavior. If a route publishes a cache-hit line, use it explicitly. If it does not, fall back to the normal input rate in the estimate and mark cache-specific savings as unknown. That is more honest than inventing a cache discount from the model name.

For AIWave, the live pricing endpoint checked during this run exposed current route rows for examples including deepseek-v4-pro, deepseek-v4-flash, glm-5, qwen3-max, MiniMax-M3, and doubao-seed-2-1-pro-260628. Treat that as a dated route-card observation, not a permanent catalog promise.

Separate retries from fallbacks

Retries and fallbacks have different meanings.

A retry says: the same route was attempted again because the previous attempt failed or timed out.

A fallback says: the router changed route after a condition was met.

Your ledger should keep those apart because they create different actions. Too many retries might point to timeout settings, regional connectivity, or request size. Too many fallbacks might point to an unhealthy primary route or a policy that is too aggressive.

function recordRetry(row: RouteCostLedgerRow) {
  row.retry_count += 1
}

function recordFallback(row: RouteCostLedgerRow, previousRoute: string) {
  row.fallback_from = previousRoute
  row.decision_reason = "fallback-after-timeout"
}

The distinction matters for cost review. If a request paid twice because of a retry, the fix may be idempotency and timeout tuning. If it paid more because of a fallback, the fix may be a narrower policy or a clearer route budget.

Put gates in front of route changes

Most teams review model quality before a route change. Fewer teams review cost variance with the same discipline.

Before moving a production workload to a different route, run a gate like this:

1. Replay a recent sample without storing request bodies in the ledger.
2. Compare estimated cost per workflow, not only per request.
3. Break out cache-hit input, uncached input, output, tool tokens, retries, and fallbacks.
4. Require a signed pricing snapshot date.
5. Ship the change as a canary with an automatic stop condition.

The stop condition should be written in operational language:

type CostGate = {
  releaseId: string
  maxWorkflowCostIncreasePct: number
  maxRetryRatePct: number
  maxFallbackRatePct: number
  sampleWindowMinutes: number
}

function shouldStopCanary(metrics: {
  workflowCostIncreasePct: number
  retryRatePct: number
  fallbackRatePct: number
}, gate: CostGate) {
  return (
    metrics.workflowCostIncreasePct > gate.maxWorkflowCostIncreasePct ||
    metrics.retryRatePct > gate.maxRetryRatePct ||
    metrics.fallbackRatePct > gate.maxFallbackRatePct
  )
}

This avoids a common failure mode: a new route looks fine in average request cost, but the whole workflow becomes more expensive because it produces longer outputs, triggers more tools, or causes a second pass in the agent loop.

Keep the ledger private, publish the method

A cost ledger will contain sensitive operational metadata. It may reveal release IDs, internal workflow names, route policy names, or workload shape. Keep the raw rows private.

What you can publish safely is the method:

• pricing evidence is dated
• request bodies are not stored in the ledger
• estimates and settled values are separated
• retries and fallbacks are separated
• canaries include cost variance stop conditions

That gives developers and procurement reviewers a way to ask precise questions without exposing private traffic.

Reconcile by workflow, not by invoice line

The most useful review happens one level above a single request. A coding agent, RAG pipeline, data extraction job, or customer support assistant usually makes several calls to finish one unit of work. If you only inspect individual requests, you can miss the cost shape that actually matters.

Create a daily reconciliation job that groups ledger rows by workflow_id, release_id, and policy_id:

type WorkflowCostSummary = {
  workflow_id: string
  release_id: string
  policy_id: string
  request_count: number
  retry_count: number
  fallback_count: number
  estimated_usd: number
  final_usd: number
  variance_usd: number
}

The job should produce three views.

First, show release variance. If release-2026-09-21-a changes the median workflow cost, the owning team should see it before the next rollout expands.

Second, show route variance. A route can be technically healthy while still producing a longer answer shape, more tool calls, or more retries than the route it replaced.

Third, show policy variance. If a fallback policy saves latency but creates a higher settled cost for a workflow, that is a real trade-off. It may still be the right trade-off. The point is to make it visible.

This is also where you compare estimates with settled values:

function varianceUsd(summary: {
  estimated_usd: number
  final_usd: number
}) {
  return Number((summary.final_usd - summary.estimated_usd).toFixed(8))
}

If variance is consistently positive, do not patch it by changing a dashboard label. Check whether your pricing snapshot is stale, cache-hit accounting is missing, retries are being estimated once but billed multiple times, or output token counts are arriving after the estimate is written.

Decide what never enters the ledger

The ledger should be intentionally incomplete. Do not store prompts, responses, raw user identifiers, payment details, or reusable API keys in a cost ledger. You can usually answer the operational question with route ID, token counts, pricing version, release ID, and workflow ID.

For debugging, store a request hash or a short-lived trace pointer instead of the body. Keep the body in the system that already owns request observability and retention controls. The cost ledger should remain narrow enough that engineers, finance, and procurement can inspect it without turning it into a sensitive content database.

A simple rollout checklist

Before you send a multi-route AI workload into production, ask:

• Can every request point to the route and pricing snapshot used at dispatch time?
• Can you explain cache-hit input separately from uncached input?
• Can you identify which release introduced a cost change?
• Can you tell whether retries or fallbacks created the variance?
• Can you stop a canary based on workflow-level cost, not only error rate?

If the answer is no, do not start with a new pricing argument. Start with evidence capture.

Pricing will keep changing. Model catalogs will keep changing. Provider capabilities will keep changing. A request-level ledger will not make those changes disappear, but it gives your team a stable way to reason about them.

That is the real operational win: every route decision becomes explainable after the fact.