How to Compare Pricing and Token Cost for One API Key Across a Multi-Provider Router

How Can One API Key Across Model Providers Preserve the Cost Signal?

Short answer: one API key across OpenAI, Claude, and Gemini does not automatically create comparable telemetry. Instrument the runtime around a request's lifecycle, preserve the dimensions that explain work, and aggregate from immutable events rather than from a single “cost” field. The useful unit is not a dashboard number. It is a traceable relationship between an input, the work performed, the result delivered, and the resources consumed.

That sounds straightforward until a production trace contains retries, streaming output, tool calls, batching, and a provider response that arrives in more than one piece. A telemetry design that records only latency and a final token count can look healthy while hiding the work that produced the result.

What should one API key across model providers compare?

Start with an operational question, not with a list of available metrics. For example:

Why did this class of request take longer, use more model work, or produce a different result than expected?

That question defines the minimum context an event must carry. A timestamp without a request identity is hard to connect. A request identity without an attempt identity cannot distinguish a retry from a second user request. A token count without a model or operation name cannot support a useful comparison.

I use three levels of identity:

1. Request identifies the user-visible operation.
2. Attempt identifies one execution of that operation, including retries.
3. Span identifies a step such as queueing, model invocation, tool execution, or response assembly.

The hierarchy matters because the totals are different. A request can have one successful attempt and one failed attempt. A model invocation can produce several streamed chunks. A tool span can be retried independently of the model span. Flattening these into one row makes the arithmetic look tidy and the diagnosis unreliable.

Which fields preserve the explanation across providers?

The core event should be boring and explicit. A compact record can carry the fields below:

{
  "request_id": "req_01J7Q4M2",
  "attempt_id": "att_01J7Q4N8",
  "span_id": "spn_01J7Q4P1",
  "operation": "model.generate",
  "model": "runtime-model-id",
  "status": "ok",
  "started_at": "2026-10-26T08:10:12.104Z",
  "ended_at": "2026-10-26T08:10:13.991Z",
  "input_units": 1840,
  "output_units": 276,
  "retry_count": 0,
  "streamed": true
}

The exact unit depends on the runtime and the model interface. The important property is consistency: the same field must mean the same thing across comparable events. If one producer reports estimated input units and another reports provider-confirmed units, label them separately instead of silently mixing them.

Dimensions explain variation; measurements support aggregation. I keep high-cardinality values such as raw prompts, generated text, and arbitrary user metadata out of metric labels. They belong in a controlled trace or log record, subject to the application's privacy policy. A metric series should remain bounded enough to query during an incident.

How do retries and a router change the cost picture?

Retries are where many otherwise reasonable dashboards become misleading. If a request succeeds after two failed attempts, the user sees one result, but the runtime performed three attempts. A request-level “successful” counter and an attempt-level work counter are both correct; they answer different questions.

Record the retry reason and the parent attempt. Then calculate at least two totals:

• Delivered work: work associated with the attempt that produced the user-visible result.
• Performed work: work from every attempt, including unsuccessful attempts and cancelled streams.

Do not infer performed work by multiplying successful requests by an average retry count. The distribution is usually uneven, and that shortcut erases the very conditions operators need to find.

An event can represent this relationship without special casing the dashboard:

request=req_01
  attempt=att_01 status=timeout retry_reason=deadline
    span=model.generate input_units=1800 output_units=0
  attempt=att_02 status=ok retry_reason=deadline
    span=model.generate input_units=1800 output_units=290

The second attempt is the delivered attempt. Both attempts are performed work. Keeping that distinction makes cancellation, timeout, and retry analysis possible without rewriting history.

Streaming introduces a tempting but dangerous shortcut: count each chunk as a complete response. A chunk is an observation, not a new request. The runtime should attach chunks to one span and close that span once the stream reaches a terminal state.

At minimum, record the terminal state as one of the states the runtime actually supports, such as completed, cancelled, or failed. Store cumulative usage when it is available. When it is not available, preserve the fact that the usage is partial rather than presenting an estimate as a confirmed total.

The sequence below is enough to reconstruct the lifecycle:

const stream = startModelSpan({ requestId, attemptId });

try {
  for await (const chunk of modelStream) {
    stream.observeChunk({ bytes: chunk.length });
    yield chunk;
  }
  stream.end({ status: "completed", usage: getFinalUsage() });
} catch (error) {
  stream.end({ status: "failed", errorType: classify(error) });
  throw error;
}

The code does not turn a partial observation into a precise bill. It makes the uncertainty visible, which is more useful for operations and later reconciliation.

How can a startup app connect telemetry to quality?

Operational cost is not a quality metric by itself. A lower-work response can be wrong, incomplete, or unusable. Join runtime telemetry with an outcome record that contains only the evaluation fields needed for the workflow: for example, whether a required tool call completed, whether a response passed a validation step, or whether a human reviewer accepted it.

Keep the join key stable and avoid copying sensitive content into every telemetry event. A useful analysis can compare outcome rate, latency, performed work, and retry rate by operation and model version without exposing the full request.

This is also where sampling deserves care. If slow or failed requests are sampled less often than ordinary requests, the resulting averages are optimistic. Tail events should have a higher retention priority than routine successes. Sampling policy is part of the measurement design, not merely a storage setting.

A router can make this comparison harder, not easier. It may select OpenAI for one request, Claude for another, or Gemini after a policy check. Record the selected route and the reason at the attempt level. Otherwise, a change in the router can look like a model regression.

There is a real tradeoff here. One API key and one common event shape simplify an app's instrumentation, but provider-specific features and usage semantics still differ. A small startup may prefer a direct integration while its traffic is simple; a router adds coordination and another failure boundary. The shared schema is useful only when it keeps those differences visible.

What does a trustworthy cost view look like?

A trustworthy view shows provenance and scope beside the aggregate. I expect to see the time window, the population included, whether retries are included, whether usage is confirmed or estimated, and the runtime version that emitted the events.

For a daily review, a small table is often more useful than a single score:

Slice	Requests	Delivered work	Performed work	Retry rate	Outcome rate
Operation A	12,480	100%	104%	3.1%	96.8%
Operation B	3,210	100%	119%	11.4%	91.2%

The percentages above are illustrative shapes, not a pricing claim. The point is the comparison: Operation B has a larger gap between delivered and performed work and a weaker outcome rate. That is a prompt to inspect deadlines, retry policy, input growth, and model behavior together.

Avoid turning a changing provider price into the central design assumption. Store the measurable runtime quantities first. If a financial system later supplies a rate or allocation rule, the calculation can be reproduced from those quantities and its assumptions can be audited.

Common telemetry mistakes I would look for in review

One record per user request hides attempts and child operations. Use a request record for the user-visible lifecycle, then attach attempts and spans.

A mutable “current cost” field overwrites evidence. Append events or retain revisions so an aggregate can be recomputed.

Retry counters without retry reasons tell you that work repeated, but not why.

Unlimited labels make metric storage expensive and queries unstable. Put request IDs, prompts, and exception text in traces or logs with retention controls.

Sampling before classification can discard the most valuable evidence. Identify failure, cancellation, or high latency before applying the sampling policy.

Mixing estimates and confirmed usage is another boundary. A rough value is useful when marked as rough; it becomes harmful when indistinguishable from a confirmed measurement.

Before treating the telemetry as an operational source of truth, I check:

• Can one request be followed across queueing, model work, tool work, and response assembly?
• Can a retry be separated from a new request?
• Are cancelled and failed streams represented as terminal states?
• Are input and output units defined, versioned, and comparable?
• Can performed work be calculated independently from delivered work?
• Are high-cardinality and sensitive fields governed separately from metrics?
• Does the dashboard show the population, window, sampling, and estimation status?
• Can an aggregate be recomputed from retained events?

If any answer is “no,” the dashboard may still be useful as a signal. It should not yet be treated as an explanation.

Conclusion

Telemetry for an AI runtime becomes dependable when it preserves causality: one request, its attempts, the spans that did the work, and the outcome that followed. That structure supports latency analysis, reliability work, quality review, and resource accounting without pretending that one number can answer every question.

The design decision I return to is simple: record enough immutable context to explain a result, keep estimates visibly distinct from confirmed measurements, and aggregate only after the lifecycle is understood. That is how a cost signal remains useful when the runtime grows more concurrent, more streamed, and more failure-aware.

References

• OpenTelemetry, “Traces,” https://opentelemetry.io/docs/concepts/signals/traces/
• OpenTelemetry, “Metrics,” https://opentelemetry.io/docs/concepts/signals/metrics/
• OpenTelemetry, “Semantic conventions,” https://opentelemetry.io/docs/concepts/semantic-conventions/