LLM Observability Cost Starts With a Trace Budget

LLM observability cost is not a single price per trace. It is the sum of several workload multipliers: requests become traces, traces contain spans, selected spans become evaluation jobs, judges consume tokens, payloads occupy storage, and retention keeps those bytes billable over time.

The practical fix is to build a trace budget before comparing vendor plans. Start with six variables you can measure in your own application: monthly traces, spans per trace, bytes per span, evaluation sampling rate, judge tokens per score, and retention days. A vendor tier is affordable only after those units have been converted into your workload.

TL;DR: Forecast events, bytes, scores, judge tokens, and retained data separately. Then apply each vendor's meter to the same trace budget. Never compare tier names as if they represented the same physical quantity.

This article is for engineers and technical leads operating production LLM features. It provides a worked example, a copyable budget receipt, and guardrails that prevent silent cost growth.

Why one trace is not one billing unit

A trace represents one end-to-end interaction, but it can contain a changing number of observations. A simple retrieval request might create spans for orchestration, embedding, vector search, reranking, generation, and a tool call. Add retries or parallel tools and the same user request can create more events without increasing request count.

That is the first trap in AI observability pricing: two systems with 100,000 monthly traces can produce very different ingestion volumes. One may average three spans per trace; the other may average 18. If a platform meters observations, processed bytes, or both, “trace count” is only the first input.

The budget therefore belongs inside a production AI engineering workflow, where feature ownership, instrumentation, evaluation, and spend share the same boundary. Otherwise the application team controls trace shape while a separate platform team receives the bill after the multiplier has already changed.

The four cost layers inside LLM observability

Treat the bill as four independent layers. A platform may bundle some of them, but your model should keep them separate.

1. Ingestion

Ingestion is driven by events or bytes. Its useful variables are monthly traces, observations per trace, and serialized bytes per observation. Recording complete prompts, tool arguments, retrieved documents, and outputs can make byte volume grow much faster than the event count.

2. Retention

Retention is a volume-time problem. Storing 1 GB for 90 days consumes roughly three times the steady-state capacity of storing the same monthly inflow for 30 days. Indexes, replicas, compression, and vendor-specific accounting can change the invoice, but they do not remove the underlying time multiplier.

3. Evaluation

Evaluation adds scores. A trace can receive zero, one, or several scores from deterministic checks, human review, or LLM-as-a-judge workflows. The Braintrust pricing model, for example, exposes processed data, scores, and retention as distinct meters. That separation is a useful reminder that capturing a trace and judging it are different operations.

4. Judge-model inference

An LLM judge creates another model call. Its input may include the original prompt, response, retrieved context, rubric, and reference answer. The observability platform's score meter and the model provider's token charge are therefore two different cost lines for the same evaluation job.

Build the LLM observability cost equation

Use a vendor-neutral equation first:

monthly_observability_cost =
    ingestion_cost(events, bytes)
  + retention_cost(bytes, days)
  + evaluation_platform_cost(scores)
  + judge_model_cost(input_tokens, output_tokens)

The variables become easier to audit when they are arranged as an LLM observability cost model by trace volume, because every conversion assumption sits beside the meter it converts. For each feature, record these six inputs:

Variable	How to measure it
Monthly traces	Count root interactions by feature and environment
Spans per trace	Use p50 and p95, not only an average
Bytes per span	Measure serialized payload after redaction
Evaluation rate	Divide scored production traces by eligible traces
Judge tokens	Capture input and output tokens per evaluation
Retention days	Record the policy for raw, redacted, and aggregated data

Do not replace unknown values with zero. Mark them unknown, add an instrumentation task, and calculate a range. An explicit range is more useful than a precise total built on missing multipliers.

A worked budget for 100,000 monthly traces

Consider an application with these stated planning assumptions:

• 100,000 traces per month
• 6 spans per trace on average
• 2 KB per span after redaction and serialization
• 10% evaluation sampling, with one score per selected trace
• 1,200 judge input tokens and 120 judge output tokens per score
• 30 days of raw-trace retention

The conversions are straightforward:

observations = 100,000 × 6 = 600,000
raw_ingestion = 600,000 × 2 KB = 1.2 GB/month
scores = 100,000 × 10% = 10,000/month
judge_input = 10,000 × 1,200 = 12,000,000 tokens/month
judge_output = 10,000 × 120 = 1,200,000 tokens/month

At a steady ingestion rate, increasing raw retention from 30 to 90 days raises the retained raw-data estimate from about 1.2 GB to 3.6 GB before vendor-specific compression, indexing, and replication. Increasing evaluation coverage from 10% to 25% raises the monthly score count from 10,000 to 25,000 and multiplies both judge-token lines by 2.5.

These are not market prices. They are workload units. Apply current vendor rates only after this conversion, and keep the rates in a separate configuration so a pricing change does not rewrite the workload model.

Instrument the inputs, not just the total

A dashboard that shows only total dollars cannot explain a change. Record the dimensions that caused it: feature, environment, model, trace name, observation type, score name, and retention class.

Langfuse's trace-structure guidance describes observations grouped into traces and recommends carrying model, usage, and cost details on generation observations. Stable names matter because a renamed generation can break a cost series even when the application still works.

Token coverage also needs its own reliability metric. The OpenTelemetry GenAI attribute registry says instrumentation should make a best effort to populate input-token usage. In practice, that means tracking the percentage of generation spans with usable token counts. A total derived from 62% coverage should not be presented as the full bill.

Four guardrails that keep the trace budget stable

1. Set an event budget per feature. Alert when p95 spans per trace crosses the expected boundary. This catches retry loops and accidental nested instrumentation before the monthly invoice does.
2. Sample evaluations by decision value. Score all release candidates and high-risk paths, then sample routine production traffic. A flat 100% judge rate is rarely the only useful policy.
3. Tier retention. Keep raw payloads only as long as debugging or audit needs require; retain redacted summaries and aggregate metrics longer when they carry the necessary signal.
4. Fail visibly on missing usage. Report token-coverage and byte-coverage percentages beside every cost estimate. Missing telemetry is uncertainty, not free usage.

Review those guardrails whenever a prompt, tool graph, model, evaluator, or retention policy changes. Each can change the bill without changing request volume.

A copyable trace-budget receipt

Keep the assumptions next to the result:

period: 2026-08
feature: support-agent
monthly_traces: 100000
spans_per_trace:
  p50: 5
  p95: 9
bytes_per_span: 2048
evaluation_sample_rate: 0.10
scores_per_sampled_trace: 1
judge_tokens_per_score:
  input: 1200
  output: 120
retention_days:
  raw: 30
  aggregate: 365
coverage:
  token_usage: 0.98
  serialized_bytes: 1.00

Version this receipt with the instrumentation configuration. When the estimate changes, the diff should reveal whether traffic, trace shape, evaluation policy, token usage, or retention caused it.

Frequently asked questions

What is the biggest driver of LLM observability cost?

There is no universal biggest driver. Event-heavy agent workflows amplify span counts, verbose payloads amplify processed bytes, broad judge coverage amplifies scores and model tokens, and long retention amplifies storage. Measure all four layers for your workload before optimizing one of them.

Should every production trace be evaluated?

Usually not by an expensive LLM judge. Evaluate high-risk paths and release candidates deliberately, then sample routine traffic at a rate that can detect meaningful regressions. Deterministic checks and user feedback can cover additional traces without paying judge-model inference for every request.

How often should a trace budget be recalculated?

Recalculate monthly and whenever a model, prompt, tool graph, evaluator, sampling rule, payload policy, or retention period changes. Those changes can alter observations, bytes, scores, or tokens even when user traffic remains flat, so request-count monitoring alone will miss the shift.

Start with units, then compare plans

An LLM observability cost estimate becomes defensible when every price is attached to a measured unit and every unknown remains visible. Count traces, expand them into observations and bytes, apply evaluation coverage, calculate judge tokens, and extend storage across the real retention window.

Then compare vendors against the same receipt. The cheapest tier name is not the answer. The answer is the plan that fits your measured trace budget without hiding the multiplier most likely to grow next.