Cost per action: the number your LLM spend dashboard cannot produce

TL;DR. Cost dashboards for LLM systems report spend per model, per provider, per thousand calls. Finance asks what the support copilot costs per resolved ticket, and the dashboard cannot answer because no call carries the ID of the action it served. One user action fans out to many calls, the fan-out multiplies when providers degrade, and per-call metrics hide the whole mechanism. The fix is attributing cost to the user action through a propagated trace ID, then alerting on cost per action instead of daily spend.

The question the dashboard cannot answer

The cost dashboards that come with LLM tooling all answer the same three questions. Spend by model. Spend by day. Tokens by endpoint. Useful for exactly one conversation: the one with the provider.

The conversation that actually happens is with a product owner, and it goes: this feature has a budget of X per month, are we inside it, and which change blew it up on Tuesday. Per-call metrics cannot answer that, because no call knows which feature it served or which user action it belonged to.

What one action costs

Take a support-copilot answer as a worked example. The numbers below are a model of the shape, not a bill I am quoting; substitute your own.

One "answer the ticket" action, healthy path:

• 1 routing call on a mini-tier model
• 1 retrieval rerank
• 1 main answer call on a frontier-tier model
• 2 tool round-trips (order lookup, refund policy)
• 1 log-summary call

Six calls for one action. The per-call dashboard renders this as six unrelated rows in three model buckets. The thing with a budget, the action, exists nowhere.

Now degrade the provider. Latency climbs until a quarter of calls blow through your 30-second client timeout, and your retry policy allows two retries. Expected attempts per call become 1 + 0.25 + 0.0625, about 1.31. Six calls become roughly 7.9 attempts on average, and an abandoned attempt is not free: depending on provider and streaming mode, it can still bill some or all of the tokens it generated before you hung up, so a degraded action pays for failures and their replacements both.

The average hides the tail. Even with timeouts modeled as independent coin flips, the p95 action in this setup bills 10 attempts, up two-thirds from the healthy six. Real incidents are worse: timeouts cluster in time, so the actions inside the degradation window take nearly all their calls through the retry ladder at once. The mean moved a little. The tail moved a lot. Every spend spike I have personally chased started life as a provider incident, not a traffic bump, and daily spend alerting noticed hours after the retry ladder had been billing the whole time.

The join key

The fix is unglamorous: one action ID, stamped on every call the action causes, all the way down through retries and fallbacks.

The instrumentation half already has a standard. The OpenTelemetry GenAI semantic conventions define token accounting on spans as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens (the conventions moved into their own semconv repository this year and are still evolving, so pin the version you adopt). Put your action ID on the enclosing trace, and the rollup becomes a GROUP BY:

SELECT
  trace_id,
  feature,
  SUM(input_tokens)  * :in_price  +
  SUM(output_tokens) * :out_price AS action_cost,
  COUNT(*)                        AS calls_in_action
FROM llm_calls
GROUP BY trace_id, feature;

Prices are placeholders; take yours from the provider's sheet. From this table, three numbers per feature: p50 cost per action, p95 cost per action, and calls per action. The last one is your amplification factor, and it is the earliest signal you have. Calls per action rises the moment a dependency starts timing out, minutes before spend-per-hour looks unusual and hours before the invoice does.

Two honest gaps

Unattributed spend never reaches zero. Batch jobs, warmup calls, evals in CI. Give them synthetic action IDs and report the unattributed share explicitly. When that share creeps up, someone shipped a code path outside the instrumentation, and that is worth knowing regardless of cost.

And per-action pricing does not tell you whether the action was worth it. A resolved ticket at 4 cents and an abandoned session at 4 cents cost the same. Joining cost to outcome is the next join, and it needs the same trace ID, which is one more reason to lay it now.

What I'd page on

• p95 cost per action, per feature, against a budget line, because daily spend alerting pages you only after the damage is done.
• The amplification factor (billed attempts per action) drifting above its baseline. That is a retry ladder or a fallback loop announcing itself, minutes before spend charts move.
• Unattributed spend share rising week over week. A new code path escaped the trace.
• Actions that exhausted their retry budget. Each one paid the full fan-out and still failed the user, which makes them the most expensive failures you have.