Per-token price is the wrong metric for LLM API cost

The sticker price on an LLM API page is almost never the number on your invoice.

Most pricing tables show $ / 1M input tokens. That is a useful starting point, but it ignores three things that dominate real workloads:

1. Output tokens cost more. On many popular models the output rate is 4-8x the input rate. A summarization service that reads 10k tokens and writes 200 tokens has a completely different cost profile from a coding agent that reads 2k tokens and emits 1.5k tokens of diff.
2. Cache hit rate changes the bill. Repeated system prompts and few-shot prefixes can be 10x cheaper on providers that expose prompt caching. Two models with identical list prices can diverge by 2-3x depending on how much of your prefix is reusable.
3. Retries and failures are invisible in price tables. A cheap model that needs 1.4 attempts per valid result is more expensive than a pricier one that gets it right once. Cost per valid result is the metric that matters, not cost per token.

A practical way to compare:

• Weight input and output by your own traffic mix (3:1 is a reasonable default for chat, 1:3 for generation-heavy work) to get a blended cost.
• Run the same 50-prompt eval across candidates and count tokens consumed per successful answer, not per request.
• Recompute monthly. Providers change prices and quietly add cheaper variants far more often than teams revisit their model choice.

I maintain a small public comparison site for this: TokenDealHub ranks models by task with input, output, context window and an intelligence index side by side, and the /compare pages put two models head to head. It refreshes hourly. Disclosure: I built it, so treat it as one data source alongside the vendor pricing pages, not as an authority.

What weighting do you use for your own workloads? I have seen 1:1, 3:1 and "just bill it and see" in roughly equal measure, and I suspect the right answer depends more on the product than on the model.