You have measured that the new provider’s tokenizer produces some percentage more tokens for your text. Turning that into a monthly figure is one multiplication, and the multiplication is usually wrong, because a single average ratio applied to a heterogeneous corpus does not commute with the way the bill is actually assembled.
Every price on this page is a placeholder you replace with the figure on the provider’s current pricing page. No per-token price is asserted here, because any figure quoted would be stale within a quarter and a stale price in a cost model is worse than no price at all.
The naive calculation and its error
The obvious approach: take last month’s total input and output tokens from the incumbent’s usage export, multiply by a tokenizer ratio measured on a sample, multiply by the new prices.
bill_new = TOK_IN_OLD * T_in * P_IN_NEW
+ TOK_OUT_OLD * T_out * P_OUT_NEW
This is right only if T_in is constant across your traffic. It almost never is, for a specific and predictable reason: the tokenizer ratio is a property of the text, and your traffic is a mixture of text types with very different ratios. Code, English prose, non-Latin-script prose, structured JSON payloads and retrieved document chunks each have their own ratio, and they can differ from each other by more than any of them differs from 1.
Applying a mean ratio to a total is only correct when the ratio and the volume are uncorrelated. They usually are correlated, and in the unhelpful direction: the traffic class with the worst tokenizer ratio is frequently the high-volume one, because bulk work — document ingestion, log analysis, code review — is exactly the work whose text is unlike ordinary English. Weighting the ratio by request count rather than by token volume compounds the error.
Segmenting by traffic class
The correction is to compute a ratio per class and weight by that class’s share of tokens, not of requests. Pick segments you can actually separate in your logs, which in practice means by endpoint, by prompt template id, or by tenant locale:
- By prompt template. The best segmentation if you have a prompt registry, because each template has a stable text character and a countable volume.
- By language or script. Essential if you serve more than one. The ratio between two tokenizers on the same non-Latin text can be far from the ratio on English, in either direction, depending on which vocabulary covers that script better.
- By content type. Retrieved chunks versus user-typed text versus machine-generated payloads. A RAG workload’s input is dominated by retrieved text, so its ratio is the retrieved corpus’s ratio, not the user query’s.
The derivation
Three segments, with every input labelled. The volumes and ratios below are illustrative assumptions chosen to show the structure; substitute measurements from your own logs.
Assumed monthly input tokens on the incumbent, by segment:
S1 English support replies 600M tokens measured T_in = 1.02
S2 Source-code review 300M tokens measured T_in = 1.22
S3 Japanese-language traffic 100M tokens measured T_in = 0.88
--------
total 1,000M tokens
Naive: a single mean ratio of (1.02 + 1.22 + 0.88)/3 = 1.04
1,000M * 1.04 = 1,040M tokens on the new provider
Segmented (weighting each ratio by that segment's token volume):
600M * 1.02 = 612M
300M * 1.22 = 366M
100M * 0.88 = 88M
-------
1,066M tokens
Effective ratio 1.066, not 1.04. The naive figure understates by 26M tokens.
Convert to money with prices as inputs. Keeping them symbolic is the point: the arithmetic below is the deliverable and it stays correct when the prices change.
P_IN_NEW = ... # per 1M input tokens, from the pricing page, dated
P_OUT_NEW = ... # per 1M output tokens
input_cost_new = 1_066 * P_IN_NEW # 1,066M tokens = 1,066 units of 1M
output_cost_new = TOK_OUT_NEW_M * P_OUT_NEW
delta = (input_cost_new + output_cost_new) - last_month_actual_invoice
Note what is on the right of the last line: the actual invoice, not your model of last month. Comparing a model of the new provider against a model of the old one compounds two errors and hides both. Anchor on the number you were charged.
Which input to go and measure again
A cost model with six measured inputs has six ways to be wrong, and they are not equally important. Rather than refining everything, vary one input at a time by a fixed proportion and see which moves the answer:
- Take the assembled monthly figure as your baseline.
- Multiply each segment’s measured ratio by 1.2, one segment at a time, and record the change in the total. The segment producing the largest change is the one whose ratio needs measuring on a bigger sample.
- Do the same for the output-token volume. Output is normally priced several times above input, so a proportional error in output volume costs several times what the same proportional error in input costs — which is why measuring output length carefully is a better use of a day than refining an input tokenizer ratio to two decimal places.
- Vary the prompt-cache hit rate between zero and your observed rate. If that swing is larger than every tokenizer effect combined, then your cost model is really a caching model and the tokenizer question is a distraction. On workloads with a long shared system prefix this is common.
The ordering that falls out of this is stable across most workloads: output volume matters most, cache behaviour second, input tokenizer ratio third. That is the opposite of the order in which people investigate them, because the tokenizer difference is the one that is visible in a single response and the other two are only visible in aggregate.
Reporting it honestly
The figure will be quoted back to you, probably in a decision meeting, so the way it is written down matters:
- Give a range, not a point. Run the assembly at your p50 and p95 output lengths and quote both. A single number implies a precision the inputs do not have.
- Date the prices in the document itself. “At prices published on 4 August 2026” costs nine words and saves the model being re-quoted a year later as though it were current.
- Separate the one-off from the recurring. The tokenizer effect is recurring and permanent. Migration engineering, dual-running overlap and a retrained fine-tune are one-off. A payback period needs both and they are frequently reported as one blended number, which makes the recurring saving look worse and the decision look closer than it is.
- State the direction the estimate is likely to be wrong. If your sample excluded the evaluation suite and the nightly batch job — and most samples do — then the estimate is low, and saying so is more useful than a confidence interval that pretends the error is symmetric.
Finally, put a review date on it. A tokenizer ratio is stable as long as both models are, but a model version bump can change the encoding under you, and prices move. Re-run the segmented calculation when either changes, and reconcile it against a real invoice at least once, as set out in re-baselining cost estimates after a switch.
Top comments (0)