Why the Same Prompt Costs a Different Amount on Two Providers

#ai #machinelearning #llm #programming

You sent byte-identical text to two providers and the two line items do not have the ratio their published prices led you to expect. The published price is one of two multipliers. The other is that the two providers did not agree on how many tokens your text is.

No per-token price appears on this page as a fact. Prices in this market are revised without notice and a quoted table would be wrong before this page is a quarter old. Everything below is written with prices as named inputs you substitute from each provider’s current pricing page on the day you do the calculation.

Two multipliers, not one

For a single request with a fixed prompt and a fixed reply, cost on provider A is:

cost_A = N_in_A * P_in_A + N_out_A * P_out_A

where N is a token count and P is a price per token. Provider B has the same form with its own four values. The mistake is assuming N_in_A = N_in_B. Nothing enforces that. The two counts come from two different tokenizers, each of which is a learned byte-pair-encoding vocabulary specific to that model family, and the same string can segment into materially different numbers of pieces under each.

So write the ratio of the two bills as a product of two independent ratios. Taking input tokens alone for clarity:

cost_B / cost_A = (N_in_B / N_in_A) * (P_in_B / P_in_A)
                =        T        *        R

T is the tokenizer ratio and R is the price ratio. Only R is on a pricing page. T you have to measure, it depends on your text, and for anything other than plain English it is not close to 1.

Getting the tokenizer ratio

The reliable way to get T for your own text is to ask each provider, because the number the provider returns is the number they bill. Three mechanisms exist and they are not equivalent:

Count locally with the published encoder. OpenAI publishes tiktoken, which implements the encodings its models use — cl100k_base for the GPT-4 and GPT-3.5-turbo generation and o200k_base for the GPT-4o generation, the names giving the approximate vocabulary sizes of roughly 100,000 and 200,000 entries. A larger vocabulary can represent more sequences as single tokens, so the same text generally segments into fewer o200k_base tokens than cl100k_base tokens, with the gap widest on non-English text and on code.
Ask the API to count. Anthropic exposes a token counting endpoint on the Messages API that returns an input_tokens figure for a request body without running inference; Google’s Gemini API exposes a countTokens method returning totalTokens. Both are the authoritative count for those providers, and neither has a published offline tokenizer you should rely on instead.
Read it back off a real call. Send the request and take the usage object. This is the only method that captures the per-message structural overhead — the tokens a provider adds for role markers and turn boundaries, which no local encoder of the raw string includes. OpenAI’s cookbook documents that overhead for its chat models as a fixed number of tokens per message plus a fixed number priming the reply.

Use the third for anything you are going to build a budget on. The first two are for iterating quickly.

The derivation

Work it through with symbols and one set of clearly-labelled illustrative inputs. The four numbers below are assumptions for the sake of the arithmetic, not measurements. Substitute your own before you conclude anything.

Assumed for illustration only:
  N_in_A  = 1,000 tokens      N_in_B  = 1,150 tokens   -> T_in  = 1.15
  N_out_A =   300 tokens      N_out_B =   345 tokens   -> T_out = 1.15
  P_in_B  / P_in_A  = 0.80    (B's input price is 20% below A's)
  P_out_B / P_out_A = 0.80

Input line:   0.80 * 1.15 = 0.92   -> B is 8% cheaper on input
Output line:  0.80 * 1.15 = 0.92   -> B is 8% cheaper on output

The headline "20% cheaper" delivered 8%.

The structure is what matters, not the digits. A tokenizer that is 15% less efficient on your text consumes three quarters of a 20% price cut. If the tokenizer ratio were 1.25 instead, the same 20% price cut would be a cost increase of 0%. Above 1.25 it is a loss. That crossover is worth naming.

The break-even price ratio

Set the ratio of the two bills equal to 1 and solve. For input alone:

T * R = 1
R = 1 / T

If T = 1.15, B must be priced at 1/1.15 = 0.87 of A -- a 13% cut -- to break even.
If T = 1.30, B must be priced at 1/1.30 = 0.77 of A -- a 23% cut -- to break even.
If T = 0.90, B breaks even at 1.11 of A: it can be 11% dearer and still cost less.

That last line is the case people miss. A provider with a nominally higher price can be the cheaper one on your traffic if its tokenizer is more efficient on your text, and this is common when the text is not English — the vocabulary a tokenizer was trained on decides how many pieces a given script breaks into. The general treatment of that effect is in the tokenizer language tax.

Compute T separately for input and for output. They are not the same number: your input is your text, in your domain and your language, while the output is the model’s text, and two models write differently. If you only have budget to measure one, measure output, because output is normally priced several times higher than input.

The term that ruins the tidy version

The two-multiplier decomposition is exactly right for one fixed prompt and one fixed reply. Real requests introduce a third term, and on some workloads it dominates both of the others.

The reply is not fixed. You control the prompt; you do not control how long the model chooses to answer. If model B writes longer for the same instruction, N_out_B reflects both the tokenizer and a genuinely longer answer, and the decomposition quietly conflates them. To separate the two, compare token counts of the two replies under a single tokenizer: any remaining difference is verbosity, and what is left over is tokenization. That distinction matters because they have different fixes — verbosity responds to an instruction and to an output cap, tokenization does not respond to anything.

Reasoning tokens are billed and invisible. On models that produce internal reasoning, those tokens are charged as output but are not in the text you received. A cost figure derived by counting the characters of the reply will understate the bill, sometimes severely. Take the usage object, not the string length.

Cached input is a different rate. If part of your prompt is served from a prompt cache, that portion is billed at a reduced rate and is reported in its own field rather than folded into the ordinary input count. Two providers with different caching rules produce different effective input prices for the same prompt, and this term can be larger than the tokenizer ratio on a workload with a long shared system prefix.

For one request, the two-term version is enough to explain the surprise. For a month of requests, the aggregation introduces its own error, which is worked through in what a tokenizer difference does to your monthly bill.

DEV Community