DEV Community

Cover image for How to Compare Model Prices When Each Model Counts Tokens Differently
Mayur Agnani
Mayur Agnani

Posted on Edited on Originally published at theaicommit.com

How to Compare Model Prices When Each Model Counts Tokens Differently

Two models can list the same price per million tokens and send different bills for the same prompt. Each one counts tokens with its own ruler.

Engineer's view. This is comparing cloud machines by price per vCPU when one provider's vCPU is a full core and the other's is half of one. The unit has the same name and a different size. A token is that kind of unit: every model defines its own.

TL;DR. Each model splits text into tokens with its own vocabulary, so the same prompt is a different number of tokens on each. A price per token only compares after you measure tokens per character on your own text. The same effect quietly shrinks context windows and output limits.

Explain Like I'm 5

Two shops sell rope by the "length." At the first shop, a length is your arm. At the second, a length is the shopkeeper's arm, which is shorter. Both signs say ten coins per length. You need enough rope to cross a river. At the second shop you need more lengths, so you pay more. The price signs matched, and the bills did not. To compare, measure the river once in your own steps. Then ask each shop how many of its lengths that is.

The Problem

You have shipped this bug before, in a system with no AI in it.

Your team picked a cloud machine by price per vCPU. Provider A was cheaper per vCPU, so the batch jobs moved there. The jobs ran slower, and the monthly bill went up. On provider B, a vCPU was a full core. On provider A, it was one hyperthread, about half a core. Both units were called vCPU. They were not the same size, so the comparison was wrong before anyone ran it.

Model pricing has the same bug. Every provider lists a price per million tokens, the chunks of text a model reads and writes. Comparison sites line those prices up in one table. Your usage dashboard and your plan's limits are in tokens too. It looks like one shared unit.

It is not. Each model cuts text into tokens its own way. In April 2026, Anthropic shipped Opus 4.7 at the same price as Opus 4.6, with a new tokenizer. OpenRouter then studied over a million real requests from users who switched. Their prompts now came out 32% to 45% longer in tokens. Simon Willison pasted one system prompt into both models. 5,039 tokens became 7,335. The price tag did not move, and the bill did.

The fix: convert every price to your own text before you compare, then recheck every limit you set in tokens.

Read the full post on The AI Commit

How to Compare Model Prices When Each Model Counts Tokens Differently carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.


Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

Top comments (0)