Jan Iłowski’s article argues that comparing AI models by “price per 1M tokens” is misleading. Different tokenizers split text differently, and token efficiency (how much useful work each token achieves) varies widely across models, meaning the real cost is better measured by cost per completed task rather than raw token price.
🔑 Key Insights from the Article
Tokenizers differ → Each lab (OpenAI, Anthropic, etc.) uses proprietary tokenizers. The same text may be split into 160 tokens by GPT-4o but 200 tokens by GPT-4, making direct price comparisons unreliable.
Even within one lab → Pricing per token can’t be compared across models because tokenization rules change over time.
Token efficiency matters → The hidden “chain of thought” tokens (used internally by models to reason) can dominate costs. Two models with similar per-token prices may differ drastically in overall task cost.
Benchmark evidence →
GPT-5.5 appears more expensive per token than Claude Opus, but completes benchmark tasks at half the cost.
Chinese models like GLM-5.2 or DeepSeek V4 Pro advertise much lower per-token prices, but their efficiency is lower, so the cost per task isn’t proportionally cheaper.
Real ROI metric → The meaningful measure is cost per benchmark task (or per unit of useful work), not “price per 1M tokens.”
💡 Why This Matters
For businesses: Don’t just chase the lowest token price — evaluate models by task-level ROI.
For developers: Benchmarking models against real workloads is essential to avoid hidden costs.
For AI adoption: Token efficiency will become a key differentiator, not just headline pricing.
👉 Read the full article here: https://janilowski.pl/en/blog/2026/price-per-m-tokens/?utm_source=copilot.com
Top comments (0)