$0.14 per million tokens looks cheap. $0.14 per million tokens, multiplied by a feature that goes viral next Tuesday, might not be.
Most "is this AI cheap" evaluations stop at the first number. That's the mistake I want to walk through, because the actual question was never "what does it cost right now" — it's "what shape does the cost take as usage grows," and those are very different questions that happen to share a price tag.
The Number Everyone Compares, and Why It's Incomplete
When people evaluate whether an AI API is "cheap," they almost always mean: what's the per-token or per-request price, compared to alternatives. That number is real and it matters — but it's a snapshot, not a trajectory. It tells you what one unit costs. It tells you nothing about what a thousand units cost, or a million, or what happens at the specific volume your actual product will hit if it works.
Cost only behaves the way that snapshot implies if it scales linearly forever — the 1,000th call costs exactly what the 1st call cost, with nothing changing in between. That's the assumption baked into "cheap AI" as most people evaluate it. It's also, in practice, rarely how these systems actually behave once you're operating at real scale.
Three Shapes Cost Actually Takes, Not Two
Cost as a function of usage tends to fall into a few recognizable shapes, and knowing which one you're dealing with matters more than the sticker price:
Linear. Cost per unit stays flat regardless of volume. The 1,000th call costs what the 1st call cost. This is what a simple per-token price implies, and it's genuinely how some providers price things — but it's worth verifying rather than assuming, because it's the exception more often than the default once you look closely.
Step function. Cost stays flat within a tier, then jumps at a threshold — a rate limit you exceed and get bumped to a more expensive plan, a concurrency cap that forces you onto premium infrastructure, a free-tier boundary you cross without noticing until the bill arrives. This is common with anything that has both a free and paid tier, and with time-based pricing structures where crossing from off-peak into peak hours effectively doubles your rate mid-workload.
Marginal-decreasing. Cost per unit actually drops as volume increases — committed-use discounts, volume pricing tiers, negotiated enterprise rates. Less common at small scale, but real once you're a big enough customer for a provider to want to keep you.
Almost nobody evaluating "is this cheap AI" is thinking in these terms. They're comparing a single number, implicitly assuming shape one, when the actual answer might be shape two — which means the "cheap" evaluation was accurate at low volume and actively misleading at the volume that matters.
Where This Actually Goes Wrong
Here's the concrete failure mode: a team evaluates a model at prototype-scale usage, sees a low per-token cost, concludes "this is cheap," and builds a cost model for the product around that number. Then the product works. Usage climbs. And somewhere along the way, they cross a threshold they didn't know existed — a concurrency limit that forces an upgrade, a rate structure where their peak-hours traffic pattern (because of course their users are active during business hours) now costs meaningfully more than the number they budgeted around.
None of this is deceptive pricing. It's usually clearly documented — but documented in a pricing page that gets read once, at evaluation time, at a volume that doesn't reflect where the product ends up. The gap between "cheap at the volume I tested" and "cheap at the volume I'll actually run" is exactly the gap this framework is trying to make visible before it becomes a surprise on an invoice.
This is also worth connecting to a specific, current example: several major model providers have recently introduced peak/off-peak pricing structures, where identical usage costs meaningfully more during business hours than at night. That's a step function hiding inside what looks, on the pricing page, like a single number — and it's a good illustration of why "what's the price" and "what's the cost curve" are genuinely different questions right now, not just a hypothetical concern.
How to Actually Evaluate "Cheap" Correctly
A few concrete things worth doing differently, given this framework:
Ask about the shape, not just the number, when you evaluate a provider. Is this price linear indefinitely? Is there a rate limit or concurrency threshold where the effective cost jumps? Does pricing vary by time of day or usage tier? These are usually documented, just rarely asked about upfront.
Model your cost at the volume you expect to hit if things go well, not the volume you're at during evaluation. A price that's cheap at prototype scale and a price that's cheap at the scale you're actually building for are different claims, and testing at the wrong scale is how teams get surprised.
Treat unpredictability as its own cost, separate from the price itself. A cost structure that varies by time of day or usage threshold requires monitoring and budgeting work that a flat, predictable rate doesn't — and that overhead is real, even when the headline price is competitive.
Standardized, flat-rate access reduces the step-function risk specifically. This is part of why unified access layers — services that sit in front of multiple model providers with flat, consistent pricing rather than tiered or time-based structures — have become a relevant category to know about, separate from any single provider's rate card. RouteAI is one example of this, offering flat per-token pricing without a peak-hour surcharge structure across several model providers — worth mentioning as a case study in what a linear-shaped alternative looks like, not as a claim that it's the cheapest option available, which would require a comparison this piece isn't making.
The Actual Point
"Cheap AI" isn't a property of a number on a pricing page. It's a property of how that number behaves as your usage grows into whatever your product actually becomes. Two providers can have an identical headline price and radically different actual cost outcomes at scale, depending entirely on the shape underneath that number — and the shape is almost never what gets compared when people search "cheapest AI API" or "cheap AI" and pick based on the first number they see.
The question worth asking isn't "what does this cost me today." It's "what does this cost me at the volume where it actually matters" — and that's a question the pricing page alone usually won't answer for you.
TL;DR: Comparing "cheap AI" options by per-token price alone assumes cost scales linearly, but real pricing often follows a step function (rate limits, peak-hour surcharges, tier jumps) or occasionally marginal-decreasing pricing (volume discounts) instead. Evaluate providers by asking about the shape of cost as usage grows, not just the sticker price, and model your cost at the volume you'll actually hit — not the volume you tested at. Standardized, flat-rate access layers (like RouteAI, among other options) can reduce step-function risk specifically, though that's a different claim from being the cheapest option outright.
Here's the tool I referenced in this post: www.fastrouteai.com
Top comments (0)