TL;DR — Cloud teams treat multi-year GPU capacity commitments as a straightforward discount on compute, the same way they treat reserved instances for web servers. That mental model breaks for AI workloads because what you're actually locking in is a specific hardware shape matched to a specific model generation. When the model changes, the shape stops fitting, and the 'savings' evaporate. Capacity commitments for AI inference need to be evaluated like financial hedges, with an explicit view on correlation risk between the reserved shape and future workload shape.
Every infrastructure team eventually gets the same pitch: commit to a fixed amount of GPU capacity for one to three years, and the discount against on-demand pricing looks enormous. The math is seductive. It's the same math that made reserved instances a default choice for web servers, databases, and batch jobs for over a decade. So teams running LLM inference apply the same logic, sign the commitment, and treat the discount as free money.
It isn't free. For stateless web workloads, a reserved instance is a volume discount because the thing you're buying — vCPUs and RAM — stays useful regardless of what your application does next. For AI inference, what you're actually reserving is a specific ratio of memory bandwidth to compute to interconnect, tuned to a model that exists today. That ratio has a shelf life measured in months, not years. Treating a GPU capacity commitment as a discount instead of a hedge is the single most underpriced risk in AI infrastructure economics right now.
What You're Actually Buying
A GPU isn't a generic unit of compute the way a vCPU is. It's a bundle of three things that matter independently for LLM serving: raw FLOPs, high-bandwidth memory capacity, and memory bandwidth. The ratio between these determines whether a given model is compute-bound or memory-bound on that hardware, and that ratio is set by the model's architecture — parameter count, attention pattern, context length, quantization, whether it's dense or mixture-of-experts.
When you sign a multi-year capacity commitment, you are not committing to "GPU compute." You are committing to a fixed point in that three-dimensional space. A dense 70B-class model at a modest context length has a very different FLOPs-to-memory-bandwidth requirement than a long-context model with a large KV cache, or a sparse MoE model that needs fast interconnect between experts far more than it needs raw FLOPs per chip. The hardware generation you reserve is implicitly optimized for one region of that space. If your workload lives somewhere else, the discount you locked in is a discount on the wrong thing.
Model Churn Breaks the Hedge
This would be a minor footnote if models changed shape as slowly as server workloads change. They don't. A model swap — upgrading to a new base model, moving to a longer context window, switching from dense to MoE, adopting a new quantization scheme — can change the memory-to-compute ratio of your serving workload by a wide margin in a single release cycle. That's not a hypothetical; it's the normal cadence of how teams actually ship.
Every time the shape changes, the fixed hardware you committed to either has spare compute sitting idle while memory bandwidth saturates, or spare memory capacity sitting idle while compute saturates. Either way, you paid for a bundle and are now using one dimension of it while wasting the other two. The committed discount rate doesn't adjust for this. You're still paying the reserved rate for capacity that's now mismatched to the job it's doing, and the effective cost per useful token goes up even though the sticker price per GPU-hour went down.
The failure mode teams actually hit is subtle: they notice the reserved fleet is "underutilized" by some naive utilization metric, so they stuff more traffic onto it to justify the commitment — even traffic that would run more cheaply and more efficiently on a different accelerator generation or a different provider's spot capacity. The commitment starts dictating architecture decisions instead of the other way around. That's the hedge going wrong: it was supposed to reduce risk, and instead it's forcing the organization to keep making bad decisions to avoid looking like it wasted money.
Pricing Capacity Like an Option, Not a Discount
The correct mental model here comes from derivatives, not procurement. A multi-year GPU commitment is structurally closer to a forward contract with an embedded correlation bet: it pays off only if your future workload shape stays correlated with the shape you reserved for. The discount is the premium you're being paid for taking on that correlation risk. If you believe your model architecture, context length, and routing strategy will stay stable for the life of the commitment, the discount is real money. If you believe any of those will shift — and in a field where the dominant architecture pattern has changed multiple times, that's the safer assumption — the discount is compensation for a risk you're underpricing.
Once you frame it that way, the right question before signing isn't "what's the discount rate?" It's "what's the probability-weighted cost of this commitment across the range of model shapes we're likely to run in year two and year three?" That's a scenario analysis, not a spreadsheet of hourly rates. It means modeling: what does utilization and effective cost-per-token look like if we stay on roughly today's architecture, if we move to a much longer context window, if we adopt a sparse MoE model, if quantization drops our memory footprint by half? A commitment that looks cheap in the base case and catastrophic in two of the other three cases isn't actually cheap — it's underpriced risk with a flattering headline number.
What This Means for Capacity Planning
None of this is an argument against reserved or committed GPU capacity. It's an argument for sizing and structuring commitments the way you'd structure any hedge: smaller, shorter, and diversified against the specific thing that's uncertain. A few practical shifts follow directly from the hedge framing.
Favor shorter commitment terms even at a worse headline discount, because the option value of being able to re-shape your fleet when the next model generation lands is usually worth more than the marginal percentage saved on a longer lock-in. Spread commitments across more than one hardware generation or accelerator family rather than concentrating in the single shape that fits today's model best — a diversified hedge against architecture churn, not a bet on one future. Separate capacity planning for prefill-heavy and decode-heavy workloads explicitly, since they stress compute and memory bandwidth differently, and a single blended commitment tends to be optimized for neither. And build the actual scenario model before signing: lay out two or three plausible future model shapes, estimate utilization and effective cost-per-token under the existing commitment for each, and only sign if the worst case is tolerable, not just the base case attractive.
The teams that get burned here aren't making an arithmetic error. The discount percentage on the contract is real. The error is categorical: pricing a correlated, architecture-dependent bet as if it were an uncorrelated volume discount. Fix the category, and the rest of the capacity planning conversation gets a lot more honest.
Top comments (2)
Calling reservations a hedge rather than a discount is exactly right. The decision should combine utilization uncertainty, interruption cost, and the ability to move workloads—not just compare hourly prices. A monthly forecast-versus-actual review is what keeps the hedge from becoming idle spend.
I cannot agree more !!