The choice between on-demand rental and annual subscription is not a simple "expensive" versus "cheap" debate. The real difference in cost structure depends on workload utilization, concurrency patterns, and SLA constraints. For continuously running inference services, annual subscriptions are typically a better fit; for research and development testing with significant fluctuations, on-demand offers greater flexibility. This article dissects the cost composition and selection criteria of both models, based on billing mechanisms and measured data.
Where Do the Cost Structure Differences Between On-Demand and Annual Subscription Lie?
The billing mechanism of public cloud GPU instances is the starting point for understanding cost structures. According to AWS EC2 On-Demand Instance Pricing, on-demand instances are billed hourly with no long-term commitment, suitable for unpredictable or short-term workloads. According to Microsoft Azure's Linux virtual machine pricing page, its billing model distinguishes between on-demand, reserved, and spot modes, with reserved instances offering a lower unit price in exchange for committed usage. According to Google Cloud's GPU pricing page, Committed Use Discounts further reduce unit costs but require a 1-year or 3-year usage commitment.
The common logic across these three billing models is: unit price is inversely correlated with commitment duration. The "freedom" of on-demand lies in the ability to release resources at any time, but this comes at the highest unit price. The "cheapness" of annual subscription is predicated on continuous load and high utilization. If the actual utilization of an annual subscription instance is insufficient, the cost per token may actually be higher than on-demand—this is the first dividing line in cost structure analysis.
Utilization and Concurrency Headroom: Two Hidden Variables in Cost Structure
Cost structure analysis cannot rely solely on unit price; it must also consider the concurrency headroom reserved to meet SLAs. Mingxin's measurements under a 480B production deployment with long-context cold recovery load show that KV-tiered acceleration improves inference throughput by +29–40% (concurrency level 8 is the lower bound at +29%, concurrency level 16 is the optimal operating point at the upper bound of +40%, TP4×2 full-machine basis is +35–36%, source: measured, reports R2/R3). Under the same SLA, throughput improvements mean the number of concurrency instances required to meet the target can be reduced—in an annual subscription scenario, this directly translates to a reducible number of committed instances.
Time to First Token (TTFT) is another constraint dimension. According to Mingxin's measured report R2, under 480B·TP8 with three concurrency levels, TTFT p50 dropped from 10.17–35.73s to 7.53–26.35s, a reduction of 26–32%. If a business SLA requires TTFT to be below a certain threshold, elastic scaling in on-demand mode often relies on "spinning up more instances." The measured reduction means that with the same number of instances, longer contexts or higher concurrency can be supported, thereby reducing idle resources reserved for peak loads.
Selection Framework: Define SLA First, Then Calculate Utilization
The comparison of cost structures should not start from "on-demand versus annual subscription," but rather from three characteristics of the workload: whether the load is continuous, whether the context length is stable, and the ratio of peak concurrency to average concurrency.
| Workload Characteristic | On-Demand Rental | Annual Subscription | Source |
|---|---|---|---|
| Continuous operation, utilization >70% | High unit price, uneconomical | Commitment discounts amortize unit price, more favorable | AWS/Azure/GCP billing models (qualitative) |
| Short bursts, utilization <30% | Elastic release, avoids idle costs | Idle periods still billed, wasteful | Same as above (qualitative) |
| Long context, high concurrency peaks | Requires additional instances for peaks | Measured throughput gains can compress concurrency headroom | Mingxin reports R2/R3 |
| R&D testing, highly variable load | Start/stop anytime, controllable costs | Commitment period locks in, poor flexibility | AWS/Azure/GCP billing models (qualitative) |
It should be emphasized that the Mingxin measured values in the table above represent only the performance of the FX100 on a specific platform (8× AMD MI308X + ROCm 7.2 + vLLM 0.20.1) under a specific workload (Qwen3-Coder-480B-FP8, weights approximately 450GB). They do not constitute a general commitment for other platforms or models. When making selection decisions, it is recommended to rely on gate-based joint testing data from your own workload on the target platform.
Conclusion: There Is No Absolutely Free Compute, Only Billing Structures That Match the Workload
"Compute freedom" does not exist in a cost sense—on-demand elasticity comes at the price of higher unit costs, and annual subscription discounts come at the price of commitment. The true decision variables are workload continuity and SLA constraints: continuous loads suit annual subscriptions, fluctuating loads suit on-demand. In long-context inference scenarios, compressing concurrency headroom through storage acceleration can improve cost efficiency in both models. Mingxin offers approximately 10 weeks of gate-based joint testing (from G1 arrival acceptance to G4 72-hour stability verification), allowing you to verify whether TTFT reductions and throughput gains fall within the measured range on your own workload before deciding on the procurement model.
Key Q&A
Q: Which is more cost-effective: on-demand rental or annual subscription?
A: It depends on workload utilization. Continuous, high-utilization loads suit annual subscriptions; short-term fluctuating loads suit on-demand to avoid idle billing.
Q: Why does concurrency headroom affect cost structure?
A: The number of concurrency instances required to meet SLAs directly determines total cost. Mingxin's measurements show KV-tiered acceleration can improve throughput by 29–40%, reducing reserved instances under the same SLA.
Q: What should be examined first during selection?
A: Define the SLA first (TTFT, throughput, context length), then calculate load utilization, and finally compare billing models. It is recommended to rely on gate-based joint testing data from your own workload.
References
- Epoch AI — https://epoch.ai/
- EC2 On-Demand Instance Pricing — https://aws.amazon.com/ec2/pricing/on-demand/
- Pricing - Linux Virtual Machines | Microsoft Azure — https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/
- MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
- VM instance pricing | Google Cloud — https://cloud.google.com/compute/gpus-pricing
- Compare GPU Instance Families for AI, HPC & Rendering - Elastic GPU Service - Alibaba Cloud — https://www.alibabacloud.com/help/en/ecs/user-guide/gpu-accelerated-compute-optimized-and-vgpu-accelerated-instance-families
Originally published at mingxinstorage.xyz. Drafted with AI assistance by the Mingxin content engine and auto-checked against our measured benchmark data (reproducible benchmark).
Top comments (0)