Ask five providers what an H100 costs per hour and you'll get five numbers that aren't within 40% of each other. Ask the same provider twice — once for an on-demand instance, once for a committed reservation — and you'll get two more. This is not a market discovering a price. It's a market where the published price is a marketing artifact and the real price is a negotiation.
I spent the last few months tracking GPU rental markets for a compute research forum, and the pattern is consistent enough to state plainly: the headline rate for a GPU tells you almost nothing about what a given workload will cost. Here's where the number you see diverges from the number you pay.
1. The rate is quoted per GPU-hour. Nobody buys GPU-hours.
Real workloads are gated by memory, interconnect and storage throughput, not by raw compute. If your job is bandwidth-bound, buying the fastest card on the price list is buying the most expensive idle silicon in the rack. The honest unit is cost per completed job — and almost nobody quotes it, because it makes comparisons unflattering.
2. Reserved capacity is priced like a futures contract, and behaves like one.
Reserved tiers typically price 30–50% below on-demand. What that discount actually buys is a commitment to pay whether or not you use the capacity. Teams over-reserve because the discount looks like savings, then pay for idle reservations and quietly buy spot capacity on top when their demand spikes. The blended rate ends up above on-demand, and the dashboard still shows the discounted rate.
3. Egress and storage are where the arithmetic breaks.
Compute is metered honestly. Data movement usually isn't. Pull a multi-terabyte dataset in, checkpoint it repeatedly, and ship the outputs out, and you can add 20–60% to the compute line without ever changing instance type. The providers selling "the cheapest H100" are frequently the ones charging the most for the pipes.
4. Idle time and failures are billed to you.
A 30-day training run is not 720 hours of utilization. It's restarts, checkpoint rollbacks, node failures and queue waits — all of it metered at the same rate as productive work. Industry anecdotes put effective utilization for large runs between 60% and 85%. Your quoted hourly rate should be divided by that number before you compare it to anything.
5. The spread between reserved and spot is real money almost nobody arbitrages.
On the same hardware, the spot market can clear 30–50% below reserved pricing for genuinely interruptible work — batch rendering, data preprocessing, evaluation sweeps, hyperparameter searches. We logged the spread on published rates over several weeks, and the gap is stable enough to plan around. The reason teams don't take it is organizational, not technical: interruptible workloads live in a different team's backlog than the capacity budget.
6. And then there's the price nobody publishes.
Enterprise agreements are negotiated, not listed. Two customers, same instance type, same region, same month, can differ by 3x depending on volume commitment, term and how badly the provider wants the logo. The public price list is the ceiling, not the market.
What a decentralized alternative actually changes
This is the part where I should be transparent: I work on Omniverse Compute (OMC), a decentralized GPU network, so I have a dog in this fight. Read the section above as the argument for why that fight exists — and read this section knowing my bias.
Aggregating idle GPUs into a permissionless grid doesn't magically make compute free. What it changes is the shape of the pricing. Supply is composed of hardware that's already paid for — consumer cards, data-center spare capacity, post-PoW mining rigs — so the marginal cost of onboarding is close to zero, and the network can clear at prices set by hardware owners rather than by a sales organization.
It also changes the billing mechanics, and this matters more than the headline discount:
- Jobs are matched to nodes by a scoring engine that weighs VRAM, bandwidth, latency and reputation — not by whoever has the best sales team.
- Requesters lock fees in escrow before dispatch, so the price is the price.
- Each job runs under a verification policy — spot-check re-execution or full redundancy across 2–3 nodes — which is what makes trusting a stranger's GPU possible in the first place.
The honest caveat: a decentralized network is not going to beat a hyperscaler on guaranteed uptime, compliance paperwork or a decade-old support org. If your workload must not fail, buy the boring thing. If your workload is interruptible, cost-sensitive and technically self-sufficient, the spread on offer is large enough to be worth the operational friction.
How to actually evaluate a GPU provider
- Ask for cost per completed job on your workload, not per GPU-hour. If they can't produce it, they don't know it.
- Add egress and storage to the model before comparing anything.
- Divide the quoted rate by realistic utilization (0.6–0.85), not by 1.0.
- Test spot/interruptible pricing with a workload that tolerates restarts.
- For multi-node jobs, price the interconnect — NVLink-class bandwidth changes the math on sharded training more than clock speed does.
The market for GPU hours is priced like a market for a commodity and behaves like a market for enterprise software. Until buyers start asking for the invoice-shaped number instead of the GPU-shaped one, the six different prices will keep being six different prices.
Originally published at the Decentralized Compute Forum (https://forum.omc.network) — an editorial site on GPU markets and the economics of AI compute, operated by the OMC team. Coverage of OMC and its competitors plays by the same rules: dated numbers, linked sources, public corrections. Nothing here is financial advice.
Source post and dataset: "Why GPU Cloud Pricing Is a Scam" → https://forum.omc.network/posts/gpu-cloud-pricing-is-a-scam
The reserved-vs-spot spread I referenced in section 5 was logged here → https://forum.omc.network/posts/reserved-vs-spot-gpu-spread
Top comments (0)