Runcrate

Posted on Oct 21

How to compare GPU cloud prices without fooling yourself

#gpu #machinelearning #python #cloud

"H100 from $X an hour" is the least useful number in a GPU price list. Two offers with the same headline can differ by a factor of two in what you pay at the end of the month, and two offers with very different headlines can come out the same.

We publish GPU prices at Runcrate, and other people's comparison sites read them, so I spend a lot of time looking at how a price turns into a bill. This is the checklist I use. It works for any provider, ours included. I'll use our public pricing endpoint for the examples because it's the data I can show you with a script, not because the method depends on it.

1. Get to one unit: dollars per GPU-hour

Some lists quote the machine, some quote the GPU. An 8x H100 node at a per-machine price looks eight times worse than a 1x H100 at a per-GPU price. Divide by the GPU count first, always.

Runcrate's endpoint already reports per GPU-hour and needs no API key:

import requests

data = requests.get("https://www.runcrate.ai/api/gpu-pricing").json()
print("fetched at", data["fetched_at"])

for g in data["gpus"]:
    s = g["specs"]
    per_gpu = s["num_gpus"] or 1
    print(
        f'{g["name"]:12} {g["vram"]:>6}  '
        f'${g["min_price_hr"]:.2f} to ${g["max_price_hr"]:.2f} per GPU-hr  '
        f'{g["available_options"]:>2} configs  GPUs per machine {g["gpu_counts"]}  '
        f'~{s["vcpus"] // per_gpu} vCPU / {s["memory_gb"] // per_gpu} GB RAM per GPU'
    )

Part of what it printed when I ran it (fetched at 2026-10-21T00:51 UTC):

H100 SXM       80GB  $3.38 to $7.79 per GPU-hr   7 configs  GPUs per machine [1, 8]  ~20 vCPU / 128 GB RAM per GPU
A100 80GB      80GB  $1.76 to $4.26 per GPU-hr  12 configs  GPUs per machine [1, 2, 8]  ~28 vCPU / 120 GB RAM per GPU
L40S           48GB  $1.14 to $1.95 per GPU-hr  10 configs  GPUs per machine [1, 2, 4, 8]  ~12 vCPU / 72 GB RAM per GPU
A6000          48GB  $0.65 to $2.51 per GPU-hr  18 configs  GPUs per machine [1, 2, 4]  ~28 vCPU / 58 GB RAM per GPU

Notice the ranges. One GPU model can have a dozen configurations with a 2x to 4x spread between them. A comparison that keeps only the minimum hides that the minimum might be one 8-GPU node in one region.

2. Compare the same GPU, not the same name

"A100" is two products (40 GB and 80 GB). "H100" is SXM or PCIe, with different memory bandwidth and interconnect. A "4090" with 24 GB will not hold the model your 48 GB card holds. Before comparing prices, match:

VRAM (the endpoint has vram per GPU type)
form factor and interconnect for multi-GPU work (interconnect, for example NVLink)
GPU count per machine, because you can't split an 8-GPU node

If your model needs 80 GB, the price of a 40 GB card is irrelevant, however low it is.

3. Count what comes with the GPU

A GPU-hour also buys CPU cores, RAM and local disk, and those vary a lot. In the output above, an A6000 can come with about 28 vCPUs, an L40S with about 12. For data loading, tokenization or anything CPU-bound in your pipeline, the "slower" card with more cores can finish first.

So note, per GPU: vCPUs, RAM, local disk, and whether persistent storage costs extra. Runcrate's volumes, for example, are billed separately at $0.03 per GB per month. Your other quotes will have their own line for it, or bundle it.

4. Billing granularity matters more than the rate for short work

This is the one most comparisons skip. Take a 7-minute eval on a $1.14/hr GPU:

billed per second: 1.14 x 7 / 60 = $0.13
billed per started hour: $1.14, nearly 9x more

For long training runs granularity barely matters. For evals, CI jobs, agent experiments and anything bursty, it dominates. Check three things:

The billing increment (second, minute, hour). Runcrate bills most instances by the second and a few GPUs by the minute; the endpoint's min_price_per_minute flag marks the latter.
When the meter starts: at request, at boot, or once the machine is running. Runcrate starts it when the machine is running, so provisioning isn't billed.
When it stops: only when you remember to delete the machine, or also on its own. Idle shutdown and maximum runtimes turn "I forgot it overnight" from a 12-hour bill into a 30-minute one, which is worth more than a few cents on the rate.

5. Listed is not launchable

A price only matters if you can get the machine. Availability moves by the minute: in the same hour I wrote this, a GPU type showed one configuration on the pricing endpoint while a live launch query showed none. When comparing:

try an actual launch, or at least a live availability query, in the region you need
note the count of configurations and regions, not just the headline price
separate on-demand from spot or interruptible capacity, which is cheaper because it can disappear

6. Date every number

GPU prices change weekly, sometimes daily. A comparison without a date is a guess. Put the fetch time next to every price (the endpoint returns fetched_at for that reason), and re-run your script rather than reusing last month's table.

A template you can reuse

For each offer, one row:

Field	Why
GPU model, VRAM, form factor	same product?
GPUs per machine	can you buy just one?
$/GPU-hour	the normalized rate
vCPU, RAM, disk per GPU	the rest of the machine
Billing increment, meter start	cost of short jobs
Auto-stop options	cost of mistakes
Storage and egress	cost of data
Region, availability checked at	can you launch it
Fetched at	how old the number is

Then price your actual workload (hours per run, runs per month, data moved) against each row. The offer with the best headline wins less often than you'd think.

DEV Community