Renting a cloud GPU looks simple until the invoice arrives. Before you pick between an H100, an A100 and an L40S, run through this checklist.
1. VRAM first, FLOPS second
- H100: 80 GB (HBM3). A100: 40 GB or 80 GB variants. L40S: 48 GB (GDDR6).
- If the model plus KV cache does not fit, speed is irrelevant. Check which A100 variant a listing actually offers.
2. Interconnect
- Training across several GPUs needs NVLink or fast InfiniBand. Many cheap listings use PCIe only.
- The L40S has no NVLink, so it suits inference and single-GPU fine-tuning better than multi-GPU training.
3. Egress and storage
- Ask what moving your checkpoints and datasets out costs. Some providers charge per GB, others nothing.
- Persistent volumes are often billed even when the instance is stopped.
4. Spot vs on-demand
- Spot can be far cheaper but can be reclaimed with little warning. Only use it with frequent checkpointing.
- On-demand suits long jobs that cannot restart, and anything serving users.
5. Billing granularity
- Per-second, per-minute and per-hour billing differ a lot for short experiments. A 10-minute test billed as a full hour is a real cost.
- Check minimum commitments and idle-time charges.
6. Read the date on the price
GPU prices move often. Always note when a number was read from the provider's pricing page.
If you want a side-by-side view, AskGPU is an independent comparison of GPU cloud, AI API and hosting prices, each read from the provider's page and dated, with no paid rankings.
What else do you check before you rent? Tell me in the comments.
Top comments (0)