If you are fine-tuning and you jump straight to the biggest GPU, you are probably wasting money.
The fast answer
- RTX 4090 is enough for a lot of LoRA and QLoRA work
- A100 80GB is the move when VRAM becomes the limiting factor
- H100 only makes sense when you already know your fine-tune is large enough to need it
For most teams, start here
Start with RTX 4090.
It is usually the best starting point for:
- smaller model fine-tunes
- LoRA workflows
- QLoRA workflows
- cheaper experimentation
When you should move up
Move to A100 when VRAM is blocking you.
Move to H100 only when the workload proves it.
The mistake that burns budget
A lot of people optimize for the GPU they have heard of, not the GPU the workload actually needs.
Fine-tuning is especially easy to overspend on because LoRA and QLoRA let smaller GPUs do far more than people expect.
The simple rule
Start with the cheapest GPU that can hold the job.
Upgrade only when the workload forces you to, not because the bigger card feels safer.
Top comments (0)