A100 vs H100: is the H100 worth it for LLM training?

The H100 is worth it when it finishes your training job at least as many times faster as it costs more per hour. On Nodus the NVIDIA H100 SXM is listed at $2.60/hr and the A100 80GB at $1.09/hr, so the H100 must run 2.39 times faster to be the cheaper way to finish. Large BF16 or FP8 training usually gets there; small, memory-bound or poorly fed jobs often do not.

That ratio is the whole decision, and it changes with each provider's price list. Under the line, the A100 gives you the cheaper finished job. Over it, the H100 gives you a cheaper job that also finishes sooner.

How much faster is an H100 than an A100 on paper?

Between 1.50 and 3.17 times, depending on which part of the chip your job leans on. Every number here comes from NVIDIA's own spec tables for the A100 80GB SXM and the H100 SXM.

	A100 80GB SXM	H100 SXM	H100 ÷ A100
BF16 Tensor Core, dense	312 TFLOPS	989.5 TFLOPS	3.17x
FP8 Tensor Core, dense	Not supported	1,979 TFLOPS	New on H100
Memory	80 GB HBM2e	80 GB	Same capacity
Memory bandwidth	2,039 GB/s	3,350 GB/s	1.64x
NVLink per GPU	600 GB/s	900 GB/s	1.50x
Max power	400 W	Up to 700 W	1.75x

NVIDIA lists H100 Tensor Core figures with sparsity (1,979 BF16 and 3,958 FP8 TFLOPS). Dense training gets half of those, which is what the table shows.

These ratios bracket what a real job sees. A job limited by matrix math, like a large model at a big batch size, moves toward the 3.17x compute ratio. A job limited by moving bytes, like a small model, a short sequence or heavy optimizer traffic, moves toward the 1.64x bandwidth ratio. Neither number is a benchmark, and a slow data loader holds both GPUs to the same speed.

The Nodus price ratio sits between the bandwidth ratio and the compute ratio, so your workload decides the winner.

When does the H100 cost less per finished job?

When your measured speedup beats the hourly price ratio. The rule is one division: H100 price per hour divided by A100 price per hour. If the H100 finishes the job faster than that multiple, it is the cheaper choice.

The ratio depends on where you run. We divided each provider's own listed rates for the 80 GB A100 and the H100 SXM:

Where	A100 80GB per hour	H100 SXM per hour	Speedup the H100 needs
Nodus	$1.09	$2.60	2.39x
RunPod (Pods, A100 SXM)	$1.79	$3.99	2.23x
Modal ($0.000694 and $0.001097 per second)	$2.50	$3.95	1.58x

Sources: Nodus pricing, RunPod pricing, Modal pricing. Prices checked November 2, 2026.

A cheap A100 rate raises the bar the H100 has to clear.

On Modal the H100 wins almost by default, because its premium over the A100 is small. On Nodus the A100 is cheap enough that the H100 has to earn its place with a compute-bound job.

What does that look like for a real fine-tune?

Take a fine-tune that needs 20 hours on one A100 80GB. On Nodus that is 20 x $1.09 = $21.80. The same job on one H100 at the three speedups that matter:

H100 speedup	H100 hours	H100 cost on Nodus
1.64x (bandwidth-bound)	12.20 h	$31.71
2.39x (breakeven)	8.38 h	$21.80
3.17x (compute-bound, at the spec ceiling)	6.31 h	$16.40

A memory-bound job costs 45% more on the H100; a compute-bound one costs 25% less.

The spread is large. Picking the wrong GPU for this job costs between $5.40 and $9.91, before you count the hours you wait. On Modal the same job runs $49.97 on the A100 and $48.16 on an H100 at only 1.64x, which is why the answer flips between providers.

How do you measure your own speedup in 30 minutes?

Run a short probe of your real training script on each GPU and compare steady-state throughput. Keep the model, batch size, sequence length and data identical. Skip the first few dozen steps, because compilation and warm-up distort them.

Add a timer to your loop that prints tokens or samples per second every 50 steps. Then launch the probe twice with a hard spending cap, using the nodus run flags from the Nodus Jobs guide:

nodus run --name probe-a100 --gpu A100-80GB --image nodus/pytorch --max-cost 1.00 --timeout 20m -- python train.py --max-steps 300
nodus run --name probe-h100 --gpu H100 --image nodus/pytorch --max-cost 1.00 --timeout 20m -- python train.py --max-steps 300

At listed rates, 15 minutes costs $0.27 on the A100 and $0.65 on the H100. Divide the H100's tokens per second by the A100's. Compare that number with the breakeven for where you will run the full job.

If you want FP8, test it as a third probe. FP8 only exists on the H100, and NVIDIA credits its Transformer Engine with up to 4x faster training than the prior generation on GPT-3 175B. That is NVIDIA's projection for a very large model, so check that your loss curve holds before you count on it.

Where is the A100 the better choice?

The A100 wins whenever your job cannot keep an H100's Tensor Cores busy. These are the common cases:

• Small models and LoRA fine-tunes on 7B to 13B models, where batches are small and memory traffic dominates.
• Data pipelines that cannot feed the GPU faster. If an A100 already waits on the loader, an H100 waits longer.
• Code pinned to FP16 or TF32 kernels that predate Hopper, or libraries without FP8 support.
• Long runs without a deadline, where a lower cost matters more than finishing sooner.
• Power or rack limits on your own hardware: 400 W per A100 against up to 700 W per H100.

Memory is a tie. Both have 80 GB, so a model that does not fit on an A100 80GB will not fit on an H100 SXM either. For more memory per GPU you need an H200 or a B200, and our VRAM guide for 70B models shows the math.

When is the H100 clearly worth it?

Pay for the H100 when time matters or the job is compute-bound. Pretraining and continued pretraining, full fine-tunes of 30B and larger models, long sequences and FP8 recipes all push toward the compute ratio.

Multi-GPU jobs lean the same way. NVLink is 1.5x faster on the H100, so gradient and tensor-parallel traffic between 8 GPUs costs less time. For why the H100 holds its price in the first place, see why the H100 is so expensive.

A deadline can also justify an H100 below the breakeven. At 1.64x, our 20-hour job finishes 7.8 hours sooner for $9.91 more.

How do you let Nodus pick between A100 and H100?

List both types in the Job and set a cost cap. The Nodus GPU guide says the scheduler picks among the accepted types by expected cost to finish.

apiVersion: nodus.dev/v1
kind: Job
metadata:
  name: finetune
spec:
  image: nodus/pytorch
  command: [torchrun, train.py]
  resources:
    gpu: {type: [A100-80GB, H100], count: 8}
  timeout: 24h
  maxCostUSD: "250.00"

If your probe already settled the question, pin one type instead, for example --gpu H100:8. Run nodus apply -f job.yaml --dry-run=server -o estimate first to see the offering, the expected cost and the startup time without spending anything. For an 8-GPU launch with torchrun, our multi-GPU PyTorch guide walks through the setup.

FAQ

Is the A100 better than the H100?

Not on any performance spec; it only draws less power. The A100 is the better value when your job runs less than about 2.4 times faster on the H100 at Nodus list prices.

Is the A100 40GB worth considering for LLM training?

Only for small models or LoRA on 7B-class models with short sequences. Its 40 GB fills quickly once you add optimizer states and activations, so most LLM training plans should start from the 80 GB version.

Can I train in FP8 on an A100?

No. The A100 has no FP8 Tensor Cores, so FP8 recipes fall back to BF16 or fail. The H100 adds FP8 through its Transformer Engine.

Do H100 PCIe-based cards match the H100 SXM?

No. NVIDIA lists the PCIe-based H100 NVL at 1,671 BF16 TFLOPS with sparsity against 1,979 for the SXM, and 600 GB/s NVLink against 900 GB/s. On Nodus, H100 accepts any variant, while H100-SXM or H100! pins the SXM version.

Does a cheaper A100 rate still pay off if the run gets interrupted?

Only if the run resumes from a checkpoint. A restart from zero repeats the work and erases the saving. Nodus restores the latest checkpoint on new capacity, which our checkpointing guide covers.

Originally published on the Nodus blog.