DEV Community

Viswaretas Kotra
Viswaretas Kotra

Posted on Originally published at nodus-compute.ai

How do you estimate GPU hours and cost for a training run?

Estimate GPU-hours as 6 × parameters × training tokens divided by (one GPU's peak dense FLOPS × utilization × 3,600), then multiply by the hourly rate per GPU to get the cost. On Nodus you can turn that estimate into an enforced per-job limit with maxCostUSD, and a job that reaches it checkpoints and suspends instead of losing its work. This post packages the formula as a short Python function we ran, prices one run with it, shows how far utilization and token count move the bill, and gives the rule we use to set the cap.

Our post on what it costs to train an LLM from scratch checks the formula against published runs, and how long training takes covers the wall-clock side. Here we only build and use the calculator.

What does a training cost calculator need as inputs?

Six numbers, and only one of them is hard to know. You need the parameter count, the number of training tokens, the GPU's peak dense BF16 throughput, the utilization you expect, how many GPUs you will use and the hourly rate per GPU. The 6 comes from Kaplan et al.'s scaling-laws paper: about 2 operations per parameter per token for the forward pass and 4 for the backward pass.

Utilization is the hard one. The standard name is model FLOPs utilization, or MFU, defined in Google's PaLM paper as the share of peak throughput that goes into the model's own math. Plan at 30% unless you have measured your own code.

Peak numbers come from NVIDIA's spec pages, and you must use the dense figure. The H100 page lists 1,979 teraFLOPS of BF16 for the SXM card, a sparsity figure, so dense is 989. The A100 page lists 312 dense. The HGX page lists 36 petaFLOPS of sparse BF16 for an 8-GPU B200 or B300 board and says dense is half, so 2,250 teraFLOPS per GPU for both.

What does the calculator look like in Python?

It is ten lines. Copy it, change the inputs, and keep flops_per_param_token at 6 unless the next sections tell you otherwise.

def estimate(params, tokens, peak_tflops, mfu, gpus, usd_per_gpu_hr,
             flops_per_param_token=6):
    """GPU-hours, wall-clock hours and compute cost for one training run."""
    total_flops = flops_per_param_token * params * tokens
    useful_flops_per_gpu_hr = peak_tflops * 1e12 * mfu * 3600
    gpu_hours = total_flops / useful_flops_per_gpu_hr
    return {
        "gpu_hours": round(gpu_hours, 1),
        "wall_clock_hours": round(gpu_hours / gpus, 1),
        "cost_usd": round(gpu_hours * usd_per_gpu_hr, 2),
    }

# 8B model, 20B tokens of continued pretraining, 8 x H100 SXM at $2.60/hr
print(estimate(params=8e9, tokens=20e9, peak_tflops=989,
               mfu=0.30, gpus=8, usd_per_gpu_hr=2.60))
Enter fullscreen mode Exit fullscreen mode

We ran this exact file with Python 3 on November 19, 2026. It printed:

{'gpu_hours': 898.8, 'wall_clock_hours': 112.3, 'cost_usd': 2336.82}
Enter fullscreen mode Exit fullscreen mode

So continued pretraining of an 8B model on 20 billion tokens needs about 899 H100-hours, about 4 days and 16 hours on one 8-GPU machine, and $2,336.82 at the H100 SXM rate listed at $2.60/hr on the Nodus pricing page. Wall-clock assumes the 8 GPUs scale perfectly, which is close for one machine and optimistic across many.

How much do utilization and token count move the cost?

Utilization moves it by up to 2x across the realistic range, and tokens move it in exact proportion. We ran the same function for the 8B example at 25%, 30% and 50% MFU and at four dataset sizes.

Training tokens At 25% MFU At 30% MFU At 50% MFU
5B $701.04 $584.20 $350.52
10B $1,402.09 $1,168.41 $701.04
20B $2,804.18 $2,336.82 $1,402.09
40B $5,608.36 $4,673.63 $2,804.18

8B parameters on 8 x H100 SXM at $2.60/hr, our runs of the function above. Prices checked November 19, 2026.

Estimated cost of an 8B training run on 8 x H100 at $2.60/hr: at 25% MFU $701.04, $1,402.09, $2,804.18 and $5,608.36 for 5B, 10B, 20B and 40B tokens; at 30% MFU $584.20, $1,168.41, $2,336.82 and $4,673.63; at 50% MFU $350.52, $701.04, $1,402.09 and $2,804.18.
Doubling tokens doubles the bill; going from 25% to 50% MFU halves it.

Two cells match: 40B tokens at 50% MFU cost the same $2,804.18 as 20B tokens at 25%. Doubling utilization saves as much as halving your dataset, without giving up any data.

Which GPU gives the cheapest run for the same job?

The one with the lowest hourly rate per unit of dense math, provided the model fits. Running the 8B, 20B-token job at 30% MFU on 8 GPUs of each type gives these costs at Nodus list rates: A100 80 GB at $1.09/hr comes to $3,105.41, H100 SXM at $2.60/hr to $2,336.82, B200 at $5.50/hr to $2,172.84 and B300 at $6.943/hr to $2,742.91.

Estimated cost of the same 8B, 20B-token run at 30% MFU on 8 GPUs: A100 80 GB $3,105.41, B300 $2,742.91, H100 SXM $2,336.82, B200 $2,172.84.
For BF16 training, the B300 has the same listed peak as the B200 but a higher rate, so it costs more per run.

The B200 run also finishes in 49.4 hours instead of 112.3. One caveat: we used the same 30% MFU on every card, and real utilization differs between GPU generations, so measure a short run on your top two choices before committing. Our 70B fine-tuning cost post ranks the same cards by price per petaFLOPS.

How do you adjust the formula for fine-tuning?

Change the token count first, then decide whether to change the 6. For supervised fine-tuning, training tokens are examples × average tokens per example × epochs. Count prompt tokens too: the GPU runs the forward and backward pass over the whole sequence even when the loss only covers the answer.

The 6 changes in two cases, and both are arithmetic, not measurements:

  • Activation checkpointing reruns the forward pass, adding about 2 operations, so a full fine-tune does about 8 per parameter per token. Narayanan et al. count the same extra forward pass in their Megatron-LM FLOP formula.
  • LoRA freezes the base weights, so the backward pass skips weight gradients and drops from about 4 operations to 2. That gives roughly 4 without checkpointing and 6 with it.

Keep one convention when you plug in a measured utilization. MFU as PaLM defines it leaves recomputation out, so if your number is MFU, keep the 6. If your profiler reports hardware utilization that counts the recompute, use 8.

How do you budget for evaluation, retries and startup?

Price each one as its own line, because each has a different cause. Evaluation is forward passes only, so set flops_per_param_token=2. Our example evaluates 20 million tokens after every billion trained, 400 million eval tokens in total, which the function prices at 6.0 GPU-hours and $15.58.

Retries cost the work since your last checkpoint plus the restart. We plan for 3 interruptions, each losing 30 minutes of training and 10 minutes of restart on 8 GPUs: 16 GPU-hours, or $41.60. Startup adds 15 minutes on 8 GPUs, 2 GPU-hours or $5.20. Those three numbers are our planning assumptions, not measurements; replace them with your own checkpoint interval and history.

The planned total for the 8B example comes to $2,399.20: $2,336.82 of training plus $62.38 of overhead.

Budget for the 8B, 20B-token run on 8 x H100: planned at 30% MFU, $2,336.82 training, $15.58 eval, $41.60 retries and $5.20 startup; capped at the 25% floor, $2,804.18 training plus the same overheads.
The cap is the plan repriced at the lowest utilization you would accept.

How do you turn the estimate into a spend cap on Nodus?

Set the cap at the cost of your worst acceptable utilization plus overheads, and no higher. For the 8B example, the floor is 25% MFU: $2,804.18 of training plus $62.38 gives $2,866.56, so we would cap the job at $2,870.

That rule works because of how the Nodus cap behaves. Per the Budgets and spending caps docs, a Job never holds more than its maxCostUSD cap; when its spend reaches the cap, it checkpoints and becomes Suspended with reason MaxCostReached, and raising the cap resumes it from the checkpoint. You can raise a cap at any time but never lower it, so start tight.

A tight cap doubles as a utilization alarm. If the job suspends before it has trained its 20 billion tokens, it ran below 25% MFU. Check throughput before you raise the cap.

Before launch, nodus run --dry-run prints the estimate only, and the placement docs say a server dry run returns expected cost at p50 and p90 and startup time. Compare its p90 with your own number, then launch with the cap:

nodus run --dry-run --gpu H100:8 -- torchrun train.py
nodus run --gpu H100:8 --max-cost 2870 -d -- torchrun train.py
Enter fullscreen mode Exit fullscreen mode

Placement also skips any offering whose expected cost exceeds the money left under your cap or Budget, so the cap shapes where the job runs, not just when it stops.

FAQ

Is 6 x params x tokens accurate for mixture-of-experts models?

Use active parameters, not total. Each token only passes through the experts the router picks, so the math per token follows the active count.

Does a longer context length change the estimate?

Yes, at long sequences. The 6 counts the weight math only; attention adds work that grows with sequence length and becomes a visible share past a few thousand tokens per sequence for small models. Measure a short run at your real sequence length.

What should I do if the job hits its Nodus cap early?

Look at tokens per second per GPU before raising the cap. A low number usually means a data loader, communication or batch-size problem, and fixing it saves more than the cap increase costs.

Can one cap cover a whole hyperparameter sweep?

Yes. The Nodus docs say a cap on a Pipeline or Sweep is shared by every run it creates, so price one run with the calculator and multiply by the number of runs.

Originally published on the Nodus blog.

Top comments (0)