Pick the smallest GPU that fits your model in Unsloth's VRAM table, save the trainer's checkpoints to a directory that survives a restart, and run your script as a batch job with a spending cap. Nodus runs that script on an RTX 4090 listed at $0.29/hr or an H100 at $2.60/hr, and restores the checkpoint directory on a new machine if the GPU is reclaimed. A preemption then costs you minutes of training, not the whole run.
We build Nodus, so weigh that. Unsloth facts below link to Unsloth's own docs, other platforms' prices link to their pricing pages, and every cost figure shows its math.
What does Unsloth change about fine-tuning?
Unsloth makes the same fine-tune need a much smaller GPU. Its GitHub README says it trains LLMs 2x faster with 70% less VRAM and no accuracy loss. The core package is Apache 2.0, so commercial use is fine.
The memory saving matters more than the speed for your bill. By Unsloth's own table, a 14B model needs 33 GB for 16-bit LoRA but 8.5 GB with 4-bit QLoRA. That moves it from a 48 GB L40S, listed at $0.793/hr on the Nodus pricing page, to a 24 GB RTX 4090 at $0.29/hr.
Unsloth plugs into Hugging Face TRL's SFTTrainer. That detail is what makes cloud runs easy to recover, because the Trainer already knows how to save and resume.
Which GPU do you need for your model size?
Use Unsloth's own VRAM requirements table and add headroom. Unsloth calls these numbers "the absolute minimum", and longer sequences or bigger batches need more.
| Model size | QLoRA (4-bit) minimum | LoRA (16-bit) minimum | A GPU that fits QLoRA with headroom |
|---|---|---|---|
| 3B | 3.5 GB | 8 GB | NVIDIA L4 or RTX 4090 (24 GB) |
| 8B | 6 GB | 22 GB | RTX 4090 or L4 (24 GB) |
| 14B | 8.5 GB | 33 GB | RTX 4090 or L4 (24 GB) |
| 32B | 26 GB | 76 GB | L40S or RTX A6000 (48 GB) |
| 70B | 41 GB | 164 GB | L40S (48 GB) for short sequences, A100 80 GB or H100 for room |
QLoRA keeps every model up to 70B on a single GPU; 16-bit LoRA needs an 80 GB card at 32B and several cards at 70B.
Our rule: start with QLoRA on a 24 GB card for anything up to 14B. Switch to 16-bit LoRA only when you have measured a quality gap that matters to you. On Nodus you can also skip naming a GPU and set resources.gpu.minMemory in the Job spec, and Nodus picks any accelerator with at least that much memory.
Is a bigger GPU ever cheaper per finished run?
Yes, but only if it finishes proportionally faster. Divide the bigger card's hourly price by the smaller card's to get the speedup it must deliver just to break even. Here is that ratio against an RTX 4090 at Nodus list prices:
| GPU | Listed on the Nodus pricing page | Speedup needed vs RTX 4090 |
|---|---|---|
| RTX 4090 (24 GB) | $0.29/hr | 1.0x |
| L4 (24 GB) | $0.43/hr | 1.5x |
| L40S (48 GB) | $0.793/hr | 2.7x |
| A100 80 GB | $1.09/hr | 3.8x |
| H100 SXM 80 GB | $2.60/hr | 9.0x |
Prices checked October 28, 2026 on the Nodus pricing page.
An H100 has to train 9x faster than an RTX 4090 before it saves money on the same job.
We have not benchmarked Unsloth across these cards, so measure it yourself. Run 50 steps on two GPUs, compare seconds per step, and multiply by the hourly price. Unless the H100 comes close to 9x faster on your job, the cheaper card finishes it for less. Pay for the H100 when you need the result sooner or the model does not fit anywhere smaller.
How do you make an Unsloth script resume after a restart?
Point the Trainer's output_dir at the Nodus state directory and resume from the newest checkpoint on start. The script below is Unsloth's Llama 3.1 8B Alpaca notebook with those two changes, plus a callback that saves when Nodus asks.
from unsloth import FastLanguageModel # import first so its patches apply
import os
from datasets import load_dataset
from transformers.trainer_utils import get_last_checkpoint
from trl import SFTConfig, SFTTrainer
import nodus
STATE = os.environ.get("NODUS_STATE_DIR", "/nodus/state")
OUTPUTS = os.environ.get("NODUS_OUTPUTS_DIR", "/nodus/outputs")
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Llama-3.1-8B", max_seq_length=2048, load_in_4bit=True)
model = FastLanguageModel.get_peft_model(
model, r=16, lora_alpha=16, lora_dropout=0, bias="none",
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth", random_state=3407)
prompt = "### Instruction:\n{}\n\n### Input:\n{}\n\n### Response:\n{}"
def fmt(ex):
return {"text": [prompt.format(i, x, o) + tokenizer.eos_token
for i, x, o in zip(ex["instruction"], ex["input"], ex["output"])]}
dataset = load_dataset("unsloth/alpaca-cleaned", split="train").map(fmt, batched=True)
trainer = SFTTrainer(
model=model, tokenizer=tokenizer, train_dataset=dataset,
dataset_text_field="text", max_seq_length=2048,
callbacks=[nodus.checkpoint.HFTrainerCallback()],
args=SFTConfig(
output_dir=STATE, save_steps=200, save_total_limit=2,
per_device_train_batch_size=2, gradient_accumulation_steps=4,
num_train_epochs=1, learning_rate=2e-4, warmup_steps=5,
optim="adamw_8bit", logging_steps=10, seed=3407, report_to="none"))
last = get_last_checkpoint(STATE) if os.path.isdir(STATE) else None
trainer.train(resume_from_checkpoint=last)
model.save_pretrained(f"{OUTPUTS}/lora-adapter")
tokenizer.save_pretrained(f"{OUTPUTS}/lora-adapter")
Three lines do the recovery work. output_dir=STATE puts Trainer's checkpoints where Nodus snapshots them. get_last_checkpoint finds the newest checkpoint-N folder after a restore, and stays None on a first start.
HFTrainerCallback comes from the Nodus SDK and makes Trainer save at the end of the current step when Nodus asks, including when the capacity gives notice that it is about to be reclaimed. Outside Nodus the SDK calls do nothing, so the same file runs on your laptop or in Colab.
save_total_limit=2 keeps the directory small. With QLoRA each checkpoint holds the adapter and optimizer state, not a copy of the 8B base model.
How do you launch it as a Nodus Job?
Install the CLI, sign in, and run from the folder that holds train.py. These flags come from the Nodus checkpoints guide and the nodus run reference:
$ pip install nodus-compute
$ nodus login
$ nodus run --name unsloth-llama8b --gpu L4 --image nodus/pytorch \
--checkpoint /nodus/state --interruptible \
--max-cost 10 --timeout 24h -d \
-- sh -c "pip install unsloth nodus-compute && python train.py"
$ nodus logs -f job/unsloth-llama8b
$ nodus cp job/unsloth-llama8b:outputs/lora-adapter/adapter_model.safetensors .
An L4 fits this 8B QLoRA run with room to spare; nodus get gpus lists the other types and their current rates, including the RTX 4090. --interruptible lets Nodus use interruptible capacity, which is exactly the case the checkpoint setup covers.
Add --dry-run first to see the cost estimate without starting anything. --max-cost 10 is a cap, not a quote: at the cap the Job saves its state and suspends until you raise it. Everything under /nodus/outputs is collected when the Job succeeds, so copy adapter_config.json the same way. Installing Unsloth at start adds install time to every attempt, including restored ones. For repeat runs, pin the versions, push your own image to a registry and pass it to --image.
What does a preemption cost with and without checkpoints?
Say a 10-hour QLoRA run on one H100 is reclaimed at hour 6. At $2.60/hr the whole run costs $26.00.
Without checkpoints you start over and pay the first 6 hours twice, an extra $15.60. With save_steps set so Trainer saves about every 30 minutes, the most you redo is 30 minutes, or $1.30. The Nodus callback usually shrinks that further, because Nodus asks for a save when the capacity gives notice. The checkpointing comparison covers which other platforms restore a run like this for you.
What does the same run cost on other platforms?
At list prices, 10 hours on one H100 SXM is cheapest on Nodus, with RunPod's Community Cloud close behind:
| Platform | H100 SXM hourly rate | 10 hours |
|---|---|---|
| Nodus | $2.60/hr | $26.00 |
| RunPod Community Cloud | $2.69/hr | $26.90 |
| Modal | $3.949/hr | $39.49 |
| RunPod Secure Cloud | $3.99/hr | $39.90 |
Prices checked October 28, 2026: Nodus pricing page, RunPod pricing, Modal pricing.
The rate gap is small next to the GPU choice: the same 10 hours on an RTX 4090 at $0.29/hr is $2.90.
The hourly rate is only part of it. A RunPod Pod gives you a machine where you run and restart the script yourself. Modal restarts a preempted function on the same input and asks you to add your own checkpoints. If you already work in either, Unsloth runs there too, and the script above works there apart from the state path and the Nodus callback.
When is Colab or your own GPU enough?
For learning Unsloth and short experiments, start in Google Colab. Unsloth's README links a free Colab notebook for each of its featured models. The catch is that Colab's FAQ says free resources are not guaranteed, idle VMs are deleted, and lifetimes and GPU types vary over time.
A 24 GB card you already own is enough for QLoRA up to 14B by Unsloth's table. It costs nothing per hour and your data never leaves the machine. Move to a cloud job when the run has to finish overnight without you watching it, when the model outgrows your card, or when you want several runs at once.
If you would rather not maintain a training script, the managed nodus/sft runtime in Nodus Training (Beta) does full, LoRA or QLoRA fine-tuning from a JSONL file. It is not Unsloth, so you give up Unsloth's kernels for less code.
FAQ
Does Unsloth work on multiple GPUs?
Unsloth's README says it supports multi-GPU setups. For most QLoRA jobs up to 70B, one GPU is enough by its own VRAM table, and a single card avoids the communication overhead entirely.
Do I need a Hugging Face token?
Only for gated models. Unsloth publishes copies of many open models under its Hugging Face organization. For a gated one, store your token as a Nodus Secret and pass it with --secret.
Can I use Unsloth's Docker image instead of pip?
Unsloth publishes an official Docker image, unsloth/unsloth, built mainly for running Jupyter and Unsloth Studio locally. Nodus runs prebuilt public registry images, but test the image's startup command before a long run.
How do I keep a large dataset from uploading on every run?
Put it in a Nodus Volume once with nodus volume put, then mount it with --volume NAME:/data:ro. Only the code folder uploads with each run.
What do I do with the adapter afterwards?
Load it back with FastLanguageModel.from_pretrained pointed at the adapter folder, or merge it into 16-bit weights with Unsloth's save_pretrained_merged. The VRAM guide covers what serving the merged model needs.
Originally published on the Nodus blog.
Top comments (0)