DEV Community: Viswaretas Kotra

Nodus vs Google Cloud GPUs: which is cheaper for training?

Viswaretas Kotra — Thu, 29 Oct 2026 23:12:35 +0000

Per GPU-hour, a Google Cloud on-demand H100 costs about four times the Nodus listed rate, and Google Spot VMs cost about two and a half times as much. Spot narrows the gap, but Compute Engine can reclaim those machines at any time and leaves checkpoint and restart logic to you. Nodus lists an NVIDIA H100 at $2.60/hr on its pricing page, saves your state when interruptible capacity is reclaimed, and can route work through a connected GCP project so you spend Google credits before Nodus capacity.

We build Nodus. Every Google Cloud fact below links to Google's own pricing page or docs, the arithmetic is shown, and one section covers where Google Cloud is the better choice.

How do Google Cloud and Nodus GPU prices compare?

Google Cloud prices accelerator-optimized VMs as a full machine: GPUs, vCPUs, memory and local SSD bundled together. The A3 High machine type gives you eight NVIDIA H100 GPUs, 208 vCPUs, 1,871 GB of memory and 6,000 GB of SSD. Nodus lists the GPU hour on its own, with CPU and memory priced separately on the same page.

GPU	Google Cloud on-demand	Google Cloud Spot	Nodus listed rate
8 x H100 SXM (A3 High)	$88.49/hr ($11.06/GPU)	$52.70/hr ($6.59/GPU)	$20.80/hr ($2.60/GPU)
8 x H100 SXM (A3 Mega)	$93.40/hr ($11.68/GPU)	$45.60/hr ($5.70/GPU)	$20.80/hr ($2.60/GPU)
8 x H200 SXM (A3 Ultra)	$84.81/hr ($10.60/GPU)	$50.87/hr ($6.36/GPU)	$28.74/hr ($3.59/GPU)
8 x A100 40 GB (A2 Standard)	$29.39/hr ($3.67/GPU)	$16.75/hr ($2.09/GPU)	$7.23/hr ($0.904/GPU)

Prices checked October 29, 2026. Google Cloud rates from the accelerator-optimized pricing page for us-central1 (Iowa). Nodus rates from the pricing page.

At list prices, Nodus is about 76% cheaper than Google Cloud on-demand and 61% cheaper than Spot per H100-hour.

What does the same training job cost on each?

We priced the same 8 x H100 run for 10 hours, the worked example we use across our comparison posts.

Platform and capacity	Hourly rate	80 GPU-hours
Google Cloud A3 High, on-demand	$88.49/hr	$884.90
Google Cloud A3 High, Spot	$52.70/hr	$527.00
Nodus	$20.80/hr	$208.00

Prices checked October 29, 2026. Google Cloud: accelerator-optimized pricing, us-central1. Nodus: pricing page.

The Nodus run costs less than a quarter of the GCP on-demand price and about 39% of the Spot price for the same GPU-hours. One caveat keeps this fair: Google's price includes 208 vCPUs and 1,871 GB of memory, while the Nodus figure is the GPU line only. Nodus lists CPU and memory separately (4 vCPU with 16 GiB is $0.1054/hr), so a job that needs a large host adds to the Nodus side.

The same 80 H100-hours cost $677 less on Nodus than on Google Cloud on-demand.

Google Cloud's Spot rate looks closer, but the Spot VM docs give a best-effort shutdown period of up to 30 seconds, and the run does not resume by itself. You write the checkpoint code, save to Cloud Storage and relaunch. On Nodus, an interruptible run whose machine is reclaimed gets a replacement with its /nodus/state directory restored, so you save to that path and Nodus handles the restart. That pattern is covered in How do you make spot GPU training survive preemption?.

Can you use both Google Cloud and Nodus?

Yes, and if you hold Google credits it is often the cheapest setup. Nodus can connect a GCP project read-only to show its instances and spend, and, as a separate Beta step, run your single-node Jobs in that project first so they spend your Google Cloud credits before Nodus capacity.

The read-only connection is an OAuth grant limited to read-only scopes (compute.readonly, bigquery.readonly and cloud-platform.read-only). Viewing is free and starts no machines. Running work is an explicit upgrade with a monthly credit limit. Nodus then creates its own network and a service account with roles that can only touch machines it named, and launches in us-central1, us-east1, us-east4 or us-west1.

When the project is full or out of credit, the default is to continue on Nodus capacity at Nodus rates. You can instead make Jobs wait for your approval or wait for your account. Once the owner setup script has finished, the CLI step is one line, copied from the docs:

nodus cloud connect gcp gcp-lab --credit-limit 500 --gcp-project lab-ml-prod

Each attempt on the job page shows whether it ran in your project or on Nodus capacity. Multi-node runs and Sandboxes use Nodus capacity, and the console shows the routing terms before you enable it. The full steps and permissions are in Connect your cloud accounts.

Is Google Cloud Spot stable enough for training?

Spot is stable enough only if your code checkpoints and restarts. Google's Spot VM overview says Compute Engine can preempt Spot VMs at any time, recommends them for fault-tolerant workloads, and gives a best-effort shutdown period of up to 30 seconds. Compute Engine does not manage your checkpoint.

For a 10-hour fine-tune, one lost run without a checkpoint is 10 hours of GPU time and $527 of Spot spend wasted. The safe pattern is to save model, optimizer and step every N minutes and again when you receive the termination signal. On Nodus, you write to /nodus/state and Nodus restores it on a replacement attempt. On GCP, you write to Cloud Storage and write the code that detects and reloads the checkpoint.

Spot discounts also do not combine with commitments. Google's pricing page says Spot prices are final: "Spot prices apply to all Spot VMs (and preemptible VMs), so Spot VMs (and preemptible VMs) cannot receive CUDs or SUDs."

Where is Google Cloud the better choice?

Google Cloud is the better choice in four situations, and we would tell a friend to stay in each of them.

You already have Google credits or commitments

Startup credits, enterprise agreements and committed-use discounts change the math. On Google's pricing page, a one-year resource commitment on A3 High is $61.38/hr and a three-year commitment is $38.86/hr, or $4.86 per GPU-hour. That is still above the Nodus rate but locked in for three years. Credits are also the case where using both wins: connect the project and Nodus spends them first.

Your data lives in Google Cloud Storage

Training next to your data avoids moving it. Sending datasets out of Google Cloud incurs internet egress charges (network pricing), which matter if you retrain often on large data.

You want TPUs

TPUs are only available on Google Cloud, and Nodus does not offer them. If your stack is built on JAX and TPUs, this comparison does not apply.

Compliance requires Google Cloud

Some teams must keep data and compute inside a specific cloud for contractual or regulatory reasons. Then the per-hour price is secondary.

If none of these apply, Nodus is the cheaper choice for GPU training and handles recovery when interruptible capacity is reclaimed. For comparisons with other providers, see Nodus vs Lambda and RunPod alternatives for AI training.

How do you move a training job from Google Cloud to Nodus?

For a simple PyTorch job, install the CLI, add a budget and run the same command:

pip install nodus-compute
nodus login

# 8 x H100, a $40 spending cap, detached
nodus run --gpu H100:8 --max-cost 40 -d -- torchrun train.py

The --gpu H100:8 form and the --max-cost cap come from the nodus run reference; add --interruptible to allow interruptible capacity.

Checkpoint code that saves to a local directory works on Nodus if you point it at /nodus/state. Nodus snapshots that directory periodically and restores it on a replacement attempt. The full pattern is in the checkpoint guide.

If you run Vertex AI custom training today, keep your training script and replace the Cloud Storage checkpoint path with /nodus/state.

FAQ

Does work in my connected GCP project show up on my Google bill?

Yes. Machines Nodus starts in your project are billed by Google to that project, against its credits, and Nodus tracks them against the monthly credit limit you set. That limit is spent at Nodus's list estimate, which can differ from Google's bill, so the docs advise setting it a little below the credit you hold.

Which regions does Nodus use in a connected GCP project?

Nodus launches in us-central1, us-east1, us-east4 and us-west1. If a GPU quota is used up, it skips that GPU model in that region for an hour and runs the work elsewhere.

What happens if my GCP Spot VM is preempted mid-training?

On Google Cloud alone, the VM shuts down after a shutdown period of up to 30 seconds. You restart it or launch a new one and reload your checkpoint from Cloud Storage yourself. On Nodus with interruptible capacity, the platform launches a replacement machine and restores your /nodus/state directory automatically.

Is Google Cloud's DWS Flex-start cheaper than Nodus?

DWS Flex-start lists an A3 High at $38.32/hr ($4.79/GPU-hr), lower than on-demand but still above the Nodus rate of $2.60/GPU-hr. Flex-start works through resize requests that wait for capacity rather than starting at once (Flex-start docs).

How do Nodus and Vertex AI compare for managed training?

Vertex AI keeps training inside Google Cloud, next to your data and IAM. Nodus restores /nodus/state on a replacement attempt for you, and can use your connected GCP project as its first place to run.

Originally published on the Nodus blog.

Can you use serverless GPUs for long training runs?

Viswaretas Kotra — Thu, 29 Oct 2026 18:17:21 +0000

You can, up to a point: Modal caps a single function execution at 24 hours, RunPod Serverless caps a job at 7 days, and on both a preempted or failed run starts over unless your own code saved a checkpoint. Nodus runs long training as a Job or TrainingJob with an optional time limit, saves your state directory automatically, and resumes from the latest checkpoint when capacity is reclaimed. Our rule: serverless is fine for training that finishes well inside one execution and is cheap to repeat; past that, use a platform that owns the checkpoint.

How long can a serverless GPU function run?

Each serverless platform puts a hard ceiling on one execution, and the defaults are much lower than the maximums.

Platform	Default limit per execution	Maximum	What happens at the limit
Modal	5 minutes	24 hours	`FunctionTimeoutError` after any retries
RunPod Serverless	10 minutes	7 days	The job fails and the worker stops
Nodus Job or TrainingJob	No limit unless you set `timeout`	You can raise or remove it on a running Job	`DeadlineExceeded`; time spent suspended does not count

Sources: Modal timeouts, RunPod endpoint settings, Nodus Jobs guide. Checked October 29, 2026.

So a 30-hour fine-tune does not fit in one Modal call at all. It fits in one RunPod Serverless job only if you raise the execution timeout from its 10-minute default, which is easy to forget until the first job dies at minute ten.

What happens when a long serverless run is interrupted?

The platform restarts the work, but your progress only survives if your code wrote it somewhere first. Modal's preemption guide says all Modal Functions are preemptible by default, that a preempted Function is restarted on the same input, and that training Functions should checkpoint because the chance of interruption grows with run time. GPU Functions cannot opt out of preemption.

Here is what that costs. Take a 30-hour run on one H100 that is interrupted at hour 20:

Without checkpoints, the restart begins at step 0, so you pay for 20 + 30 = 50 GPU-hours.
With a checkpoint every hour, you lose at most one hour, so you pay for at most 31 GPU-hours.

At Modal's H100 rate of $0.001097 per second (about $3.95 per hour), those 20 lost hours are $78.98 of repeated work. The fix is not exotic, but it is code you have to write, test and keep correct: save model, optimizer and step state, then detect a restart and load it.

One interruption without checkpoints makes a 30-hour run bill 50 GPU-hours.

How does Nodus keep a multi-day training run going?

Nodus treats checkpointing as the platform's job. Your program writes what it needs to continue under /nodus/state, and Nodus copies that directory while the run is going. With the default interval: auto, the checkpoint guide says the cadence follows how often the capacity is interrupted and how long a save takes, with at least four checkpoints over the expected run. Nodus also checkpoints when the capacity gives notice that it is about to be reclaimed, then starts the next attempt with the directory restored and NODUS_RESTORED=1 set.

The managed TrainingJob runtimes do the save and restore for you, including optimizer, scheduler and RNG state. This is the SFT example from the docs:

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: support-sft
spec:
  runtime: nodus/sft
  model:
    uri: hf://Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
    secret: hf-token            # only for gated repositories
  data:
    volume: support-chats       # a Volume holding chats.jsonl
    path: chats.jsonl
    format: JSONL
    columns: {prompt: question, completion: answer}
  parameters:
    steps: 2000
    lora: {r: 16, alpha: 32}
  maxCostUSD: "20.00"

The maxCostUSD cap is the other half of running long jobs safely. When the cap is reached, the run is suspended rather than failed, and raising the cap resumes the same run from its checkpoint. For a broader look at which platforms do this for you, see Which GPU platforms checkpoint and resume training for you?

What does a 30-hour H100 fine-tune cost on each option?

At list prices, Nodus is the cheapest of the four for the same 30 GPU-hours, and it is the only one where the restart logic is not your code.

Option	H100 list rate	30 GPU-hours	Fits in one run?
Nodus Job or TrainingJob	$2.60/hr	$78.00	Yes
Modal Function	$3.95/hr ($0.001097/s)	$118.48	No, 24-hour cap
RunPod Pod (H100 SXM)	$3.99/hr	$119.70	Yes, not serverless
RunPod Serverless (H100 PRO)	$5.39/hr	$161.70	Yes, if the timeout is raised

Prices checked October 29, 2026: Nodus H100 SXM listed at $2.60/hr on the Nodus pricing page, Modal pricing (GPU only; Modal bills CPU and memory separately), RunPod pricing for Pods and Serverless.

The math is 30 times each hourly rate. On those numbers the Nodus run costs 34% less than the Modal run, before counting any repeated hours from an interruption.

Same 30 GPU-hours, list prices: the gap is the hourly rate before any lost work.

When is serverless the right place to train?

Serverless training makes sense when the run is short, cheap to repeat, or already part of a serverless app. A LoRA fine-tune of a small model that takes two hours fits inside Modal's 24-hour cap with room to spare. An interruption there costs at most two hours, so writing restart logic may not be worth it.

Use this decision rule. If the run would take less than a few hours and a full restart is cheap, train where your app already lives. If it takes longer than one execution allows, or one lost run would cost more than you would spend writing and testing checkpoint code, use a platform that checkpoints for you.

Where is Modal or RunPod the better choice?

Modal is the better choice when your training code is already a Modal app and runs fit in 24 hours. You keep one deployment model for training and inference, and its Python-first workflow avoids writing manifests. Modal also publishes a long training example that shows the checkpoint pattern if you want to stay.

RunPod Serverless is the better choice when one job can take days but you want queue-based workers that scale down when idle. Its 7-day execution limit is far longer than Modal's. RunPod Pods suit teams that want a plain GPU machine and are happy to handle restarts themselves.

Nodus is the better choice when runs are long, you want them on the cheapest capacity that will finish them, and you would rather the platform own checkpoints, recovery and the spending cap. We cover the platform-level comparison in What are the best serverless GPU platforms in 2026?.

How do you move a long training run to Nodus?

If you use one of the managed runtimes, save the manifest above as sft.yaml, check the estimate, then apply it:

$ pip install nodus-compute
$ nodus login
$ nodus apply -f sft.yaml --dry-run -o yaml     # status.estimate: cost to completion, first hold, start time
$ nodus apply -f sft.yaml
$ nodus logs job/support-sft -f

For your own training script, point its save and load paths at /nodus/state and run it as a Job with nodus run --checkpoint /nodus/state. The checkpoint guide has a complete Python example.

FAQ

Can I chain several Modal calls to train past 24 hours?

Yes. Each call loads the last checkpoint from a Volume, trains until close to the limit, saves, and the next call continues. You own the orchestration, the save format and the edge cases, such as a preemption that lands mid-save.

Does the 24-hour Modal limit include time waiting for a GPU?

No. Modal's timeout measures execution time only, not scheduling, and it applies per execution attempt when retries are configured.

How long does Nodus keep checkpoints after a run finishes?

Checkpoint settings take a retainAfterFinish value, shown as 168h in the docs example, after which only the final checkpoint is kept. nodus describe job/NAME shows the latest one.

Does a Nodus TrainingJob need my code to handle checkpoints?

Not with the managed runtimes such as nodus/sft, nodus/dpo and nodus/grpo-lora; they save full recovery state themselves. Custom code only needs to write and read its files under /nodus/state.

Originally published on the Nodus blog.

What are the best serverless GPU platforms in 2026?

Viswaretas Kotra — Thu, 29 Oct 2026 15:48:48 +0000

The best serverless GPU platform in 2026 depends on whether someone is waiting for the GPU: Modal or RunPod Serverless for endpoints a user waits on, and Nodus for training, fine-tuning and batch jobs that run for hours. Nodus lists an NVIDIA H100 at $2.60/hr on its pricing page and restores your latest checkpoint on a new machine when capacity is reclaimed, so a long job costs less and survives an interruption. Baseten, Replicate, Cerebrium and Google Cloud Run GPUs each fit a narrower case, and the decision rules in this post say which.

We build Nodus, so weigh that. Every outside number links to the company's own page, and there is a section on where each of the others is the better pick.

What makes a GPU platform serverless?

A serverless GPU platform starts your container on a GPU when work arrives, bills while it runs and lets the GPU go when things are quiet. The platforms differ in the details that decide the bill: how fast a cold worker gets ready, how long an idle worker stays billed, and how long one run may last.

Those details split the field in two. Endpoint platforms tune for start time, because a person is waiting on the first token or image. Job platforms tune for price per GPU-hour and for finishing, because a run that dies at hour eight wastes eight hours of GPU time.

How do the main serverless GPU platforms compare?

The table lists what each company publishes on its own pages. Start times are the vendor's own claims, measured in different ways, so treat them as a guide rather than a benchmark.

Platform	Built for	NVIDIA H100	NVIDIA L4	Billing	Start time the vendor states
Modal	Python functions, web endpoints, batch	$0.001097/s ($3.95/hr)	$0.000222/s ($0.80/hr)	Per second; idle containers billed for 60 s by default	Sub-second starts on large custom images (homepage)
RunPod Serverless	Container endpoints	$5.39/hr flex worker (H100 PRO)	$0.75/hr flex worker (24 GB class, includes L4)	Per second, rounded up; 5 s idle timeout by default (docs)	Under 200 ms with FlashBoot
Baseten	Production model serving and training	$0.10833/min ($6.50/hr)	$0.01414/min ($0.85/hr)	Per minute, including deploy and scaling time	"Fast cold starts", no figure
Replicate	Public models by API, custom models with Cog	$0.001525/s ($5.49/hr)	Not listed	Per second; most private models also pay for setup and idle time	Not stated
Cerebrium	LLM, voice and image apps	$0.000944/s ($3.40/hr)	$0.000222/s ($0.80/hr)	Per second; CPU and memory billed separately	Not stated
Google Cloud Run GPUs	Containers on Google Cloud	Not offered (L4 and RTX PRO 6000 only)	$0.0001867/s ($0.67/hr) (pricing)	Per second, instance-based; an L4 needs at least 4 vCPU and 16 GiB	About 5 s
Nodus	Training, fine-tuning, batch, Python Functions	$2.60/hr	$0.43/hr	Prepaid credit; each run holds funds before it starts	A cold and warm start estimate per Function, before you run

Prices checked October 29, 2026. Hourly figures in brackets convert the listed per-second or per-minute rate; Modal, Cerebrium and Cloud Run bill CPU and memory on top of the GPU, and Cloud Run's L4 rate is the non-zonal-redundancy price. RunPod's endpoint docs show lower per-second rates for some tiers; we used the rates on its pricing page.

Which serverless GPU platform should you pick for each workload?

Match the platform to who is waiting and for how long.

Someone is waiting on the response

Pick RunPod Serverless or Modal. RunPod states sub-200 ms cold starts with FlashBoot, deploys any container, and by default keeps an idle worker for only 5 seconds before it scales down. Modal fits better when your service is Python: it states sub-second starts on large custom images and bills per second, with a 60-second keep-alive that you can change.

Pick Baseten if you want a managed inference stack and per-minute billing is fine for your traffic. Pick Cloud Run if the rest of your app already lives on Google Cloud and an L4 or RTX PRO 6000 is enough. Google states instances start in about 5 seconds.

Traffic is steady all day

Skip scale-to-zero and pay for warm capacity on purpose. Serverless rates price in the freedom to drop to zero, which you never use if the GPU is busy around the clock. RunPod offers always-on active workers with discounts through sales, and a Nodus Function's min_workers setting keeps workers warm at the Function's worker rate. Compare that monthly bill with a reserved option before you commit.

The job runs for hours

Pick Nodus. For training, fine-tuning and continued pretraining, the price of a GPU-hour and what happens on an interruption matter far more than a fast first request. Your script writes its state to /nodus/state. Nodus checkpoints it at least four times over the expected run time and restores the latest copy into a new attempt if the machine is lost.

The endpoint platforms can run long jobs too, within limits they publish. Modal caps one execution at 24 hours and makes Functions preemptible by default, restarting an interrupted one on the same input. RunPod Serverless gives each endpoint a 10-minute execution timeout by default, configurable up to 7 days, and one GPU per worker unless you raise it.

A batch job where nobody waits

Embedding a corpus or scoring a dataset overnight has the same economics as training, so choose on price per GPU-hour and on resuming, not on cold starts. On Nodus, a Job with Restartable continuity keeps the progress counter you report, so a replacement attempt can skip the items already done. A Function's .map() fans one function out over thousands of inputs. Modal is a good fit too if your code already runs there: its batch processing guide scales functions to thousands of parallel containers.

You only need to call an open model

You may not need a GPU platform at all. Baseten's Model APIs, Replicate's public models and Nodus Inference (Beta), which speaks the OpenAI API format, charge per token or per prediction, so you never size a worker.

What does the same GPU work cost on each platform?

At list prices, a 10-hour job on one H100 ranges from $26.00 on Nodus to $65.00 on Baseten. We priced a LoRA fine-tune that needs 10 hours on one NVIDIA H100, which is 36,000 GPU-seconds or 600 GPU-minutes, at each platform's listed H100 rate from the table:

Nodus: 10 h × $2.60 = $26.00
Cerebrium: 36,000 s × $0.000944 = $33.98
Modal: 36,000 s × $0.001097 = $39.49
RunPod Serverless: 10 h × $5.39 = $53.90
Replicate: 36,000 s × $0.001525 = $54.90
Baseten: 600 min × $0.10833 = $65.00

The same 10 H100-hours cost 34% less on Nodus than on Modal at list rates, before Modal adds CPU and memory.

This counts the GPU line only: Modal and Cerebrium add CPU and memory on top, and cold starts and idle windows are seconds on a 10-hour job. The ratios hold per GPU-hour, so eight GPUs scale the same way.

For a small model served on an L4, the gap is smaller in dollars but similar in shape. Nodus lists the L4 at $0.43/hr, against $0.67 on Cloud Run before its required CPU and memory, $0.75 on RunPod Serverless, $0.80 on Modal and Cerebrium, and $0.85 on Baseten.

On an L4 the price spread is cents per hour, so cold-start behavior usually decides an endpoint, not the rate.

What happens to the bill when a long run is interrupted?

An interruption costs whatever work you have to redo, and that can outweigh any rate difference. Take the same 10-hour job, lose the machine at hour 8, and price every case at $2.60/hr so only the recovery changes:

No interruption: 10 GPU-hours = $26.00
Checkpoint every 30 minutes: at most 0.5 hours redone, 10.5 GPU-hours = $27.30
No checkpoint: 8 hours lost plus a 10-hour rerun, 18 GPU-hours = $46.80

Without a checkpoint, one lost machine at hour 8 nearly doubles the bill.

Any platform here can resume if your code saves to persistent storage and reloads it on start, as Modal's preemption guide recommends for training; the difference is who does the plumbing. On a reclaim notice, Nodus asks your program for an urgent checkpoint and prepares a replacement machine. The replacement starts only once the old one is provably gone, so two copies never write the same checkpoint (attempts and recovery). If the old machine comes back first, the replacement is released and you are not charged for it.

Our guide to surviving spot preemption walks through the checkpoint code.

Can Nodus serve a latency-sensitive endpoint?

Partly, and we would not pick it first for that today. Nodus Functions scale with demand: min_workers defaults to 0, so a Function scales to zero, and max_workers goes up to 1,000. An idle worker leaves after its scaledown_window, 1 minute by default and up to 20 (scaling docs). Before you deploy, .estimate() reports the expected cold and warm start bands for that Function.

What Nodus does not publish is a fixed cold-start figure, and hosted Inference is still in Beta. If first-request latency is the metric your product lives or dies on, start with RunPod Serverless or Modal, which publish start times, and test Nodus against them with your own model.

Where are Modal, RunPod and the others the better choice?

Each of them beats Nodus somewhere specific.

Modal is the better choice for Python web endpoints that need quick container starts and per-second billing, and its Starter plan includes $30 of compute credit a month.
RunPod Serverless is the better choice when cold start is everything: it states sub-200 ms starts with FlashBoot, and deploys straight from GitHub with rollbacks.
Baseten is the better choice for teams that want a managed production inference platform; its pricing page lists SOC 2 Type II and HIPAA compliance and a self-hosted option on Enterprise.
Replicate is the better choice for calling public models through an API without deploying anything, and its fast-booting fine-tunes bill only active time.
Cloud Run is the better choice if you already run on Google Cloud and an L4 or RTX PRO 6000 is enough, with no GPU reservation to manage.
Cerebrium has the lowest per-second H100 rate among the endpoint platforms in our table, though CPU and memory are billed on top.

For a deeper head-to-head on the two most common picks, see Modal vs RunPod.

How do you move a long job from an endpoint platform to Nodus?

Keep your training script and make it save to and load from /nodus/state. Then install the CLI, sign in, and run the same script with a spend cap. Add --dry-run first to see the estimate without starting paid compute. The commands are copied from the Nodus docs:

pip install nodus-compute
nodus login
nodus run --gpu H100 --image nodus/pytorch --checkpoint /nodus/state --max-cost 40 -d -- python train.py

The $40 cap leaves headroom over the $26.00 the job costs at the listed rate, and nodus describe job/NAME shows each attempt, the latest checkpoint and the cost so far.

FAQ

Is a serverless GPU cheaper than a dedicated one?

Only when the GPU would otherwise sit idle. Modal's own pricing FAQ makes the same point: per-hour serverless rates can look higher, and the savings come from spiky or unpredictable traffic. If your GPU is busy most of the day, a lower hourly rate or reserved capacity wins.

Do you pay for cold starts on serverless GPUs?

Usually, yes, for the part where your code loads. RunPod's docs bill from worker start until your handler is ready, which is mostly loading the model into GPU memory, while image pulls and cached model downloads happen unbilled. Modal bills application load time too, so smaller weights and lazy imports cut both latency and cost.

Can you fine-tune a model on a serverless GPU platform?

Yes. Nodus runs fine-tuning as a Job that resumes from its latest checkpoint, or through managed TrainingJobs (Beta) on the GPU you pick. Modal and RunPod Serverless also run training code, within the timeouts above.

What happens on Nodus if credit runs out mid-run?

Nodus is prepaid and renews a funded hold on your balance every 5 minutes while work runs. When a renewal cannot be funded, the run stops gracefully inside time already paid for, keeps what it saved, and can resume once you add credit.

Originally published on the Nodus blog.

What is the cheapest way to fine-tune Llama on an H100?

Viswaretas Kotra — Thu, 29 Oct 2026 01:40:56 +0000

The cheapest way to fine-tune Llama on one H100 is the lightest method that fits (LoRA for Llama 3.1 8B, QLoRA for a 70B), a checkpoint every few minutes and billing only while the job runs. Nodus lists the H100 SXM at $2.60/hr on its pricing page, stops billing when the job finishes and restores the latest checkpoint on a new machine if the GPU is reclaimed. In our worked example, that keeps a 50-million-token Llama 3.1 8B LoRA run at $8.35, where an idle hour or one unprotected preemption pushes the same run to $10.95 or $15.50.

We build Nodus, so weigh that. Memory and speed figures below come from Hugging Face, torchtune and Unsloth, other platforms' prices link to their own pages, and every cost shows its arithmetic.

Which fine-tuning method fits Llama on one H100?

LoRA and QLoRA fit the 8B comfortably, and only QLoRA fits a 70B. The figures are Hugging Face's own estimates from its Llama 3.1 launch post, which describes them as approximate:

Method	Llama 3.1 8B	Llama 3.1 70B	Fits one H100 80 GB?
Full fine-tuning	60 GB	500 GB	8B only, with memory-saving settings
LoRA (16-bit base)	16 GB	160 GB	8B yes, 70B no
QLoRA (4-bit base)	6 GB	48 GB	Both

On one 80 GB H100, the 70B has exactly one option: QLoRA.

The full fine-tuning figure deserves a second look. The textbook cost of mixed-precision training with the AdamW optimizer is 16 bytes per parameter, as laid out in the ZeRO paper. Llama 3.1 8B has 8,030,261,248 parameters, so that is 128 GB before activations. Hugging Face's lower 60 GB, and torchtune's 18.9 GiB on a 24 GB RTX 4090 in its README, depend on lower-precision optimizer states and other memory savers.

For the 70B, Unsloth reports that QLoRA of Llama 3.3 70B fits in 41 GB in its Llama 3.3 post. A 16-bit LoRA needs two or more H100s, which our 70B VRAM guide works through in detail.

Is QLoRA cheaper than LoRA for Llama 3.1 8B on an H100?

Usually not. QLoRA saves memory, and on an 80 GB card the 8B does not need the saving, while the 4-bit base can slow every step.

torchtune measured both on the same hardware. For Llama 3.1 8B on an RTX 4090, LoRA trained at 3,083 tokens per second and QLoRA at 2,413, so QLoRA needed 28% more GPU time (3,083 / 2,413 = 1.28). For Llama 3.2 3B on an A100, the gap was small: 8,210 against 8,035 tokens per second. These are torchtune's numbers on those GPUs, not H100 measurements.

QLoRA's slowdown ranged from 2% to 22% in torchtune's tests, and you pay for every extra second.

GPU time is the bill, so the method that fits with the most headroom and runs fastest is the cheap one. For an 8B on an H100, that is 16-bit LoRA. Measure both on 100 steps if your sequences are long, because the trade can shift.

What does a Llama fine-tune cost per finished run?

It costs the hourly rate times every hour you are billed, not only the training hours. We use one formula:

cost per finished run = rate x (training tokens / tokens per second / 3,600 + startup + idle + redone hours)

Here is a worked example, built from stated assumptions rather than a measured run. The job is a LoRA fine-tune of Llama 3.1 8B: 25 million tokens of data, 2 epochs, so 50 million training tokens.

For speed we use torchtune's 4,699 tokens per second for Llama 3.1 8B LoRA on one RTX A6000. We found no published H100 figure for this exact setup, so the A6000 number is a stand-in: replace it with the tokens per second you measure. We also assume 15 minutes of billed startup and teardown, as in our H100 and B200 cost-per-job post.

Training takes 50,000,000 / 4,699 = 10,641 seconds, or 2.96 hours. At $2.60/hr that is $7.70 of training plus $0.65 of startup, before anything goes wrong:

Scenario	Billed hours	Cost
Clean run, billing stops when the job ends	2.96 + 0.25 = 3.21	$8.35
Machine left on for 1 hour after training	3.21 + 1 = 4.21	$10.95
Preempted at 2.5 hours, no usable checkpoint	3.21 + 2.5 redone + 0.25 restart = 5.96	$15.50
Preempted at 2.5 hours, checkpoint every 10 minutes	3.21 + at most 0.17 redone + 0.25 restart = 3.63	$9.44

The rate is listed at $2.60/hr on the Nodus pricing page for the H100 SXM 80 GB. Prices checked October 28, 2026.

One preemption without a checkpoint adds $7.15 to the run; with 10-minute checkpoints it adds $1.09.

The lesson for Llama on an H100 is that the waste lines are bigger than any rate difference you can shop for. A one-hour idle tail adds $2.60, or 31% of this run. A lost run nearly doubles it.

Why are LoRA checkpoints cheap enough to save every few minutes?

Because LoRA trains a tiny fraction of the model, its recovery checkpoint is small. Llama 3.1 8B has 32 layers, a hidden size of 4,096, key and value projections of 1,024 and an MLP size of 14,336, per the model config (we read an ungated copy on Hugging Face). A rank-16 adapter on all seven linear layers adds 16 x (in + out) parameters per layer:

16 x [(4,096 + 4,096) x 2 + (4,096 + 1,024) x 2 + (4,096 + 14,336) x 3] = 1,310,720 per layer

Over 32 layers that is 41,943,040 trainable parameters, 0.52% of the model. A full fine-tune has to save optimizer state for all 8 billion parameters, so frequent checkpoints get expensive. With LoRA they barely register, which is why the 10-minute row in the table above is realistic.

Nodus resumes from full checkpoints with optimizer, scheduler and RNG state, as the training guide describes. On a reclaim notice it also asks the job for an urgent checkpoint, and a checkpoint committed during that notice counts, per the attempts and recovery docs. Our spot preemption guide covers the same pattern for your own scripts.

How do you run a Llama LoRA job on Nodus?

First, get access to the weights. Llama models on Hugging Face are gated: you accept Meta's license on the model page, such as meta-llama/Llama-3.1-8B-Instruct, then use a Hugging Face token from that account. The terms are in the Llama 3.1 Community License.

Then install the CLI, sign in and store the token as a Secret. These commands are copied from the Nodus training and secrets docs:

$ pip install nodus-compute
$ nodus login

nodus secret create hf-token --from-literal HF_TOKEN=hf_xxx

The manifest below is the LoRA example from the training guide with four changes: the Llama model pinned to its current Hugging Face commit, method, epochs, and one H100 under resources. The managed sft runtime publishes presets only for Qwen models, so a Llama run names its GPU. TrainingJobs are Beta, and we have not run this exact manifest, so start with the dry run.

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: llama-8b-lora
spec:
  runtime: nodus/sft
  model:
    uri: hf://meta-llama/Llama-3.1-8B-Instruct@0e9e39f249a16976918f6564b8830bc894c89659
    secret: hf-token            # only for gated repositories
  data:
    volume: support-chats       # a Volume holding chats.jsonl
    path: chats.jsonl
    format: JSONL
    columns: {prompt: question, completion: answer}
  parameters:
    method: LoRA
    epochs: 2
    lora: {r: 16, alpha: 32}
  resources: {gpu: {type: [h100-sxm-80g], count: 1}}
  maxCostUSD: "20.00"

$ nodus apply -f sft.yaml --dry-run -o yaml     # status.estimate: cost to completion, first hold, start time
$ nodus apply -f sft.yaml

The runtime saves a recovery checkpoint every saveSteps steps (200 by default), so set it to land roughly every 10 minutes once you know your step time. nodus describe tj/llama-8b-lora shows tokens per second while it trains, which is the number to put into the formula above. The maxCostUSD cap suspends the job at a checkpoint if spending reaches it, per the sft runtime reference and the budgets docs.

Where are Modal or RunPod the cheaper choice?

Both can beat Nodus on this run, depending on how you work.

Modal includes $30 a month of free compute on its Starter plan, and charges $0.001097 per second for an H100, or $3.9492/hr, according to Modal's pricing page. The clean run above bills 3.21 hours, which is $12.68 at that rate. That fits inside the monthly credit, so an occasional small Llama fine-tune can cost you nothing out of pocket on Modal. Modal restarts a preempted function on the same input, and you write the checkpoints yourself, per its preemption docs.

RunPod lists the H100 SXM at $2.69/hr on Community Cloud and $3.99/hr on Secure Cloud on its pricing page. If you stop the Pod the moment training ends, the clean run costs $8.63 on Community Cloud, 28 cents more than our example. Pods are billed by the second until they stop, per RunPod's Pod pricing docs, so one forgotten hour makes it $11.32. RunPod also offers a form-driven Axolotl fine-tuning flow in its fine-tuning docs, and a Pod gives you a full machine to work in interactively.

Pick Modal if your runs are small and occasional, or your code already runs as Modal functions. Pick RunPod if you want a machine you drive by hand and you reliably shut it down. Pick Nodus if your runs are long enough that idle time and preemption are the real cost. Our fine-tuning platform comparison covers per-token APIs as well.

Which setup should you pick?

Use this rule, in order:

Llama 3.1 8B: 16-bit LoRA on one H100. Do not pay QLoRA's slowdown for memory you have. If you do not need an H100's speed, a 24 GB card is cheaper; our Unsloth guide shows the break-even.
A 70B Llama on one H100: QLoRA. A 16-bit LoRA needs two or more GPUs.
Full fine-tuning: only after LoRA has measurably fallen short. For comparison, the Nodus sft runtime's preset for a full fine-tune of Qwen3-8B is 8 x H100.
Every run: billing that stops when training stops, a checkpoint at least every 10 minutes, and 100 measured steps before you commit the full budget.

FAQ

Can I use a fine-tuned Llama commercially?

Yes for most companies, under the Llama 3.1 Community License. If you distribute the model or a product built on it, the license requires "Built with Llama" to be displayed and the model's name to start with "Llama". Companies above 700 million monthly active users need a separate license from Meta.

How long a context can I fine-tune a 70B Llama on with one 80 GB GPU?

Unsloth reports up to 89,000 tokens of context for Llama 3.3 70B QLoRA on an 80 GB GPU with its long-context settings, against 6,900 with Hugging Face plus Flash Attention 2. Context length drives activation memory, so it is often what pushes a 70B run off a single GPU.

Should I pick a B200 instead of an H100 for Llama fine-tuning?

Only if it finishes faster than its price ratio. The B200 is listed at $5.50/hr on the Nodus pricing page, 2.1 times the H100's $2.60, so it has to train more than 2.1 times faster to cost less per finished run.

Does Nodus pick the GPU if I leave `resources` out?

For Llama, the sft runtime falls back to its default: one RTX 4090, L40S, RTX A6000 or A100 80 GB. That suits QLoRA of an 8B but not a 70B, so name the H100 explicitly when you need 80 GB.

Originally published on the Nodus blog.

What are the best Together AI alternatives for fine-tuning?

Viswaretas Kotra — Wed, 28 Oct 2026 23:08:00 +0000

For teams leaving Together AI's fine-tuning API, the strongest alternatives are Nodus for per-GPU-hour billing with managed recipes, Fireworks AI for another per-token API, and Modal or RunPod for running your own trainer. Nodus lists an NVIDIA H100 at $2.60/hr, runs SFT, DPO and GRPO recipes on a GPU you pick, and resumes a run from its latest checkpoint if a machine is lost. Together AI bills per training token instead, so the cheaper option depends on how many GPU-hours your job really takes.

This post prices one realistic LoRA job both ways, shows the arithmetic once, and links every outside number to the company's own page. We make Nodus, so we also say plainly where Together AI is the better choice.

Who are Together AI's main competitors for fine-tuning?

They fall into two groups: managed APIs that charge per token, and platforms that charge for GPU time. Together AI and Fireworks AI are in the first group. Nodus, Modal and RunPod are in the second.

Platform	What you run	How fine-tuning is billed	Example listed price	Serving the result
Together AI	Managed fine-tuning API, LoRA or full	Per 1M training and evaluation tokens, minimum charge per job	$0.34 per 1M tokens, LoRA SFT on Llama 3.1 8B	Dedicated endpoint billed per minute, or download the weights
Nodus	Managed TrainingJob recipes (SFT, DPO, ORPO, KTO, distillation, GRPO), or your own script	Per GPU-hour from prepaid credits, with a dollar cap per job	$2.60/hr for an H100 SXM, $0.793/hr for an L40S	Download outputs with `nodus cp`
Fireworks AI	Managed fine-tuning API, LoRA or full parameter	Per 1M training tokens, by model size	$0.50 per 1M tokens, LoRA SFT up to 16B	On-demand (dedicated) deployment
Modal	Your own trainer as Python functions	Per GPU-second	$0.001097/s for an H100	Your own code
RunPod	An Axolotl fine-tuning pod, or a pod you control	Per hour, billed by the second	$3.99/hr for an H100 SXM	Your own code

Prices checked October 28, 2026 on Together AI pricing, the Nodus pricing page, Fireworks AI pricing, Modal pricing and RunPod pricing. Serving details come from Together AI's deployment docs, Fireworks AI's SFT guide, the Nodus training guide and RunPod's fine-tuning docs.

If you came here from a broader search, our guide to the best platform to fine-tune open-source LLMs compares more options. This post is for teams already on Together AI who want to know what switching would cost.

How does Together AI bill a fine-tuning job?

Together AI charges for every token the job processes, including repeats. Its fine-tuning pricing docs give the formula:

total tokens = (epochs × training dataset tokens) + (evaluations × validation dataset tokens)

The rate per 1M tokens depends on the model size, the method (SFT or DPO) and whether you train LoRA or full weights. Each job pays at least a minimum charge, $4.00 for most models on the pricing page. A fine-tuned model served on a dedicated endpoint is billed per minute on top of that, until you stop the endpoint.

One detail catches people out. The same docs say that if you turn packing off, training tokens are counted as dataset length × max_seq_length, not the tokens you actually have. A file of short chat turns can then bill as if every row were full length. Leave packing on unless you have a reason not to.

What does the same fine-tuning job cost per token and per GPU-hour?

Take a typical support-bot job: LoRA SFT on Llama 3.1 8B, a 20M-token training set, 3 epochs, and a 0.5M-token validation set evaluated 3 times. The sizes are our assumption; swap in your own.

Together AI counts (3 × 20M) + (3 × 0.5M) = 61.5M tokens. At $0.34 per 1M for LoRA, that is 61.5 × $0.34 = $20.91. The same tokens as a full fine-tune, at $0.38 per 1M, cost $23.37.

An hourly platform has no token meter, so the useful question is how many GPU-hours $20.91 buys. If the job finishes inside that time, hourly is cheaper. We divided $20.91 by each listed rate.

GPU and platform	Listed rate	GPU-hours for $20.91
L40S, Nodus	$0.793/hr	26.4 h
A100 80GB, Nodus	$1.09/hr	19.2 h
A100 80GB, Modal	$0.000694/s, or $2.4984/hr	8.4 h
H100 SXM, Nodus	$2.60/hr	8.0 h
H100, Modal	$0.001097/s, or $3.9492/hr	5.3 h

Rates from the Nodus pricing page and Modal pricing, checked October 28, 2026. These are GPU line items only; storage and any separate CPU or memory charges are left out.

The same $20.91 buys 26.4 hours of an L40S on Nodus, so a slower GPU still has a lot of room to finish first.

The L40S row matters because an 8B LoRA does not need an H100. The Nodus sft runtime presets put a QLoRA run on Qwen3-8B on one L40S, RTX A6000 or A100 80GB. A cheaper card with more hours to spare is often the better bet than the fastest one.

Model size moves both sides. The same 61.5M tokens on Llama 3.3 70B cost 61.5 × $2.03 = $124.85 as a LoRA job on Together AI. That buys 48.0 H100-hours on Nodus, or 8 H100s for 6 hours.

How do you tell which side of the line your job is on?

Measure, then multiply. Run a short trial on the GPU you plan to use, read its training speed, and work out how long the full job would take.

On Nodus, nodus describe tj/<name> shows the step, loss and tokens per second while the job runs, per the training guide. Then:

hours = total tokens ÷ (tokens per second × 3,600)

Multiply the hours by the hourly rate and compare it with total tokens × the Together AI rate. For the H100 above, the job has to average about 2,124 tokens per second to finish 61.5M tokens inside 8.0 hours. Our break-even table gives the same threshold for other GPUs.

The rule we use: if your trial says the job finishes inside the GPU-hours its token bill would buy, move it to an hourly platform. If it is close, stay where you are, because a migration costs engineering time too.

Is Fireworks AI cheaper than Together AI for fine-tuning?

At the list prices we checked, no. Fireworks AI is the closest like-for-like alternative, since it also charges per training token, but its rates were higher for every model size we compared.

Model size	Together AI, LoRA SFT	Fireworks AI, LoRA SFT
Llama 3.1 8B (Fireworks: up to 16B)	$0.34 per 1M	$0.50 per 1M
Qwen3.5 27B (Fireworks: 16.1B to 80B)	$1.05 per 1M	$3.00 per 1M
Llama 3.3 70B (Fireworks: 16.1B to 80B)	$2.03 per 1M	$3.00 per 1M

Prices from Together AI pricing and Fireworks AI pricing, checked October 28, 2026. Fireworks AI prices by size bracket, so its bracket rate is shown against each Together AI model.

Per-token rates climb with model size on both APIs, which is where an hourly GPU tends to pull ahead.

Our 61.5M-token job would cost 61.5 × $0.50 = $30.75 as a LoRA job on Fireworks AI. Fireworks AI does have reasons to pick it anyway. Its pricing page says fine-tuned models are served for the same price as base models, and it lists a Serverless Training API for LoRA on its launch models with no provisioning and no idle cost.

What happens when a fine-tuning run fails or is cut short?

Together AI is generous here, and it is worth knowing before you switch. Its pricing docs say a job that fails is fully refunded, and a job you cancel or stop early pays only for completed steps.

On an hourly platform you pay for the GPU time used, so what matters is how much work a failure throws away. On Nodus, the managed runtimes write full checkpoints with optimizer, scheduler and RNG state, and a new attempt on another machine starts from the latest one. Each TrainingJob also has a maxCostUSD cap. When the cap is reached the run is suspended rather than failed, and raising the cap resumes it from its checkpoint.

Modal's preemption guide says a preempted function restarts on the same input and recommends checkpoints for long training. With Modal or a RunPod pod, writing and loading those checkpoints is your code's job.

Where is Together AI the better choice?

Stay on Together AI if any of these describe you. They are real advantages, not consolation prizes.

You fine-tune very large models. Together AI's pricing page lists fine-tuning for DeepSeek-V3.1, Kimi K2.6 and GLM-5.2, and you never size a cluster. On Nodus, a model without a runtime preset gets the default single-GPU resources unless you request more.
You want full fine-tuning of a small model. Together AI charges $0.38 per 1M tokens for full SFT on Llama 3.1 8B, barely above LoRA. The Nodus preset for a full Qwen3-8B run is 8 H100s, which costs 8 × $2.60 = $20.80 per hour.
You serve the model on Together AI. Training, hosting and inference sit in one product, and its deployment docs also let you download merged weights or the adapter with tg fine-tuning download.
Your jobs are small and occasional. The token formula is easy to predict before you start, and a failed job costs nothing.

Together AI also sells GPU time directly. Its post-training docs describe a product billed per GPU-minute for custom loops and GRPO, so a Together AI team that wants hourly billing has an in-house option to price as well.

How do you move a Together AI fine-tuning job to Nodus?

If your Together AI file uses the instruction format, one prompt and one completion per line as described in its data preparation docs, it maps directly onto the Nodus sft runtime. Install the CLI, sign in, and upload the file to a Volume. The commands come from the training guide and the Volumes guide.

$ pip install nodus-compute
$ nodus login
$ nodus create volume support-chats --size 100Gi
$ nodus volume put support-chats ./chats.jsonl /chats.jsonl

Then describe the job. This is the training guide's LoRA example with the column names set to Together AI's field names:

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: support-sft
spec:
  runtime: nodus/sft
  model:
    uri: hf://Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
  data:
    volume: support-chats
    path: chats.jsonl
    format: JSONL
    columns: {prompt: prompt, completion: completion}
  parameters:
    steps: 2000
    lora: {r: 16, alpha: 32}
  maxCostUSD: "20.00"

$ nodus apply -f sft.yaml --dry-run -o yaml
$ nodus apply -f sft.yaml

The dry run prices the job before anything runs, and maxCostUSD caps it at $20. Pin your own base model to a 40-character Hugging Face commit, as the guide recommends. TrainingJobs are in Beta, so fields can change before v1.

Teams running their own trainer instead of a managed recipe should read our Modal alternatives for training and RunPod alternatives posts.

FAQ

Can I connect my Together AI account to Nodus?

No. The Nodus cloud accounts guide lists the accounts you can connect, such as AWS, GCP, Azure, Lambda, Nebius and RunPod, and Together AI is not among them. Move the dataset and run the job on Nodus instead.

Does Together AI count my validation set in the bill?

Yes. Validation tokens are multiplied by the number of evaluations and added to the training tokens. Evaluating a large validation set often can add noticeably to a small job.

Can I fine-tune with Unsloth instead of a managed recipe?

Yes, as your own script on an hourly GPU. Our Unsloth on a cloud GPU guide walks through it on Nodus with checkpoints that survive a lost machine.

Can a coding agent start the fine-tune for me?

On Nodus, nodus mcp install claude adds the Nodus MCP server to Claude Code, and any TrainingJob the agent launches is still bound by its maxCostUSD cap.

Originally published on the Nodus blog.

What infrastructure do background coding agents need?

Viswaretas Kotra — Wed, 28 Oct 2026 20:38:51 +0000

A background coding agent needs seven pieces of infrastructure: an isolated sandbox per task, scoped access to the repo and secrets, compute that can run for hours with GPUs when the task needs them, a spend cap the agent cannot raise, logs and status it can poll, checkpoint and resume, and automatic cleanup. Nodus covers the compute side of that list through one MCP server: CPU Sandboxes, GPU Jobs that resume from checkpoints, Budgets that block new work, and logs the agent can read. Your git host and the agent loop itself come from elsewhere.

Most background agent guides stop at the sandbox, where the agent edits code and runs tests. The expensive part starts when the agent launches something long, like a fine-tune that runs overnight while nobody watches.

What are the seven components, and which ones does Nodus cover?

Nodus covers all seven at least in part, and the gaps are specific: no GPU Sandboxes, no git hosting and no loop for your coding agent. We built this map from the current Nodus docs and checked every row against the linked page.

Component	Why the agent needs it	What Nodus provides today
Isolated sandbox per task	Runs model-written code away from your laptop and other tasks	Sandboxes: CPU only, no outbound network by default, `maxCostUSD`, idle stop
Repo and secrets access	Clones private code and calls APIs without pasting keys	Secrets redacted from logs; a GitHub Connection clones one repo with a short-lived read-only token
Long-running compute and GPUs	Training, evals and builds that outlive a chat session	Jobs on a named GPU or a minimum GPU memory, with a `timeout`
A cap the agent can't raise	Stops a loop or a wrong GPU count before it costs a weekend	Budgets with `action: Block`, plus `maxCostUSD` per run
Status and logs it can poll	Lets the agent check progress instead of guessing	MCP `wait`, `logs`, `describe` and `events`; webhooks such as `job.succeeded`
Checkpoint and resume	A lost machine or a cap costs minutes, not the whole run	`recovery.continuity: Checkpointed` restores `/nodus/state` on new capacity
Cleanup	Nothing forgotten keeps billing	Sandbox `maxLifetime` and `onIdle: Delete`; Job `ttlSecondsAfterFinished`

Docs checked October 28, 2026.

GPU Sandboxes are not available yet: the Sandboxes page says a spec with resources.gpu is refused. So the agent edits and tests in a CPU Sandbox, and anything that needs a GPU goes out as a Job.

Why does every task need its own sandbox?

A separate sandbox per task keeps one agent's mistakes out of everything else. Model-written code can delete files, fork processes or try to reach the network, and you want that contained.

On Nodus, each Sandbox runs on CPU machines Nodus operates, in its own network namespace with a default-deny firewall, its own cgroup limits and a non-root user, per the isolation page. Egress is Deny unless you choose AllowList or Open. Creating a Sandbox with the same name and spec returns the existing one, which is how an agent reconnects after its own restart.

The same page is clear about the limit: anything you put in a Sandbox is readable by the code running there. Give it only the credentials its task needs, and keep egress closed unless the work needs the network.

How should the agent get repo and secret access?

Give the agent short-lived, narrow credentials, never your personal token. The GitHub side of this is well understood: GitHub's own docs prefer GitHub Apps because they use fine-grained permissions and short-lived tokens.

Nodus follows that pattern for reads. A GitHub Connection installs the Nodus GitHub App, and a Sandbox clones a pinned commit with a token that can only read that one repository and never appears in the Sandbox's files or environment. Secrets are stored once, never returned by the API, and redacted from logs. A command that pastes a secret's literal value into env is refused with SecretValueInEnv.

What Nodus does not do is host your code or open pull requests. Pushing a branch and opening the PR stays with your git host and whatever identity your agent uses there.

Where does the long-running GPU work run?

Long GPU work runs as a Job, not inside the agent's session. A Job takes an image, a command, a GPU, a timeout and a maxCostUSD, then runs to completion while the agent does something else.

The agent creates it through the Nodus MCP server. The first apply call returns a dry run with the estimated cost and an etag, and nothing is created until a second call with confirmed: true and the same etag, as the MCP guide describes. Connecting Claude Code to the hosted server is one command from the connect page:

claude mcp add --scope user --transport http nodus https://api-next.nodus-compute.ai/mcp

Then run /mcp in Claude Code, choose nodus and sign in. The MCP setup post covers Codex and Cursor too.

How much should the agent be allowed to spend overnight?

Set the per-run cap at the listed rate times GPUs times expected hours, plus about 20% headroom, and put a blocking Budget around all agent work. This is the rule we use, and here is the math for one run.

Say the agent launches a fine-tune at 6 pm that should take 8 hours on one NVIDIA H100. The H100 is listed at $2.60/hr on the Nodus pricing page, so the expected cost is 8 x $2.60 = $20.80. With 20% headroom, 8 x $2.60 x 1.2 = $24.96, so the cap is $25. That cap allows about 9.6 hours of H100 time (9.6 x $2.60 = $24.96).

apiVersion: nodus.dev/v1
kind: Job
metadata:
  name: overnight-finetune
spec:
  image: nodus/pytorch
  command: [python, train.py]
  resources:
    gpu: H100
  timeout: 10h
  maxCostUSD: "25.00"
  recovery:
    continuity: Checkpointed

Now suppose the script hangs and nothing stops it. By 6 pm the next day, 24 hours in, it has cost 24 x $2.60 = $62.40. Left until 9 am Monday, a Friday evening launch runs 63 hours and costs 63 x $2.60 = $163.80. With the cap, it checkpoints and suspends with reason MaxCostReached at $25.

A hung overnight job costs the cap, not the weekend.

The timeout: 10h is a second guard: 10 x $2.60 = $26, so the $25 cap fires first. Whichever limit is reached first stops the run.

The cap is not the ceiling the agent can't move, though. The Budgets docs say anyone can raise a run's maxCostUSD at any time, but not lower it, so an agent with write access to Jobs can raise its own cap. The hard ceiling is a Budget with action: Block, created by you, with the agent's MCP grant leaving out the scope to change Budgets. A monthly $500 Budget over everything the agent launches allows 24 such runs (24 x $20.80 = $499.20) before new work is refused with 402 BudgetExceeded.

apiVersion: nodus.dev/v1
kind: Budget
metadata:
  name: agent-monthly
spec:
  limitUSD: "500.00"
  period: Monthly
  scope:
    selector:
      matchLabels: {nodus.dev/launched-by: mcp}
  action: Block
  thresholds: [50, 80, 100]

Everything created through the MCP apply flow carries the label nodus.dev/launched-by: mcp, so this Budget covers agent work and nothing else. The spending limits post walks through the grant and key setup step by step.

How does the cap change with the GPU the agent picks?

The same $25 buys very different amounts of time depending on the GPU, so derive the cap from the GPU, not from habit. Here is what one 8-hour run costs and how long a $25 cap lasts at each listed rate.

GPU	Listed rate	8-hour run	Hours a $25 cap buys
NVIDIA L4	$0.43/hr	$3.44	58.1
NVIDIA A100 80 GB	$1.09/hr	$8.72	22.9
NVIDIA H100	$2.60/hr	$20.80	9.6
NVIDIA H200	$3.593/hr	$28.74	7.0
NVIDIA B200	$5.50/hr	$44.00	4.5

Prices checked October 28, 2026 on the Nodus pricing page. Hours are $25 divided by the hourly rate.

A $25 cap covers more than two days on an L4 and under five hours on a B200.

If an agent asks for an H200 or a B200 for the 8-hour job, a $25 cap suspends it before it finishes. That is the point of computing the cap from the request: a mismatch shows up in the estimate, before any money is spent.

How does the agent know what happened while it was away?

The agent should poll status and read logs, not trust its memory of what it launched. Through MCP, wait blocks up to 60 seconds for a state, logs reads up to 2,000 lines, and describe and events show why something stopped. Logs are kept after the machine is gone, per the logs guide.

For a person, or an agent loop that sleeps between turns, webhooks are cheaper than polling. Events include job.succeeded, job.failed, job.restored and job.max_cost_reached. One caution from the MCP guide applies here: log output comes back marked as untrusted, and the agent must treat it as data, not instructions.

What happens when a machine is lost or the cap is hit?

With checkpointing on, both cost minutes instead of the whole run. Your program writes its state to /nodus/state (the path is also in NODUS_STATE_DIR), and Nodus saves and restores that directory when capacity is reclaimed or the run moves. A Job that hits its cap also checkpoints, and raising maxCostUSD resumes it from there.

This is what lets an agent schedule GPU work on cheaper interruptible capacity without babysitting it. The checkpointing post shows what to save and how often.

What does Nodus not cover?

Nodus does not run your coding agent's loop, host your code or review your pull requests. Those come from other tools.

The agent loop: Claude Code, Codex Cloud, where each task gets its own workspace and keeps going while your computer sleeps, or Cursor Cloud Agents, which run in isolated cloud VMs and support MCP servers.
Git hosting and pull requests: your git host, ideally through an app with fine-grained, short-lived tokens.
GPU inside the sandbox: not available on Nodus yet; send GPU work out as a Job.

Nodus also has Agents, in Beta, where each run works in its own sandbox and survives restarts by replaying recorded steps. It is separate from connecting an outside coding agent over MCP. If you want the agent to go from a prompt to a trained model, the Claude Code training walkthrough shows the full loop.

FAQ

Can the agent create GPU Sandboxes on Nodus?

Not yet. The Sandboxes docs say a spec with resources.gpu is refused, so run tests in a CPU Sandbox and submit GPU work as a Job with its own cap.

Can an agent lower or remove a cap to save money?

No. A run's maxCostUSD can be raised but not lowered, and on a Sandbox a lower cap or no cap counts as a different spec. To stop spend early, suspend or cancel the run.

What happens to agent work when the Budget runs out?

New work in scope is refused with 402 BudgetExceeded. Running work in scope checkpoints and stops gracefully, then resumes when you raise the limit or the next month starts.

How do I disconnect an agent that misbehaves?

Delete its grant with nodus delete oauthgrant <name> or in Console, Settings, Connected agents. The client stops working within 30 seconds, even with a token it already holds.

Do finished Jobs keep costing anything?

Compute billing stops when a Job finishes. Jobs from nodus run are deleted after 30 days by default, and deleting a Job deletes its outputs, so download results first.

Originally published on the Nodus blog.

What is the best platform to fine-tune open-source LLMs?

Viswaretas Kotra — Wed, 28 Oct 2026 18:08:31 +0000

The best platform to fine-tune an open-source LLM is usually the one that charges least per finished run. Of the hourly platforms we priced, Nodus lists the lowest NVIDIA H100 rate at $2.60/hr and resumes a run from its latest checkpoint if a machine is lost. Together AI is the better pick if you would rather pay per training token and never choose a GPU, and Modal, RunPod or AWS SageMaker suit teams that already run their own trainer there.

No single platform wins every job, so this guide gives you rules instead of a ranking. Every fact about another company links to its own pricing page or docs, and since we build Nodus, every cost below shows its arithmetic so you can check us.

Which fine-tuning platform should you pick?

Pick by how much of the training stack you want to own, then by price. These are the rules we would give a friend.

Choose Nodus if you want managed recipes (SFT with full, LoRA or QLoRA, plus DPO, ORPO, KTO, distillation and GRPO) on a GPU you can choose, a price estimate before anything runs, a hard dollar cap per job and automatic resume after a lost machine.
Choose Together AI if you want a fine-tuning API priced per token, your base model is in its catalog or on the Hugging Face Hub, and you plan to serve the result on its endpoints.
Choose Modal if your training code already runs as Modal functions and you are happy to write your own checkpoint and resume logic.
Choose RunPod if you want a console form that launches an Axolotl fine-tuning pod, or a GPU pod you control end to end.
Choose AWS SageMaker if your data, credits and compliance reviews already live in AWS, and an 8-GPU instance is the size of job you run.

Hugging Face AutoTrain is no longer a candidate for new work. Its docs say the project is no longer maintained and bugs will not be fixed; our AutoTrain alternatives post covers what to move to.

How do Nodus, Together AI, Modal, RunPod and SageMaker compare?

They split into a managed API that prices tokens and platforms that price GPU time. That split decides most of what follows.

Platform	What you run	How you pay	Listed H100 price	If the GPU is lost
Nodus	Managed TrainingJob recipes, or your own script as a Job	Prepaid credits, hourly price per GPU	$2.60/hr (H100 SXM)	Resumes from the latest full checkpoint (optimizer, scheduler and RNG state)
Together AI	Fine-tuning API, LoRA or full	Per 1M training tokens, minimum charge per job	Not applicable	Not stated on the pricing page
Modal	Your own trainer as Python functions	Per second	$0.001097/s, or $3.9492/hr	Restarts the function on the same input; you write the checkpoints
RunPod	Axolotl fine-tuning pod, or your own pod	Per second	$2.69/hr Community Cloud, $3.99/hr Secure Cloud (H100 SXM)	Not stated in its fine-tuning docs
AWS SageMaker	Training jobs with your own container or script	Per instance-hour	$63.296/hr for ml.p5.48xlarge (8 x H100)	With managed spot training and checkpoints, copies them back from S3 and resumes

Prices checked October 28, 2026 on the Nodus pricing page, Together AI pricing, Modal pricing, RunPod pricing and the AWS price list for SageMaker (US East, N. Virginia). Behavior on a lost GPU comes from the Nodus training guide, Modal's preemption docs, RunPod's fine-tuning docs and SageMaker's managed spot training docs.

What does a full fine-tune on 8 H100s cost on each platform?

At listed rates, the same 80 H100-hours cost $208.00 on Nodus and $632.96 on SageMaker, with RunPod and Modal in between. We picked 8 x H100 because that is the preset the Nodus sft runtime uses for a full fine-tune of Qwen3-8B, and we assumed a 10-hour run. The duration is our assumption, not a measurement; scale it to your own job.

Platform	Listed rate	8 GPUs x 10 hours
Nodus, H100 SXM	$2.60 per GPU-hour	80 x $2.60 = $208.00
RunPod Community Cloud, H100 SXM	$2.69 per GPU-hour	80 x $2.69 = $215.20
Modal, H100	$0.001097 per GPU-second	80 x 3,600 x $0.001097 = $315.94
RunPod Secure Cloud, H100 SXM	$3.99 per GPU-hour	80 x $3.99 = $319.20
AWS SageMaker, ml.p5.48xlarge	$63.296 per instance-hour	10 x $63.296 = $632.96

These are GPU line items only. Storage, egress and any separate CPU or memory charges are left out for every platform, and SageMaker's figure is the whole 8-GPU instance.

At list prices, the same 80 H100-hours cost about a third as much on Nodus as on a SageMaker training instance.

SageMaker's own managed spot training docs say spot can cut that cost by up to 90%. The same page says managed spot never falls back to on-demand: if spot capacity is unavailable, the job waits until its maximum wait time and then stops.

Is paying per token cheaper than paying per GPU-hour for a LoRA run?

The answer turns on one number you can measure: training tokens per second. Below the break-even speed, Together AI's per-token price is cheaper; above it, an hourly GPU is.

Here is the rule we worked out. Together AI lists LoRA fine-tuning of Llama 3.1 8B and Qwen3.5 9B at $0.34 per 1M training tokens, with a $4.00 minimum per job, on its pricing page. An H100 on Nodus is $2.60/hr. Dividing one by the other gives the break-even:

$2.60 / $0.34 per 1M tokens = 7.65M tokens per hour, or 7,647,059 / 3,600 = 2,124 tokens per second.

If your LoRA run trains faster than about 2,124 tokens per second on one H100, the Nodus hour costs less than the same tokens at Together AI. The same division for the other hourly rates in this post gives the bars below.

GPU and platform	Listed hourly rate	Break-even vs $0.34 per 1M tokens
RTX 4090, Nodus	$0.29	237 tokens/s
L40S, Nodus	$0.793	648 tokens/s
H100, Nodus	$2.60	2,124 tokens/s
H100 SXM, RunPod Community Cloud	$2.69	2,198 tokens/s
H100, Modal	$3.9492	3,226 tokens/s

Nodus rates are from the pricing page, checked October 28, 2026, alongside the RunPod and Modal pages linked above.

A cheaper GPU lowers the bar: a small LoRA job only has to beat 237 tokens per second on an RTX 4090.

You do not have to guess your speed. A short Nodus run reports it: nodus describe tj/<name> shows the step, loss and tokens per second, per the training guide. Run a few hundred steps, read the number, and compare it with the break-even for the GPU you used. Remember that Together AI also bills evaluation tokens, and small jobs pay at least the minimum charge.

Do you need automatic checkpoints and resume?

For any run longer than an hour, yes, because a lost GPU otherwise means paying twice for the same steps. The platforms differ in who writes the checkpoint and who restarts from it.

On Nodus, the managed runtimes write full checkpoints with optimizer, scheduler and RNG state, and a new attempt on another machine starts from the latest one. Each TrainingJob also carries a maxCostUSD cap, and a dry run prices the job before it starts. Modal's preemption docs say a preempted function restarts on the same input, recommend checkpointing long training yourself, and note that the nonpreemptible option is not supported for GPU functions. SageMaker restores checkpoints from S3 when you turn on managed spot training and configure a checkpoint path.

Recovery is also where a fine-tune quietly gets expensive. Our fine-tuning benchmark against Modal was a pilot with one small model on one NVIDIA L4. Both platforms returned identical trained weights, Nodus charged 67% less for the successful run, and including a failed Nodus attempt the total was still 24.7% lower.

Where are Together AI, Modal, RunPod or SageMaker the better choice?

Each of them beats Nodus for a specific kind of team, and the reasons are concrete.

Together AI, when you do not want to size hardware at all. Its fine-tuning docs cover upload, training, hosting and inference in one product, and its pricing page lists fine-tuning for very large models such as DeepSeek-V3.1 and Kimi K2.6. On Nodus, a model without a runtime preset gets the default single-GPU resources unless you set larger ones yourself.
Modal, when your inference and data jobs already run as Modal functions and you want training in the same codebase. Its pricing page also offers $30 a month in free compute, which covers small experiments.
RunPod, when you want a form instead of a config file. Its fine-tuning page takes a Hugging Face model ID and dataset ID and launches an Axolotl pod. Its Community Cloud H100 is within 9 cents an hour of the Nodus rate.
AWS SageMaker, when the deciding factor is AWS itself: existing credits, data that must stay in your account, or a security review that already approved SageMaker. Note that Nodus can also run work in your own AWS account, GCP project or Azure subscription (Beta).

Nodus has limits too. TrainingJobs are in Beta, so fields can change before v1, and runs start from the CLI or the Python SDK rather than a web form.

How do you start a fine-tune on Nodus?

Install the SDK, sign in, and call a recipe with a GPU and a dollar cap. The commands and the recipe call are copied from the training guide and the Python recipe reference.

$ pip install nodus-compute
$ nodus login

sft = finetune.sft(model="Qwen/Qwen3-1.7B", data=nodus.Volume.from_name("support-chats"),
                   lora=finetune.LoRA(r=16), max_steps=2000).run(gpu="H100", max_cost=20)

Run it after import nodus and from nodus.recipes import finetune, with your JSONL data already in a Volume named support-chats. Nothing runs until .run(), and max_cost=20 caps the job at $20. The same job as a YAML TrainingJob, with nodus apply -f sft.yaml --dry-run -o yaml to see the estimate first, is in the training guide.

FAQ

What is the best platform to train AI models from scratch?

The same price rules apply, but the deciding factor shifts to multi-node support and recovery across machines. Nodus runs pretraining with the same sft runtime using task: Pretrain, including from a fresh initialization, and the 8 x H100 cost comparison above scales linearly with the number of machines.

Is LoRA enough, or do I need full fine-tuning?

Start with LoRA. Together AI's own docs call it the right choice for most use cases because it is faster and cheaper, and say full fine-tuning can outperform it when the base model's behavior needs to shift substantially. Full fine-tuning also moves you up a GPU class: the Nodus preset for a full Qwen3-8B run is 8 x H100.

Can I keep the fine-tuned weights?

On Nodus, the adapter or full model is written as job outputs that you download with nodus cp, and each file is checked against a recorded sha256. Together AI's docs say you can serve a fine-tuned model on a dedicated endpoint or download it.

Can a coding agent launch the fine-tune for me?

Yes. nodus mcp install claude adds the Nodus MCP server to Claude Code, and any TrainingJob the agent starts is still bound by its maxCostUSD cap.

Originally published on the Nodus blog.

How do you fine-tune with Unsloth on a cloud GPU?

Viswaretas Kotra — Wed, 28 Oct 2026 15:42:14 +0000

Pick the smallest GPU that fits your model in Unsloth's VRAM table, save the trainer's checkpoints to a directory that survives a restart, and run your script as a batch job with a spending cap. Nodus runs that script on an RTX 4090 listed at $0.29/hr or an H100 at $2.60/hr, and restores the checkpoint directory on a new machine if the GPU is reclaimed. A preemption then costs you minutes of training, not the whole run.

We build Nodus, so weigh that. Unsloth facts below link to Unsloth's own docs, other platforms' prices link to their pricing pages, and every cost figure shows its math.

What does Unsloth change about fine-tuning?

Unsloth makes the same fine-tune need a much smaller GPU. Its GitHub README says it trains LLMs 2x faster with 70% less VRAM and no accuracy loss. The core package is Apache 2.0, so commercial use is fine.

The memory saving matters more than the speed for your bill. By Unsloth's own table, a 14B model needs 33 GB for 16-bit LoRA but 8.5 GB with 4-bit QLoRA. That moves it from a 48 GB L40S, listed at $0.793/hr on the Nodus pricing page, to a 24 GB RTX 4090 at $0.29/hr.

Unsloth plugs into Hugging Face TRL's SFTTrainer. That detail is what makes cloud runs easy to recover, because the Trainer already knows how to save and resume.

Which GPU do you need for your model size?

Use Unsloth's own VRAM requirements table and add headroom. Unsloth calls these numbers "the absolute minimum", and longer sequences or bigger batches need more.

Model size	QLoRA (4-bit) minimum	LoRA (16-bit) minimum	A GPU that fits QLoRA with headroom
3B	3.5 GB	8 GB	NVIDIA L4 or RTX 4090 (24 GB)
8B	6 GB	22 GB	RTX 4090 or L4 (24 GB)
14B	8.5 GB	33 GB	RTX 4090 or L4 (24 GB)
32B	26 GB	76 GB	L40S or RTX A6000 (48 GB)
70B	41 GB	164 GB	L40S (48 GB) for short sequences, A100 80 GB or H100 for room

QLoRA keeps every model up to 70B on a single GPU; 16-bit LoRA needs an 80 GB card at 32B and several cards at 70B.

Our rule: start with QLoRA on a 24 GB card for anything up to 14B. Switch to 16-bit LoRA only when you have measured a quality gap that matters to you. On Nodus you can also skip naming a GPU and set resources.gpu.minMemory in the Job spec, and Nodus picks any accelerator with at least that much memory.

Is a bigger GPU ever cheaper per finished run?

Yes, but only if it finishes proportionally faster. Divide the bigger card's hourly price by the smaller card's to get the speedup it must deliver just to break even. Here is that ratio against an RTX 4090 at Nodus list prices:

GPU	Listed on the Nodus pricing page	Speedup needed vs RTX 4090
RTX 4090 (24 GB)	$0.29/hr	1.0x
L4 (24 GB)	$0.43/hr	1.5x
L40S (48 GB)	$0.793/hr	2.7x
A100 80 GB	$1.09/hr	3.8x
H100 SXM 80 GB	$2.60/hr	9.0x

Prices checked October 28, 2026 on the Nodus pricing page.

An H100 has to train 9x faster than an RTX 4090 before it saves money on the same job.

We have not benchmarked Unsloth across these cards, so measure it yourself. Run 50 steps on two GPUs, compare seconds per step, and multiply by the hourly price. Unless the H100 comes close to 9x faster on your job, the cheaper card finishes it for less. Pay for the H100 when you need the result sooner or the model does not fit anywhere smaller.

How do you make an Unsloth script resume after a restart?

Point the Trainer's output_dir at the Nodus state directory and resume from the newest checkpoint on start. The script below is Unsloth's Llama 3.1 8B Alpaca notebook with those two changes, plus a callback that saves when Nodus asks.

from unsloth import FastLanguageModel  # import first so its patches apply
import os
from datasets import load_dataset
from transformers.trainer_utils import get_last_checkpoint
from trl import SFTConfig, SFTTrainer
import nodus

STATE = os.environ.get("NODUS_STATE_DIR", "/nodus/state")
OUTPUTS = os.environ.get("NODUS_OUTPUTS_DIR", "/nodus/outputs")

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.1-8B", max_seq_length=2048, load_in_4bit=True)
model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=16, lora_dropout=0, bias="none",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    use_gradient_checkpointing="unsloth", random_state=3407)

prompt = "### Instruction:\n{}\n\n### Input:\n{}\n\n### Response:\n{}"
def fmt(ex):
    return {"text": [prompt.format(i, x, o) + tokenizer.eos_token
                     for i, x, o in zip(ex["instruction"], ex["input"], ex["output"])]}
dataset = load_dataset("unsloth/alpaca-cleaned", split="train").map(fmt, batched=True)

trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, train_dataset=dataset,
    dataset_text_field="text", max_seq_length=2048,
    callbacks=[nodus.checkpoint.HFTrainerCallback()],
    args=SFTConfig(
        output_dir=STATE, save_steps=200, save_total_limit=2,
        per_device_train_batch_size=2, gradient_accumulation_steps=4,
        num_train_epochs=1, learning_rate=2e-4, warmup_steps=5,
        optim="adamw_8bit", logging_steps=10, seed=3407, report_to="none"))

last = get_last_checkpoint(STATE) if os.path.isdir(STATE) else None
trainer.train(resume_from_checkpoint=last)
model.save_pretrained(f"{OUTPUTS}/lora-adapter")
tokenizer.save_pretrained(f"{OUTPUTS}/lora-adapter")

Three lines do the recovery work. output_dir=STATE puts Trainer's checkpoints where Nodus snapshots them. get_last_checkpoint finds the newest checkpoint-N folder after a restore, and stays None on a first start.

HFTrainerCallback comes from the Nodus SDK and makes Trainer save at the end of the current step when Nodus asks, including when the capacity gives notice that it is about to be reclaimed. Outside Nodus the SDK calls do nothing, so the same file runs on your laptop or in Colab.

save_total_limit=2 keeps the directory small. With QLoRA each checkpoint holds the adapter and optimizer state, not a copy of the 8B base model.

How do you launch it as a Nodus Job?

Install the CLI, sign in, and run from the folder that holds train.py. These flags come from the Nodus checkpoints guide and the nodus run reference:

$ pip install nodus-compute
$ nodus login
$ nodus run --name unsloth-llama8b --gpu L4 --image nodus/pytorch \
    --checkpoint /nodus/state --interruptible \
    --max-cost 10 --timeout 24h -d \
    -- sh -c "pip install unsloth nodus-compute && python train.py"
$ nodus logs -f job/unsloth-llama8b
$ nodus cp job/unsloth-llama8b:outputs/lora-adapter/adapter_model.safetensors .

An L4 fits this 8B QLoRA run with room to spare; nodus get gpus lists the other types and their current rates, including the RTX 4090. --interruptible lets Nodus use interruptible capacity, which is exactly the case the checkpoint setup covers.

Add --dry-run first to see the cost estimate without starting anything. --max-cost 10 is a cap, not a quote: at the cap the Job saves its state and suspends until you raise it. Everything under /nodus/outputs is collected when the Job succeeds, so copy adapter_config.json the same way. Installing Unsloth at start adds install time to every attempt, including restored ones. For repeat runs, pin the versions, push your own image to a registry and pass it to --image.

What does a preemption cost with and without checkpoints?

Say a 10-hour QLoRA run on one H100 is reclaimed at hour 6. At $2.60/hr the whole run costs $26.00.

Without checkpoints you start over and pay the first 6 hours twice, an extra $15.60. With save_steps set so Trainer saves about every 30 minutes, the most you redo is 30 minutes, or $1.30. The Nodus callback usually shrinks that further, because Nodus asks for a save when the capacity gives notice. The checkpointing comparison covers which other platforms restore a run like this for you.

What does the same run cost on other platforms?

At list prices, 10 hours on one H100 SXM is cheapest on Nodus, with RunPod's Community Cloud close behind:

Platform	H100 SXM hourly rate	10 hours
Nodus	$2.60/hr	$26.00
RunPod Community Cloud	$2.69/hr	$26.90
Modal	$3.949/hr	$39.49
RunPod Secure Cloud	$3.99/hr	$39.90

Prices checked October 28, 2026: Nodus pricing page, RunPod pricing, Modal pricing.

The rate gap is small next to the GPU choice: the same 10 hours on an RTX 4090 at $0.29/hr is $2.90.

The hourly rate is only part of it. A RunPod Pod gives you a machine where you run and restart the script yourself. Modal restarts a preempted function on the same input and asks you to add your own checkpoints. If you already work in either, Unsloth runs there too, and the script above works there apart from the state path and the Nodus callback.

When is Colab or your own GPU enough?

For learning Unsloth and short experiments, start in Google Colab. Unsloth's README links a free Colab notebook for each of its featured models. The catch is that Colab's FAQ says free resources are not guaranteed, idle VMs are deleted, and lifetimes and GPU types vary over time.

A 24 GB card you already own is enough for QLoRA up to 14B by Unsloth's table. It costs nothing per hour and your data never leaves the machine. Move to a cloud job when the run has to finish overnight without you watching it, when the model outgrows your card, or when you want several runs at once.

If you would rather not maintain a training script, the managed nodus/sft runtime in Nodus Training (Beta) does full, LoRA or QLoRA fine-tuning from a JSONL file. It is not Unsloth, so you give up Unsloth's kernels for less code.

FAQ

Does Unsloth work on multiple GPUs?

Unsloth's README says it supports multi-GPU setups. For most QLoRA jobs up to 70B, one GPU is enough by its own VRAM table, and a single card avoids the communication overhead entirely.

Do I need a Hugging Face token?

Only for gated models. Unsloth publishes copies of many open models under its Hugging Face organization. For a gated one, store your token as a Nodus Secret and pass it with --secret.

Can I use Unsloth's Docker image instead of pip?

Unsloth publishes an official Docker image, unsloth/unsloth, built mainly for running Jupyter and Unsloth Studio locally. Nodus runs prebuilt public registry images, but test the image's startup command before a long run.

How do I keep a large dataset from uploading on every run?

Put it in a Nodus Volume once with nodus volume put, then mount it with --volume NAME:/data:ro. Only the code folder uploads with each run.

What do I do with the adapter afterwards?

Load it back with FastLanguageModel.from_pretrained pointed at the adapter folder, or merge it into 16-bit weights with Unsloth's save_pretrained_merged. The VRAM guide covers what serving the merged model needs.

Originally published on the Nodus blog.

How do you train a PyTorch model on multiple GPUs?

Viswaretas Kotra — Wed, 28 Oct 2026 01:38:50 +0000

Wrap your model in DistributedDataParallel (DDP) if it fits on one GPU, or shard it with FSDP if it doesn't, and start one process per GPU with torchrun. On Nodus, nodus run --gpu H100:8 -- torchrun train.py runs that script on all eight GPUs of one machine with no launcher flags, and --nodes 2 --launcher torchrun spreads it across two machines. Fill one machine before you add a second: the second machine buys wall-clock time, not a cheaper run.

This post walks through the three decisions in order: which wrapper, how to launch it, and when more than one machine is worth paying for. The code follows the PyTorch tutorials and the Nodus docs; the memory and cost figures are our own arithmetic, shown in full so you can redo it for your model.

Should you use DataParallel, DDP or FSDP?

Use DDP by default and skip nn.DataParallel. The PyTorch DDP tutorial explains why: DataParallel is a single process with many threads, so it fights the Python GIL and only works on one machine, while DDP runs one process per GPU and works across machines too.

DDP gives every GPU a full copy of the model. Each one trains on a different slice of the batch, then an all-reduce averages the gradients so every copy stays identical. That is simple and fast, but it means the whole training state has to fit on each GPU.

FSDP removes that limit. The PyTorch FSDP tutorial describes it as sharding model parameters, gradients and optimizer states across GPUs, which "makes it feasible to train models that cannot fit on a single GPU." Each GPU holds only its shard and gathers a layer's full weights just before it needs them.

The decision rule we use: if the model, its gradients and the optimizer state fit on one GPU with room for activations, use DDP. If they don't, use FSDP. If even FSDP is tight, look at LoRA before you add hardware.

How much memory does each GPU need with DDP and FSDP?

Full training with Adam in mixed precision costs about 16 bytes per parameter before activations. That figure comes from the ZeRO paper: 2 bytes for 16-bit weights, 2 for gradients and 12 for the FP32 master weights plus Adam's two moments. DDP keeps all 16 bytes on every GPU. FSDP with full sharding divides them by the number of GPUs.

Model	Model state per GPU with DDP	FSDP on 8 GPUs	FSDP on 16 GPUs
1B parameters	16 GB	2 GB	1 GB
8B parameters	128 GB	16 GB	8 GB
70B parameters	1,120 GB	140 GB	70 GB

The math is parameters x 16 bytes, divided by the GPU count for FSDP. Activations, temporary all-gather buffers and the CUDA context come on top, so treat these as floors.

On 80 GB GPUs, an 8B model needs FSDP and a 70B model needs more than one 8-GPU machine of them.

Read it against an 80 GB NVIDIA H100. A 1B model fits comfortably with DDP. An 8B model does not fit at 128 GB, but drops to 16 GB per GPU with FSDP on one 8-GPU machine. A full 70B run needs 140 GB per GPU even when sharded over 8 GPUs, so on H100s it needs two machines.

Even then, 70 GB on each of 16 GPUs leaves little room for activations. That is why our VRAM guide for 70B models points full 70B fine-tuning at one machine with 8 NVIDIA B200s instead: a bigger single machine beats a second small one.

What does a DDP training script look like?

Four changes turn a single-GPU script into a DDP script: start a process group, pin each process to its GPU, wrap the model, and give each process its own slice of the data. This sketch follows the DDP tutorial's pattern:

import os

import torch
import torch.distributed as dist
from torch.nn.parallel import DistributedDataParallel as DDP
from torch.utils.data import DataLoader
from torch.utils.data.distributed import DistributedSampler

dist.init_process_group("nccl")
local_rank = int(os.environ["LOCAL_RANK"])
torch.cuda.set_device(local_rank)

model = DDP(build_model().cuda(local_rank), device_ids=[local_rank])
sampler = DistributedSampler(dataset)
loader = DataLoader(dataset, batch_size=32, sampler=sampler)
opt = torch.optim.AdamW(model.parameters(), lr=1e-4)

for epoch in range(epochs):
    sampler.set_epoch(epoch)
    for x, y in loader:
        loss = loss_fn(model(x.cuda(local_rank)), y.cuda(local_rank))
        opt.zero_grad()
        loss.backward()
        opt.step()

dist.destroy_process_group()

build_model, dataset, epochs and loss_fn are yours. Two lines are easy to miss. DistributedSampler stops every GPU from training on the same examples, and sampler.set_epoch(epoch) reshuffles each epoch; without it, PyTorch's own docstring warns, the same ordering is used every time.

The batch size above is per GPU. On 8 GPUs the effective batch is 8 x 32 = 256, so revisit your learning rate when you scale out.

How do you switch the same script to FSDP?

Replace the DDP wrapper with fully_shard from FSDP2, applied to each transformer block first and then to the whole model. The FSDP tutorial uses exactly this order:

from torch.distributed.fsdp import fully_shard

model = build_model()
for layer in model.layers:
    fully_shard(layer)
fully_shard(model)
opt = torch.optim.AdamW(model.parameters(), lr=1e-4)

Sharding per block matters. It lets FSDP gather one block's weights, run it and free them again, so only one block is ever whole on a GPU. Create the optimizer after sharding so it sees the sharded parameters. The data loading and training loop stay the same as the DDP version.

How do you launch it on one machine with several GPUs?

Run the script under torchrun, which starts one process per GPU and sets RANK, LOCAL_RANK and WORLD_SIZE for each. On your own machine you pass --nproc-per-node. On Nodus you ask for the GPUs and leave the flags off:

nodus run --gpu H100:8 --max-cost 250 -d -- torchrun train.py

The Nodus GPU guide explains why no flags are needed: Nodus sets PET_NPROC_PER_NODE to the GPU count, so torchrun starts one process per GPU, and /dev/shm is already sized for NCCL. --max-cost caps the spend in dollars, here just above the $208.00 estimate worked out below, and the same guide shows a --dry-run that prints the estimate before anything starts.

When is a second machine worth it?

Add a machine when one machine cannot hold the job or cannot finish it in time. More GPUs rarely make a run cheaper, because the gradient traffic between machines is slower than inside one, so each GPU does a little less useful work.

Here is a worked example. Say a run takes 10 hours on one machine with 8 H100s. H100s are listed at $2.60/hr on the Nodus pricing page, so that is 80 GPU-hours, or $208.00.

Now run it on two machines. The cost depends on scaling efficiency, the share of perfect speed-up you actually get, which you have to measure for your own model and network.

Setup	Wall-clock	GPU-hours	Cost at $2.60/GPU-hour
1 machine, 8 GPUs	10.0 h	80.0	$208.00
2 machines, 16 GPUs, 90% efficiency	5.6 h	88.9	$231.11
2 machines, 16 GPUs, 75% efficiency	6.7 h	106.7	$277.33

Wall-clock is 10 hours divided by (2 x efficiency); GPU-hours is 16 times that. Prices checked October 27, 2026. The efficiencies are assumptions for the example, not measurements.

Two machines finish sooner but cost 11% to 33% more in this example.

So the second machine is a time purchase. At 90% efficiency you pay $23.11 more to get the result about 4.4 hours sooner. If nobody is waiting on it, one machine is the cheaper answer.

How do you run one job across several machines on Nodus?

Add --nodes and --launcher torchrun to the same command. The Nodus multi-node guide uses this as its two-node smoke test:

nodus run --name torchrun-2-node --gpu H100 --nodes 2 --launcher torchrun --max-cost 2.00 -- torchrun train.py

Nodus acquires every machine, checks that they can reach each other, and only then starts your command on all of them together. It sets MASTER_ADDR, NODE_RANK and the PET_* variables that torchrun reads, so the script itself does not change. --gpus-per-node 8 gives each machine eight GPUs, and --total-gpus lets Nodus pick the machine shape for you.

Multi-node training is in Beta, and the docs are specific about its limits. Gangs top out at 8 machines.

By default all machines are placed on one private network in one region. If you allow wider placement, traffic between machines on public addresses is not encrypted in the Beta. The guide also says a two-machine DDP result does not yet qualify cross-machine FSDP or several GPUs per machine across machines, so check the exercised combinations before a large FSDP run.

What happens when one GPU in a multi-GPU run fails?

The whole run stops, because every process waits on the others in each all-reduce. That makes checkpoints more important as you add GPUs: one lost machine out of two throws away everything since the last save.

On Nodus, the multi-node guide describes the recovery. If a machine is lost or reclaimed, Nodus stops every rank, keeps the surviving machines, replaces only the lost one, and restarts all ranks with the latest checkpoint restored into the state directory. Your script has to save there and load from it on start; the checkpoint guide in the Nodus docs shows how, and our post on automatic checkpointing and resume compares how other platforms handle it.

FAQ

How many GPUs can torchrun use on one machine?

As many as the machine has, one process per GPU. On Nodus, the GPU count you ask for is 1, 2, 4 or 8 per machine, and anything larger needs a multi-node job.

Do I need NVLink for multi-GPU training?

DDP and FSDP run over NCCL on whatever links the GPUs share, so NVLink is not required. It does speed up the gradient and all-gather traffic, which matters most for FSDP because it moves weights as well as gradients.

Can I use Hugging Face Accelerate or DeepSpeed instead of torchrun on Nodus?

Nodus accepts Accelerate and Deepspeed as launchers for multi-node jobs, setting their variables or a hostfile, but the multi-node guide marks them accepted in the Beta and qualified later. torchrun, Ray and verl are documented without that caveat.

Should I use LoRA instead of FSDP for a large model?

Often, yes. LoRA freezes the base weights and trains small adapters, so gradients and optimizer state shrink to a fraction of a percent of the model. If LoRA quality is good enough for your task, it usually fits a model on one GPU that full training would need a cluster for.

Originally published on the Nodus blog.

What are the best Hugging Face AutoTrain alternatives?

Viswaretas Kotra — Tue, 27 Oct 2026 23:11:38 +0000

Hugging Face now marks AutoTrain as no longer maintained, so the best alternative depends on whether you want a managed recipe, a fine-tuning API or your own training script. Nodus is the closest managed replacement for LLM fine-tuning: its recipes take a Hugging Face model and a dataset, run on the GPU you choose and resume from the latest checkpoint if a machine is lost. Pick Together AI if you want to pay per token without choosing hardware, and Axolotl or Unsloth if you want to run open-source trainers on hardware you already have.

We build Nodus, so read this with that in mind. Every fact about another product links to its own pricing page or docs, and the arithmetic is written out so you can redo it with today's prices.

What happened to Hugging Face AutoTrain?

AutoTrain still works, but it is frozen. The AutoTrain docs and the GitHub README both open with the same notice: no new features will be added and bugs will not be fixed. Hugging Face recommends Axolotl, TRL or transformers.Trainer instead.

The tool itself is free. You pay only for compute, per the AutoTrain cost page. Locally that is your own hardware; as a Space it is billed per minute for the Hugging Face Spaces hardware you attach. The Spaces quickstart has you duplicate the AutoTrain Space, pick hardware, and train from the web UI, with logs in TensorBoard and the model pushed to a Hub repository.

That setup is pleasant for a first fine-tune. The trouble starts when a new model architecture or library version needs a fix that will not come, or when you want a GPU that the Spaces hardware list does not offer.

How do the AutoTrain alternatives compare?

The short version: Nodus and Together AI replace the managed part, Modal, RunPod and Vertex AI replace the compute part, and Axolotl or Unsloth replace the trainer.

Option	What you get	How you pay	GPU choice	If a machine is lost
AutoTrain on Spaces	No-code UI, frozen codebase	Per minute of Spaces hardware	Spaces list (T4 up to A100 and RTX PRO 6000)	`PAUSE_ON_FAILURE` pauses the Space on a failed run
Nodus	Managed recipes: SFT, LoRA, QLoRA, DPO, ORPO, KTO, distillation, GRPO	Prepaid credits, listed hourly price per GPU	You allow GPU types, or take the recipe's preset	Resumes from the latest full checkpoint
Together AI	Fine-tuning API	Per 1M training tokens, minimum charge per job	None, managed	Not stated on the pricing page
Modal	Run your own trainer as Python functions	Per second	You pick per function	Restarts the function on the same input; you write checkpoints
RunPod	GPU pods for your own trainer	Per second	You pick the pod	You handle restarts; data without a network volume is lost if the balance hits $0
Google Vertex AI	Custom training jobs	Machine type plus attached accelerator	You pick machine and accelerator	Not stated on the pricing page
Axolotl or Unsloth	Open-source trainers (Apache-2.0)	Free; you supply hardware	Whatever you run them on	Your own checkpoint and restart logic

Checked October 27, 2026: AutoTrain cost, Spaces quickstart, Hugging Face pricing, Nodus training guide, Together AI pricing, Modal pricing and preemption docs, RunPod pricing and Pods pricing docs, Vertex AI pricing, Axolotl and Unsloth.

Which is cheapest for a 10-hour fine-tune on one A100?

At listed prices, Nodus is the cheapest of the hourly options we priced for the same GPU, and AutoTrain on Spaces sits near the top. We priced one NVIDIA A100 80GB for 10 hours, GPU compute only, from each company's own page.

Where it runs	Listed price for one A100 80GB	10 hours
Hugging Face Spaces (AutoTrain), A100 large	$2.50/hr	$25.00
Modal	$0.000694/s, or $2.4984/hr	$24.98
RunPod Secure Cloud, A100 SXM	$1.79/hr	$17.90
RunPod Community Cloud, A100 SXM	$1.39/hr	$13.90
Nodus	$1.09/hr	$10.90

Prices checked October 27, 2026 on the Hugging Face pricing page, Modal pricing, RunPod pricing and the Nodus pricing page, where the A100 80GB is listed at $1.09/hr. Modal's per-second rate is multiplied by 36,000 seconds.

The same 10 A100-hours cost $14.10 less on Nodus than on an AutoTrain Space.

Google Vertex AI is harder to put in one line because it adds charges together. Its pricing page lists the A100 80GB accelerator for custom training at $3.92808/hr in Iowa (us-central1), so $39.28 for 10 hours before the cost of the machine type is added.

Together AI does not bill by the hour at all. Its pricing page charges for tokens processed: training set size times epochs, plus any evaluation tokens. A LoRA fine-tune of Llama 3.1 8B is $0.34 per 1M tokens with a $4.00 minimum.

A 20M-token dataset trained for 3 epochs is 60M tokens, so 60 x $0.34 = $20.40. That is predictable, and you never size a GPU, but you also cannot make it cheaper by picking a smaller one.

Does your GPU choice matter for a LoRA run?

Yes, and often more than the platform. A recipe that lets you pick the GPU lets you skip paying H100 prices for a job that a 24 GB card finishes for less.

Here is how we checked it. The nodus/sft runtime page publishes the measured seconds per step that its dry-run estimate uses: 0.45 on H100, 0.7 on A100 80GB, 1 on L40S, 1.2 on RTX 4090 and 1.3 on RTX A6000. Multiply by 2,000 steps, the step count in the docs' own LoRA example, then by the listed price on the Nodus pricing page.

GPU	Time for 2,000 steps	Listed price	Estimated GPU cost
RTX 4090	40 min	$0.290/hr	$0.19
RTX A6000	43.3 min	$0.333/hr	$0.24
A100 80GB	23.3 min	$1.090/hr	$0.42
L40S	33.3 min	$0.793/hr	$0.44
H100 SXM	15 min	$2.600/hr	$0.65

The H100 finishes first, but the RTX 4090 does the same 2,000 steps for less than a third of the cost.

This is arithmetic from published numbers, not a bill from a run. Real step time moves with model size, sequence length and batch size, which is why the dry run prices your actual configuration. The rule still holds: if the job fits in 24 GB and nobody is waiting on it, take the cheaper card. If someone is waiting, the H100 buys back 25 minutes for 46 cents.

The runtime's presets already make that call for small models. Qwen3-1.7B with LoRA defaults to one RTX 4090, L40S or RTX A6000, while full fine-tuning of Qwen3-8B defaults to 8 x H100.

What happens when a fine-tuning run fails halfway?

On Nodus the run resumes from its last checkpoint. On a GPU you rent directly, the restart is your job.

The training guide says recovery uses full checkpoints with optimizer, scheduler and RNG state, and the sft runtime saves one every 200 steps by default (saveSteps). If a node in a multi-node run is lost, the whole group restarts from the latest checkpoint every rank committed, up to 3 times. The spending cap behaves the same way: when maxCostUSD is reached the run is suspended, not failed, and raising the cap resumes it from the checkpoint. Our post on automatic checkpointing and resume compares how other platforms handle this.

Modal's preemption docs say a preempted function restarts on the same input and recommend checkpointing long training runs yourself. RunPod's Pods pricing docs say pods stop when your balance reaches $0, and pods without a network volume lose their data. With Axolotl or Unsloth, noticing the failure and restarting from a saved checkpoint is up to you.

Where is AutoTrain still the better choice?

AutoTrain is still a reasonable pick in a few specific cases.

You want a point-and-click UI. Nodus starts runs from the CLI or Python SDK; its console shows runs, curves and costs but does not launch them.
Your task is not an LLM. AutoTrain covers image classification, tabular data, token classification, extractive question answering and more, per its docs and README. The Nodus training catalog is built around language models.
You run it locally on your own GPU, where the AutoTrain cost page says it costs nothing.
You have a working AutoTrain config for a model it already supports, and you need one more run of it.
Your whole workflow lives on the Hugging Face Hub, and a Space that pushes the model and TensorBoard logs straight to a repository is the point.

If none of those apply, the maintenance notice is the deciding fact. A trainer that will not get fixes is a risk for anything you plan to repeat.

How do you fine-tune a Hugging Face model on Nodus instead?

Upload your data to a Volume, write a TrainingJob that names the Hugging Face model, dry-run it, then apply it. These commands and the manifest are copied from the Nodus training guide and the Volumes guide.

$ pip install nodus-compute
$ nodus login
$ nodus create volume support-chats --size 100Gi
$ nodus volume put support-chats ./chats.jsonl /

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: support-sft
spec:
  runtime: nodus/sft
  model:
    uri: hf://Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
    secret: hf-token            # only for gated repositories
  data:
    volume: support-chats       # a Volume holding chats.jsonl
    path: chats.jsonl
    format: JSONL
    columns: {prompt: question, completion: answer}
  parameters:
    steps: 2000
    lora: {r: 16, alpha: 32}
  maxCostUSD: "20.00"

$ nodus apply -f sft.yaml --dry-run -o yaml     # status.estimate: cost to completion, first hold, start time
$ nodus apply -f sft.yaml
$ nodus get tj -w
$ nodus cp tj/support-sft:outputs/adapter/adapter_model.safetensors ./adapter_model.safetensors

The columns map plays the same role as AutoTrain's column mapping, so an AutoTrain dataset usually moves over without edits. Pin the model to a 40-character commit. Nodus imports that revision once into a read-only Volume and runs the trainer offline, so billed GPU time is never spent downloading weights. To pin a GPU type, add resources: {gpu: {type: [h100-sxm-80g], count: 1}}.

From Python, the same run is one line from the finetune recipe reference, after import nodus and from nodus.recipes import finetune:

sft = finetune.sft(model="Qwen/Qwen3-1.7B", data=nodus.Volume.from_name("support-chats"),
                   lora=finetune.LoRA(r=16), max_steps=2000).run(gpu="H100", max_cost=20)

TrainingJobs are in Beta, so fields can still change before v1. In our fine-tuning benchmark against Modal, a pilot with one small model on one NVIDIA L4, Nodus produced byte-identical weights at 67% lower charges for the successful run. If you already have cloud credits, the cloud accounts docs show how Nodus can run work in your own AWS, GCP or Azure account, or in a RunPod or Lambda account you connect with its API key.

FAQ

Is AutoTrain shutting down?

Hugging Face has not said it is shutting down; its docs say it is no longer maintained. The Space and the autotrain-advanced package still work, but bugs will not be fixed and new model architectures will not be added.

What does Hugging Face recommend instead of AutoTrain?

Its notice names Axolotl, TRL and transformers.Trainer. Those are trainers, not compute, so you still need a GPU to run them on. The Nodus sft runtime page lists its checkpoint format as HFTrainer, so the concepts carry over.

Can I run Axolotl or Unsloth on Nodus?

Yes, as your own code. A custom TrainingRuntime can run any image, and Nodus decides when to checkpoint the state directory your trainer writes to. See Bring your own runtime in the training guide, or our Modal alternatives for training post for running scripts as plain Jobs.

Can a coding agent start the fine-tune for me?

Yes. nodus mcp install claude adds the Nodus MCP server to Claude Code, and the agent must accept a dry-run estimate before it creates anything. Runs stay inside your account's budgets and the job's maxCostUSD cap.

Is there a free tier like AutoTrain's local mode?

Running Axolotl, Unsloth or AutoTrain on your own GPU costs nothing beyond the hardware. Modal's $0 plan includes $30 of free credits a month, per its pricing page. Nodus uses prepaid credits; current GPU rates are on the pricing page, and our RunPod alternatives post compares hourly rates more widely.

Originally published on the Nodus blog.

Continued pretraining vs fine-tuning: which do you need?

Viswaretas Kotra — Tue, 27 Oct 2026 20:45:42 +0000

Use continued pretraining when the model lacks knowledge or vocabulary from your domain, and use fine-tuning when it knows enough but behaves wrong for your task. Nodus runs both from one managed runtime, nodus/sft, and in our October 2026 pilots it produced byte-identical weights to Modal at 42.5% lower charges for continued pretraining and 67% lower for fine-tuning. Most teams need fine-tuning only; continued pretraining pays off when you hold a large body of domain text the base model has rarely seen.

What is the difference between continued pretraining and fine-tuning?

The difference is the data and what the model learns from it. Continued pretraining keeps the original next-token objective and feeds the model raw, unlabeled text: contracts, papers, internal docs, a codebase. The model absorbs vocabulary, style and facts from that corpus.

Fine-tuning trains on labeled examples: prompt and completion pairs, chat transcripts, or chosen and rejected answers for preference methods. It teaches the model how to respond, in what format, and when to refuse.

What you want	Method	Data you need
Domain vocabulary and facts	Continued pretraining	Raw text
A fixed output format or tone	Supervised fine-tuning (SFT)	Prompt/completion pairs
Better instruction following on your task	SFT	Chat examples
Answers people prefer	DPO, ORPO or KTO	Chosen/rejected pairs or labels

Start from the data you actually have: raw text points to pretraining, examples point to fine-tuning.

When does continued pretraining pay off?

It pays off when the base model keeps misreading your domain, not when it merely answers in the wrong shape. The strongest evidence is Gururangan et al., "Don't Stop Pretraining" (ACL 2020). Across four domains (biomedical papers, computer science papers, news and reviews) and eight classification tasks, a second pretraining phase on in-domain text improved results in both high- and low-resource settings.

Our decision rule has three checks:

The base model mangles your terms, abbreviations or code conventions even with good prompts.
You have far more raw domain text than labeled examples, typically hundreds of millions of tokens or more.
The knowledge is stable. Facts that change weekly belong in retrieval, not in weights.

If all three hold, continued pretraining is worth the compute. If only the first holds, try fine-tuning on a few thousand examples first and measure. Continued pretraining can also erode general skills, so keep a held-out general benchmark next to your domain test. Our post on catastrophic forgetting after fine-tuning shows how to measure that.

When is fine-tuning enough?

Fine-tuning is enough when the model already understands the material and you need consistent behavior. Typical cases are a JSON schema it must always follow, a support tone, a classification label set, or tool calls in your format.

It needs much less data than people expect. Zhou et al. fine-tuned a 65B model on 1,000 curated examples in "LIMA: Less Is More for Alignment" and reported strong instruction following.

Quality and coverage of the examples matter more than raw count. LoRA and QLoRA train a small adapter instead of every weight, which cuts memory enough to fit larger models on one GPU (see how much VRAM a 70B model needs).

How much more does continued pretraining cost?

Cost grows with the tokens you train on, so continued pretraining usually costs tens of times more than fine-tuning. Here is the math for a 7B model, full-weight training on one NVIDIA H100, listed at $2.60/hr on the Nodus pricing page (price checked October 27, 2026).

Training compute is roughly 6 x parameters x tokens floating-point operations, the standard estimate from Kaplan et al. (2020). NVIDIA lists the H100 SXM at 1,979 BF16 teraFLOPS with sparsity, about 990 without it.

We assume 40% utilization, so about 396 teraFLOPS of useful work. That utilization is an assumption, not a measurement; your real number depends on sequence length, kernels and batch size.

Continued pretraining on 1 billion tokens: 6 x 7B x 1B = 4.2 x 10^19 FLOPs. At 396 teraFLOPS that is 106,000 seconds, about 29.5 hours, or about $77.
Fine-tuning on 50,000 examples of about 500 tokens (25 million tokens): 1.05 x 10^18 FLOPs, about 0.74 hours, or about $1.92.

The ratio is 40 to 1 because the token count is 40 to 1. Change the assumed utilization and both numbers move together, so the ratio holds. LoRA lowers the fine-tuning side further because frozen weights need no weight gradients or optimizer state.

Same model, same GPU, same rate: the 40x gap comes entirely from tokens trained.

How did Nodus compare with Modal on both jobs?

We ran the same continued-pretraining and fine-tuning recipes on Nodus and on Modal with Qwen2.5-0.5B on one NVIDIA L4. In both pilots the exported weights matched byte for byte, so the only difference was the bill.

Pilot	Nodus charge	Modal charge	Nodus share of Modal
Supervised fine-tuning	$0.048696	$0.14746756	33.0%
Continued pretraining	$0.121164	$0.21090282	57.5%

Recorded charges for successful attempts before credits, October 2026 pilots, one seed and one accepted run per platform. The pilots used standalone Nodus Jobs, not TrainingJobs. In the chart, Modal's charge is the 100.0% baseline.

Identical weights in both pilots, so the lower charge bought the same model.

Two caveats from those posts. The fine-tuning pilot also had one failed Nodus attempt; counting it, Nodus cost 24.7% less than Modal instead of 67%. And a 0.5B model on one L4 says nothing certain about a 70B run, so treat the ratio as evidence for this job size only.

How do you run continued pretraining on Nodus?

Use the nodus/sft runtime with task: Pretrain on the spec, and point it at a Volume of text rows. The default initialization: Continued starts from the model's existing weights, and method: Full trains every weight rather than an adapter.

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: domain-pretrain
spec:
  runtime: nodus/sft
  task: Pretrain
  model:
    uri: hf://Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
  data:
    volume: domain-corpus
    path: corpus.jsonl
    format: JSONL
  parameters:
    steps: 5000
    initialization: Continued
    method: Full
    packing: true
    appendEOS: false
  maxCostUSD: "50.00"

Set appendEOS: false when your rows are chunks or paragraphs rather than whole documents, so the trainer does not insert an end-of-sequence token after every row. Packing fills each training block with text instead of padding. Price it before it runs:

$ pip install nodus-compute
$ nodus login
$ nodus apply -f pretrain.yaml --dry-run -o yaml
$ nodus apply -f pretrain.yaml

The dry run returns a cost estimate, and maxCostUSD stops the run at the cap. Long pretraining runs checkpoint with optimizer state, so a lost GPU resumes the run instead of restarting it.

How do you run fine-tuning on Nodus?

Same runtime, no task field, and a columns map onto your prompt and completion fields. The lora block makes it an adapter run; leave it out for full fine-tuning.

apiVersion: nodus.dev/v1beta1
kind: TrainingJob
metadata:
  name: support-sft
spec:
  runtime: nodus/sft
  model:
    uri: hf://Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca
  data:
    volume: support-chats
    path: chats.jsonl
    format: JSONL
    columns: {prompt: question, completion: answer}
  parameters:
    steps: 2000
    lora: {r: 16, alpha: 32}
  maxCostUSD: "20.00"

The Training (Beta) guide covers validation-based checkpoint selection and preference tuning. A model you trained earlier can be the input of the next TrainingJob, which is how you chain the two methods.

Should you do both, and in what order?

Do both only when you have both kinds of data, and always pretrain first. Gururangan et al. found that adapting to the task's own unlabeled text helped even after domain-adaptive pretraining, so the stages stack.

A sequence that works: continued pretraining on the domain corpus, then SFT on labeled examples, then preference tuning if you have chosen and rejected answers. Each stage starts from the previous stage's exported model. Evaluate after every stage on the same held-out set, plus a general benchmark, so you can see which stage helped and which one cost you general skill.

FAQ

What is the difference between pretraining and fine-tuning?

Pretraining teaches a model language and facts from huge amounts of raw text with a next-token objective. Fine-tuning adjusts an already pretrained model with labeled examples so it behaves a certain way. Continued pretraining sits between them: more raw-text pretraining, but on your domain.

Can continued pretraining replace retrieval (RAG)?

Not for facts that change. Weights are a snapshot, and retraining every time a document changes is slow and expensive. Teams often pretrain for vocabulary and reasoning style, then use retrieval for current facts.

What learning rate should continued pretraining use?

Lower than the original pretraining peak, with a warmup, because a high rate on a narrow corpus speeds up forgetting. Run a short sweep on a small slice and keep the setting whose held-out general score drops least.

How do you know continued pretraining worked?

Measure perplexity on held-out domain text before and after, then check a general benchmark for forgetting. In our continued-pretraining pilot, held-out perplexity fell 21.8%, from 21.0 to 16.4, on both Nodus and Modal.

Which is cheaper on Nodus?

Fine-tuning, by roughly the ratio of tokens trained. Both run at the same listed GPU rates on the pricing page, and the dry run shows the estimate before anything is charged.

Originally published on the Nodus blog.

Which GPU platforms checkpoint and resume training for you?

Viswaretas Kotra — Tue, 27 Oct 2026 18:12:40 +0000

We checked five platforms against their own docs. Nodus, AWS SageMaker managed spot training and Modal all restart a preempted training run and hand it back the files it saved, but only Nodus also tells your program when to save, including an urgent request when capacity is about to be reclaimed. RunPod Pods and Vast.ai interruptible instances keep your disk and leave the restart to you.

One thing is true everywhere: no platform resumes your training loop for you. Your script still has to look for its last checkpoint when it starts and load it.

What differs is everything around that load, and that is the part that decides whether a run finishes unattended. This post compares the platforms. For the code itself, see how to make spot GPU training survive preemption.

Who saves, who restarts and who resumes?

Recovery after a lost GPU has five steps. Something has to trigger a save, copy it off the machine, start a new machine, put the files back and continue the loop. Here is who does each one, according to each platform's own documentation.

Step	Nodus	Modal	AWS SageMaker managed spot	RunPod Pods	Vast.ai interruptible
Trigger a save before a reclaim	Platform sends an urgent checkpoint request	Your exit handler, 30 s grace	Your code or framework callback	Your code	Your code
Scheduled saves	Platform-timed (`interval: auto`) or fixed, from 1 min to 6 h	Your code	Your code or framework callback	Your code	Your code
Copy the checkpoint off the machine	Platform, from `/nodus/state`	Platform, once you write to a Volume	Platform, syncs `/opt/ml/checkpoints` to S3	You, onto a network volume	You, to your own storage
Start a replacement	Platform, on another machine	Platform, on the same input	Platform, on the next Spot instance	You, or a beta migration you trigger	Same instance resumes when it regains priority
Put the files back	Platform restores the state directory	Volume mounted again	Platform copies S3 back to the local path	Attach the network volume	Disk stays on that machine
Continue the training loop	Your code	Your code	Your code (some built-in algorithms do it)	Your code	Your code

Sources: Nodus checkpoints, Nodus attempts and recovery, Modal preemption, Modal exit handlers, Modal Volumes, SageMaker checkpoints, SageMaker managed spot training, RunPod maintenance and data safety, RunPod Pod migration, Vast.ai instance types, Vast.ai rental types FAQ. Checked October 27, 2026.

Read across the rows and three groups appear. Nodus, SageMaker and Modal own the restart and the file handling once your code writes to the right place. Only Nodus also owns the save trigger and the interval. On RunPod and Vast.ai the platform keeps your disk, and the rest of recovery is yours to build.

What does Nodus do automatically?

Nodus treats every run as a series of attempts. When a provider reclaims interruptible capacity, Nodus sends your program an urgent checkpoint request and starts preparing a replacement machine at the same time. The replacement only starts once the old machine is provably gone, so two machines never write the same checkpoint. A checkpoint committed during the notice still counts.

Between notices, interval: auto sets the cadence from how often the capacity is interrupted and how long your saves take. A run gets at least four checkpoints, and checkpointing takes no more than about a tenth of the run time. A machine that stops sending heartbeats gets a replacement prepared after 20 seconds. On shared capacity it is replaced after 60 seconds of silence.

Recovery is bounded. A run fails with NoProgress after two attempts in a row make no progress, and it gets 8 recoveries by default, or 3 for distributed Jobs. Your code's share is to write its state to /nodus/state and read it back on start. Hugging Face Trainer users can set integration: HFTrainer and Nodus answers the checkpoint requests from inside Trainer.

Does AWS SageMaker managed spot training do the same?

Mostly. With managed spot training and a checkpoint S3 URI, SageMaker syncs your local checkpoint directory to Amazon S3 during training. After a Spot interruption it resumes the job on the next Spot instance and copies the checkpoints back before your script starts (SageMaker checkpoints). AWS Deep Learning Containers and some built-in algorithms checkpoint without script changes.

Plan around two limits. First, managed spot training never falls back to on-demand instances. If Spot capacity is unavailable, the job waits until MaxWaitTimeInSeconds and then stops (managed spot training).

Second, the save itself is still triggered by your code or framework, since SageMaker copies what is in the directory rather than asking your program for a consistent state. Our Nodus vs AWS SageMaker post compares the two on price and setup.

What does Modal handle, and what is left to you?

Modal restarts. Every Modal Function is preemptible by default, nonpreemptible is not supported for GPU Functions, and after a preemption Modal restarts the Function on the same input (Modal preemption). On a preemption your exit handler runs with a 30 second grace period (lifecycle functions).

The saving is yours. Modal's long-training example has your code write checkpoints to a Volume, look for the latest one on start, and use modal.Retries to restart.

Volumes commit in the background every few seconds, so once a file is written it leaves the machine without extra work (Modal Volumes). A single call can run for at most 24 hours (timeouts), which is another reason long runs need checkpoints there. The full comparison is in Nodus vs Modal.

What about RunPod and Vast.ai?

Both keep your data and leave recovery to you. RunPod's docs say hardware failures can happen without warning, that container storage is lost when a Pod is interrupted, restarted, stopped or terminated, and that long jobs should checkpoint every hour to every few hours onto a network volume (RunPod data safety). If your Pod's GPU is taken while it is stopped, a beta migration you trigger copies /workspace to a new Pod with a new ID (Pod migration).

Vast.ai interruptible instances are stopped when a higher bid wins, which kills running processes. A paused instance resumes when it regains priority, possibly after a long wait, and the docs tell you to save to disk often and checkpoint long jobs (rental types FAQ).

What does a lost checkpoint interval cost?

On every platform, an interruption without a save on the notice costs the work since the last scheduled checkpoint. That is half an interval on average and a full interval at worst. We priced it on two common training shapes using rates listed on the Nodus pricing page: 8 x NVIDIA H100 SXM at $2.60/hr each, or $20.80/hr, and 8 x NVIDIA B200 at $5.50/hr each, or $44.00/hr.

Checkpoint every	Average redone work	8 x H100 ($20.80/hr)	8 x B200 ($44.00/hr)
15 min	7.5 min	$2.60	$5.50
30 min	15 min	$5.20	$11.00
60 min	30 min	$10.40	$22.00
120 min	60 min	$20.80	$44.00

Prices checked October 27, 2026. Our calculation: half the interval times the hourly rate of the run. Save time and restart time are not included.

Each extra hour between checkpoints adds about half an hour of redone work to every interruption.

Now take one run: 24 hours on 8 x H100, which is 192 GPU-hours or $499.20 at $2.60/hr, interrupted three times. Saving every 2 hours redoes $62.40 of work on average (3 x 1 h x $20.80), or $124.80 at worst (3 x 2 h x $20.80). That is 12.5% to 25% on top of the run. Saving every 30 minutes cuts that to $15.60 on average (3 x 0.25 h x $20.80) and $31.20 at worst, before counting the extra save time.

A save that commits during the reclaim notice takes the redone work close to zero, which is why who triggers it matters.

A save on the reclaim notice changes the picture, but only if the save fits the warning. On Nodus the platform sends the request as soon as the notice arrives. On Modal you write that save in an exit handler with 30 seconds. On the others you watch for the notice yourself.

When the save does not fit, the scheduled interval is what you pay, and our interval model shows how to pick it.

Which one should you pick?

Our decision rule, from the table above:

Your runs are short, or a lost hour is cheap, and you are happy to restart by hand: any platform works. RunPod on-demand Pods are dedicated to you and cannot be displaced by other users (Pod pricing).
You want Spot prices and live in AWS, with data in S3: SageMaker managed spot training, accepting that it waits for Spot capacity rather than falling back.
Your training is a Python function that fits Modal's model and its 24 hour call limit with retries: Modal, with your own save and resume code on a Volume.
You want runs to finish unattended on the cheapest capacity, with the platform asking for saves and choosing the interval: Nodus.

Whether interruptible capacity pays off at all for your run is a separate question, covered in spot vs on-demand GPUs.

Where are the other platforms the better choice?

SageMaker is the better fit when your training already runs on AWS Deep Learning Containers. Some frameworks there checkpoint without script changes, and your data in S3 and your AWS commitments stay where they are. Modal is the better fit for teams that already write Modal Functions, since background Volume commits make the copy step disappear.

RunPod suits interactive work on a dedicated Pod, with a network volume that moves between Pods. Vast.ai's bidding lets you set your own price, and a paused instance keeps its disk on the same machine. If your job cannot checkpoint at all, an on-demand machine anywhere beats any recovery feature.

How do you turn it on with Nodus?

Write your state to /nodus/state, load it on start, and run:

pip install nodus-compute
nodus login
nodus run --gpu H100:8 --interruptible=allow --checkpoint /nodus/state --max-cost 550 -d -- torchrun --nproc_per_node=8 train.py

--interruptible accepts allow, prefer or never. The --max-cost cap stops the run at $550, a little above the $499.20 the 24-hour example costs at list price. nodus describe job/<name> shows each attempt, why it ended and the latest checkpoint.

FAQ

Does Nodus edit my command to add a resume flag?

No. Nodus restores files, never process memory, and never adds resume flags. A restored attempt sets NODUS_RESTORED=1, which your script can check before loading.

What happens when the run hits its `--max-cost` cap?

A checkpointed Nodus Job checkpoints and then suspends with reason MaxCostReached. It continues from that checkpoint when you raise the cap, and a Job stopped by an empty balance resumes on its own once it is funded again.

Does SageMaker copy checkpoints I upload to S3 mid-job?

No. Checkpoints already in S3 are copied into the container when the job starts, and files added to the S3 folder later are not. Deleting a checkpoint in the container also deletes it in S3.

How large can a Nodus checkpoint be?

Up to 1 TiB by default (maxSize: 1Ti), across up to 64 paths. After a Job finishes, Nodus keeps only its final checkpoint once seven days have passed.

Originally published on the Nodus blog.

DEV Community: Viswaretas Kotra

Nodus vs Google Cloud GPUs: which is cheaper for training?

How do Google Cloud and Nodus GPU prices compare?

What does the same training job cost on each?

Can you use both Google Cloud and Nodus?

Is Google Cloud Spot stable enough for training?

Where is Google Cloud the better choice?

You already have Google credits or commitments

Your data lives in Google Cloud Storage

You want TPUs

Compliance requires Google Cloud

How do you move a training job from Google Cloud to Nodus?

FAQ

Does work in my connected GCP project show up on my Google bill?

Which regions does Nodus use in a connected GCP project?

What happens if my GCP Spot VM is preempted mid-training?

Is Google Cloud's DWS Flex-start cheaper than Nodus?

How do Nodus and Vertex AI compare for managed training?

Can you use serverless GPUs for long training runs?

How long can a serverless GPU function run?

What happens when a long serverless run is interrupted?

How does Nodus keep a multi-day training run going?

What does a 30-hour H100 fine-tune cost on each option?

When is serverless the right place to train?

Where is Modal or RunPod the better choice?

How do you move a long training run to Nodus?

FAQ

Can I chain several Modal calls to train past 24 hours?

Does the 24-hour Modal limit include time waiting for a GPU?

How long does Nodus keep checkpoints after a run finishes?

Does a Nodus TrainingJob need my code to handle checkpoints?

What are the best serverless GPU platforms in 2026?

What makes a GPU platform serverless?

How do the main serverless GPU platforms compare?

Which serverless GPU platform should you pick for each workload?

Someone is waiting on the response

Traffic is steady all day

The job runs for hours

A batch job where nobody waits

You only need to call an open model

What does the same GPU work cost on each platform?

What happens to the bill when a long run is interrupted?

Can Nodus serve a latency-sensitive endpoint?

Where are Modal, RunPod and the others the better choice?

How do you move a long job from an endpoint platform to Nodus?

FAQ

Is a serverless GPU cheaper than a dedicated one?

Do you pay for cold starts on serverless GPUs?

Can you fine-tune a model on a serverless GPU platform?

What happens on Nodus if credit runs out mid-run?

What is the cheapest way to fine-tune Llama on an H100?

Which fine-tuning method fits Llama on one H100?

Is QLoRA cheaper than LoRA for Llama 3.1 8B on an H100?

What does a Llama fine-tune cost per finished run?

Why are LoRA checkpoints cheap enough to save every few minutes?

How do you run a Llama LoRA job on Nodus?

Where are Modal or RunPod the cheaper choice?

Which setup should you pick?

FAQ

Can I use a fine-tuned Llama commercially?

How long a context can I fine-tune a 70B Llama on with one 80 GB GPU?

Should I pick a B200 instead of an H100 for Llama fine-tuning?

Does Nodus pick the GPU if I leave resources out?

What are the best Together AI alternatives for fine-tuning?

Who are Together AI's main competitors for fine-tuning?

How does Together AI bill a fine-tuning job?

What does the same fine-tuning job cost per token and per GPU-hour?

How do you tell which side of the line your job is on?

Is Fireworks AI cheaper than Together AI for fine-tuning?

What happens when a fine-tuning run fails or is cut short?

Where is Together AI the better choice?

How do you move a Together AI fine-tuning job to Nodus?

FAQ

Can I connect my Together AI account to Nodus?

Does Together AI count my validation set in the bill?

Can I fine-tune with Unsloth instead of a managed recipe?

Can a coding agent start the fine-tune for me?

What infrastructure do background coding agents need?

What are the seven components, and which ones does Nodus cover?

Why does every task need its own sandbox?

Does Nodus pick the GPU if I leave `resources` out?