DEV Community

Runcrate
Runcrate

Posted on

Fine-tune an open model on one GPU with a job that stops itself ($0.05, 4 minutes)

#machinelearning #python #llm #tutorial

The expensive part of a small fine-tune is rarely the training. It's the GPU you rented for an afternoon, ran a 4-minute job on, and forgot to shut down.

This post runs a LoRA fine-tune of a small open model as a job: one command on a fresh GPU machine that stops itself, and its billing, the moment the command exits. The whole run below took 3 minutes 45 seconds of GPU time and cost $0.05. I ran it on 2026-10-21 and the output is copied from the logs.

What we're training

  • Model: Qwen/Qwen2.5-0.5B-Instruct, small enough to train in minutes
  • Data: 2,000 conversations from trl-lib/Capybara, with 5% held out for evaluation
  • Method: LoRA (rank 16, all linear layers) with TRL's SFTTrainer, 150 steps, bf16
  • GPU: one RTX A6000 (48 GB)

It's deliberately tiny so you can run it end to end for pocket change. For a real model you change the model name, the dataset and the step count; nothing else in the workflow moves.

The script

Save this as finetune.sh. It sets up a clean Python environment, writes the training script and runs it:

set -e
pip install -q uv && uv venv -q /opt/env && . /opt/env/bin/activate
uv pip install -q torch trl peft datasets --torch-backend=cu128
cat > /workspace/train.py <<'EOF'
import torch
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

data = load_dataset("trl-lib/Capybara", split="train[:2000]").train_test_split(test_size=0.05, seed=0)

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    train_dataset=data["train"],
    eval_dataset=data["test"],
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", task_type="CAUSAL_LM"),
    args=SFTConfig(
        output_dir="/workspace/qwen-capybara-lora",
        max_steps=150,
        per_device_train_batch_size=8,
        learning_rate=2e-4,
        bf16=True,
        logging_steps=25,
        eval_strategy="steps",
        eval_steps=75,
        save_steps=75,
        report_to="none",
    ),
)
print("eval loss before:", round(trainer.evaluate()["eval_loss"], 4))
trainer.train()
print("eval loss after:", round(trainer.evaluate()["eval_loss"], 4))
trainer.save_model("/workspace/qwen-capybara-lora")
print("saved adapter to /workspace/qwen-capybara-lora")
EOF
python /workspace/train.py
Enter fullscreen mode Exit fullscreen mode

Two lines in there cost me four failed runs, so here's why they're there:

  • A fresh virtualenv. Installing TRL into the machine's system Python dragged in packages that clashed with what the image already had (an old Jinja2 broke chat templates). A uv venv sidesteps all of it and takes seconds.
  • --torch-backend=cu128. The machine's NVIDIA driver supports CUDA 12.8. A plain pip install torch picked a newer CUDA build, which then refused the driver and fell back to CPU, and bf16 training failed. Match the PyTorch build to the driver (nvidia-smi shows it).

Those failed runs cost between zero and three cents each, because each job stopped as soon as the command failed. That's the point of running this as a job.

Run it

Install the CLI and sign in once:

brew install aeonmindai/tap/runcrate
runcrate login
Enter fullscreen mode Exit fullscreen mode

Then submit the job:

runcrate run --gpu A6000 --template ubuntu-train --max-runtime 40m --name qwen-lora -d -- "$(cat finetune.sh)"
Enter fullscreen mode Exit fullscreen mode 
0f122b1e-5e47-476c-9ed0-94990b8e7ccd
Job qwen-lora is starting. Follow it with: runcrate jobs logs -f 0f122b1e-5e47-476c-9ed0-94990b8e7ccd
Enter fullscreen mode Exit fullscreen mode
  • --max-runtime 40m is the ceiling. If the training hangs or loops, the job is stopped at 40 minutes. The clock starts when your command starts, not while the machine boots.
  • -d returns straight away with the job id. Leave it off and runcrate run streams the output until the job ends.
  • runcrate run exits with the job's own exit code (124 if it hit the max runtime), so you can chain it in scripts or CI.

Read the logs 

runcrate jobs logs 0f122b1e-5e47-476c-9ed0-94990b8e7ccd
Enter fullscreen mode Exit fullscreen mode

The interesting lines:

eval loss before: 1.7612
{'loss': '1.53', ... 'mean_token_accuracy': '0.638', 'epoch': '0.2101'}
{'loss': '1.425', ... 'mean_token_accuracy': '0.6525', 'epoch': '0.3151'}
{'loss': '1.346', ... 'mean_token_accuracy': '0.6669', 'epoch': '0.4202'}
{'loss': '1.455', ... 'mean_token_accuracy': '0.6505', 'epoch': '0.5252'}
{'loss': '1.507', ... 'mean_token_accuracy': '0.6433', 'epoch': '0.6303'}
{'train_runtime': '165.3', 'train_samples_per_second': '7.26', 'train_steps_per_second': '0.908', 'train_loss': '1.46', 'epoch': '0.6303'}
eval loss after: 1.4237
saved adapter to /workspace/qwen-capybara-lora
Enter fullscreen mode Exit fullscreen mode

Held-out loss went from 1.76 to 1.42 in 150 steps, which is what you'd expect from a first pass on chat data. Nothing to write home about as a model, but the loop works and it's the same loop at 7B.

What it cost 

runcrate jobs status 0f122b1e-5e47-476c-9ed0-94990b8e7ccd
Enter fullscreen mode Exit fullscreen mode 
  Status:       ● succeeded
  Exit code:    0
  Runtime:      3m45s
  Cost:         $0.05 ($0.65/hr)
  Created:      2026-10-21T01:08:23.404+00:00
  Started:      2026-10-21T01:12:35.312+00:00
  Ended:        2026-10-21T01:16:20.648+00:00
Enter fullscreen mode Exit fullscreen mode

About four minutes between submit and start went to getting the machine ready. Then 3 minutes 45 seconds of command, and the machine was gone. $0.65 an hour was the A6000 rate when I ran this; prices move, so check the live pricing before you pick a GPU.

Keeping the adapter

In this run the adapter was saved to the machine's /workspace and went away with the machine, which is fine for a smoke test and useless for anything else. Two ways to keep it:

  1. Attach a storage volume. With a volume attached, /workspace is the volume, and what you write there is saved back when the job ends, whatever its final status. The jobs docs cover it.
  2. Push to the Hugging Face Hub at the end of train.py with trainer.push_to_hub(), passing your token as an environment variable on the job rather than inside the command (commands and logs are visible to everyone in the workspace).

Scaling it up

The same command takes bigger hardware: --gpu H100, or several GPUs on one machine with --gpu-count. A job can run up to 24 hours. Past that, or when you want to sit on the machine with Jupyter or VS Code, use an instance with a max runtime and idle shutdown instead.

Or ask your agent

The Runcrate MCP server exposes jobs to Claude Code, Codex and Cursor, so "fine-tune with ./finetune.sh on an A6000, cap it at 40 minutes, and tell me the final eval loss" is a prompt that works. Setup is one command per client on the MCP setup page.

Links

If you try this with your own data and it breaks somewhere I didn't hit, tell me where. Reply here or write to hello@runcrate.ai.

Written with AI assistance. Every command and output above was run on 2026-10-21.

Top comments (0)