DEV Community

Alex Chen
Alex Chen

Posted on

I Fine-Tuned a 7B Model for $0 Using Kaggle's Free GPU. Colab Wanted $50 for the Same Job.

I wanted to fine-tune Llama-3-8B on 4,000 of my own support-ticket conversations. Colab quoted me the upgrade screen: the free T4 kept dying with OOM, and Colab Pro is $11.99/month with no guarantee of an A100.

Then I remembered Kaggle gives away 30 GPU-hours per week, free, no credit card. P100 with 16GB VRAM. Here's the exact setup that worked — and the honest numbers.

Free GPU comparison (I tested all four this month)

Platform GPU Free quota Session limit OOM'd on 8B QLoRA?
Colab Free T4 (16GB) ~3h/day, throttled 12h Yes, constantly
Kaggle P100 (16GB) 30h/week 12h No
Colab Pro T4/A100 lottery $11.99/mo 24h Depends on lottery
Lightning.ai T4 15h/mo credits — Yes (16GB, tighter)

Kaggle wins on one boring detail: the P100's memory bandwidth (732 GB/s vs T4's 320) made each training step ~2.1x faster in my runs.

The actual code (QLoRA with unsloth)

The trick that makes 16GB enough: 4-bit quantization + LoRA adapters. You only train ~0.5% of the weights.

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/llama-3-8b-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
    model, r=16,
    target_modules=["q_proj","k_proj","v_proj","o_proj"],
    lora_alpha=16,
)

from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    train_dataset=my_dataset,  # 4,000 support conversations
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=8,
        max_steps=300,
        learning_rate=2e-4,
        fp16=True,
        output_dir="out",
    ),
)
trainer.train()
model.save_pretrained("my-support-model")
Enter fullscreen mode Exit fullscreen mode

Real numbers from my run

  • 300 steps on 4,000 examples: 2h 14m of GPU time (7.4% of my weekly free quota)
  • Peak VRAM: 14.2GB / 16GB — comfortable
  • Cost: $0.00. The equivalent A100 rental on Lambda: ~$2.70. Colab Pro to avoid OOM: $11.99.
  • Result quality: on 50 held-out tickets, the fine-tuned model's draft responses were rated "usable without edits" by me 34/50 times vs 11/50 for the base model. Not magic — but my support reply time dropped from ~6 min to ~2 min per ticket.

The honest downsides

  1. Kaggle sessions cap at 12 hours — fine for LoRA on small data, useless for full fine-tunes.
  2. You must verify a phone number to unlock the GPU quota.
  3. Persistence is awkward — download your adapter weights before the session dies. I lost a 90-minute run to this. Save checkpoints to /kaggle/working and download early.
  4. If your dataset is confidential, a shared free platform is the wrong place for it. Full stop.

The take

Everyone talks about needing A100s. For learning fine-tuning and shipping a genuinely useful LoRA adapter on a few thousand examples, 30 free hours a week is more GPU than most of us will use. The $50/month I almost spent on Colab Pro would have bought me nothing but impatience relief.

I generated the data-cleaning script (dedup + format into sharegpt style) with MonkeyCode — a free open-source AI coding tool — which saved me maybe an hour of pandas wrestling.

Have you actually fine-tuned anything on free GPUs, or does everyone just keep paying? What did you fine-tune for — and was it worth it?

Top comments (0)