DEV Community

Cover image for Fine-tuning on a laptop or a free GPU: a VRAM budget walkthrough
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Fine-tuning on a laptop or a free GPU: a VRAM budget walkthrough

#finetuning #vram #qlora #budget

The most common message I get about FineTune Studio is some version of "I only have a 4 GB card, is this hopeless?". It is not. Fine-tuning a 1.7B model at about 3.2 GB of VRAM is exactly what the platform does. The trick is knowing where the memory goes and cutting the parts that do not affect quality.

Where the memory goes

  1. Base weights — a 1.7B model in 16-bit is roughly 3.4 GB; in 4-bit it is under 1.2 GB. Quantising the frozen base is the biggest single saving.
  2. Adapters and their optimiser state — small by design; tens of megabytes at rank 16.
  3. Activations — grow with sequence length and batch size. This is the part you control most directly.
  4. Framework overhead — CUDA context and caches, a few hundred megabytes you cannot avoid.

Settings that fit in 4–6 GB

  • 4-bit base with double quantisation, adapters in bf16 or fp16.
  • Per-device batch of 1 with gradient accumulation of 16.
  • Gradient checkpointing on — slower steps, far smaller activations.
  • Sequence length capped at the 95th percentile of your data, not the model maximum.
  • A paged 8-bit optimiser so state can spill to CPU under pressure.

Free and cheap GPUs

Free notebook GPUs are enough for a first run on a small model, with two caveats: sessions end, so checkpoint to persistent storage every few hundred steps, and the disk is slow, so keep datasets small and pre-tokenised. FineTune Studio's three inference paths — local, a vLLM server, or a Hugging Face Space — exist because the machine you train on is rarely the machine you serve from.

What you give up, and what you do not

You give up speed: small batches and checkpointing make steps slower. You do not give up much quality for narrow tasks; QLoRA adapters on a quantised base track 16-bit LoRA closely. The honest way to confirm that is the base-versus-tuned comparison in how to evaluate a fine-tuned model honestly.

Hardware is a constraint, not an excuse. The students who ship fine-tuned models on borrowed GPUs learn more than the ones waiting for an A100.

About Pranjul Rathour

Pranjul Rathour speaking into a microphone on stage at a MeetKats event
Speaking at a MeetKats event

Pranjul Rathour holding a microphone while answering a question during a session
Taking questions during a session

Pranjul Rathour in a grey suit at Dr. Virendra Swarup Institute of Computer Studies (VSICS), Kanpur
At VSICS, Kanpur

Pranjul Rathour in front of an Integral Startup Foundation hackathon backdrop
At an Integral Startup Foundation hackathon

Pranjul Rathour in a checked shirt inside a packed college auditorium
In a packed college auditorium

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-06.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)