Picking a fine-tuning method usually comes down to one thing, whether you can write the ideal output yourself or can only judge one when you see it. Here's a practitioner rundown of SFT, RFT with GRPO, distillation and LoRA/QLoRA, and how to choose between them for your own task.
Originally published at overmindlab.ai.
Supervised fine-tuning (SFT)
SFT is where most people start, and most people should. You give the model input/output pairs and train it until it reproduces that pattern. It's teaching by worked example: here's the prompt, here's the answer, do more of this.
You'll typically need a few thousand examples for a narrow task. The catch is that SFT only works if you know what "good" looks like.
LoRA and QLoRA
LoRA doesn't compete with SFT or RFT.
Full fine-tuning updates every weight in the model. That's slow, and it eats memory. LoRA freezes the base model and trains a small set of adapter weights on top, which gets you most of the benefit for a fraction of the compute. QLoRA pushes this further by quantising the frozen base model down to 4-bit before training the adapter. That's the reason a 7B model now fits on one consumer GPU instead of a rack of them.
Tools worth knowing:
Unsloth is open source and runs LoRA/QLoRA 2 to 5x faster with up to 80% less VRAM, using custom kernels and 4-bit quantisation. It's become the default starting point for solo builders.
Tinker, from Thinking Machines Lab, is a managed LoRA API. It handles GPU scheduling and checkpointing but still lets you control the actual training algorithm. It launched in October 2025 and supports both SFT and RFT.
Overmind is a model training platform. It runs LoRA fine-tuning on your agent traces.
Reinforcement fine-tuning (RFT)
This used to just be called "RL," or RLHF if you wanted to impress someone.
SFT needs you to write the correct answer. RFT only needs you to score it. You define a reward, hand it the model's output, and the model learns to produce completions that score higher across many attempts.
Reach for this when good is easier to recognise than to demonstrate. I can't write the perfect customer support reply off the top of my head, but I can tell you whether one resolved the ticket, stayed on brand, and didn't promise something we don't offer. RFT is also the natural fit for agents: tool calls, multi-step retrieval, anything where success depends on the whole trajectory and not any single message.
GRPO (Group Relative Policy Optimization) is the algorithm doing most of the RFT work right now. It came out of DeepSeekMath and got famous when it trained DeepSeek-R1. The trick is sampling a group of responses to the same prompt, then scoring each one against the group average instead of some absolute scale. That kills the need for a separate critic model, which is what made older methods like PPO so expensive to run. DAPO and Dr.GRPO are newer variants that patch specific instabilities GRPO runs into on long chain-of-thought outputs.
Distillation
Distillation is copying a big model's behaviour into a small one. Run your inputs through the big model, collect what it says, then train a smaller model to say the same things. You lose a little quality and gain a lot of speed, plus a much smaller bill.
This is helpful when a frontier model already nails your task but is too slow or too expensive to run at real volume. It's also a big part of why small language models have taken off this past year. Teams distill a specialist model instead of shipping the 400B-parameter original into production.
Picking one
Once you know your task, the decision is mostly mechanical.
| Method | What you must supply | When to pick it | Data volume | Relative cost |
|---|---|---|---|---|
| Supervised fine-tuning (SFT) | Input/output pairs you already know are right | You can write the ideal output yourself | A few thousand examples for a narrow task | Low |
| Reinforcement fine-tuning (RFT, usually GRPO) | A reward function or judge that can score an attempt | You can recognise a good output but can't write one, or success spans a whole agent trajectory | Prompts plus a scorer, no written answers needed | Highest, you sample many completions per prompt |
| Distillation | A teacher model that already does the task, plus your inputs | A frontier model nails the task but is too slow or too expensive at volume | As many teacher outputs as you can afford to generate | Medium, teacher inference then a cheap SFT run |
| LoRA / QLoRA | Whichever method above, plus one GPU | Almost always, unless you specifically need full-weight training | Same as the method it wraps | Lowest, a 7B run over a few thousand examples often costs a few hundred dollars |
The same tree in words.
| Your situation | Method |
|---|---|
| A big model already does the job, but it's too slow or too costly | Distillation |
| You can write the ideal output | Supervised fine-tuning |
| You can only judge the output, not write it | Reinforcement fine-tuning with GRPO |
| Any of the three, on one GPU | Run it through LoRA or QLoRA |
Data is still the hard part
Every method above needs the same raw material underneath. SFT needs labelled pairs. RFT needs a reward or a judge that can rank attempts. Distillation needs a teacher and your inputs. Pick the wrong method and you lose some efficiency. Show up with bad data and none of them work. No algorithm fixes that for you.
Fine-tuning has gotten cheap. A LoRA run on a 7B model over a few thousand examples often costs a few hundred dollars. But cheap training doesn't fix bad data, and none of these tools hand you a set of real runs labelled, or ranked, against your own definition of good. That's a separate problem, and it's the one that eats the time. We cover it in how do you turn traces into a training dataset and how to train your agent.
FAQ
Is LoRA a replacement for fine-tuning?
No. LoRA is a way to run fine-tuning or RFT cheaper, not a different goal. You still pick SFT or RFT first, then decide whether to run it through LoRA.
When should I use reinforcement fine-tuning instead of SFT?
When you can judge a good output but can't write one yourself, or you're training an agent where success depends on a whole multi-step trajectory rather than one response.
Is distillation cheaper than fine-tuning a small model from scratch?
Usually, if a large model already performs well on your task. You're paying for inference calls to the teacher model instead of building the target behaviour from labelled data by hand.
What is GRPO and why does everyone use it now?
Group Relative Policy Optimization scores a batch of responses against each other instead of an absolute baseline, which removes the need for a separate critic model. It's cheaper to run than older RL methods like PPO, which is most of why it's become the default for RFT.
Overmind is the model training platform for AI teams. It turns your production traces into specialised models you own. Get started.
Top comments (0)