DEV Community: Tyler Edwards

What are the different types of fine-tuning?

Tyler Edwards — Tue, 01 Sep 2026 15:56:43 +0000

Picking a fine-tuning method usually comes down to one thing, whether you can write the ideal output yourself or can only judge one when you see it. Here's a practitioner rundown of SFT, RFT with GRPO, distillation and LoRA/QLoRA, and how to choose between them for your own task.

Originally published at overmindlab.ai.

Supervised fine-tuning (SFT)

SFT is where most people start, and most people should. You give the model input/output pairs and train it until it reproduces that pattern. It's teaching by worked example: here's the prompt, here's the answer, do more of this.

You'll typically need a few thousand examples for a narrow task. The catch is that SFT only works if you know what "good" looks like.

LoRA and QLoRA

LoRA doesn't compete with SFT or RFT.

Full fine-tuning updates every weight in the model. That's slow, and it eats memory. LoRA freezes the base model and trains a small set of adapter weights on top, which gets you most of the benefit for a fraction of the compute. QLoRA pushes this further by quantising the frozen base model down to 4-bit before training the adapter. That's the reason a 7B model now fits on one consumer GPU instead of a rack of them.

Tools worth knowing:

Unsloth is open source and runs LoRA/QLoRA 2 to 5x faster with up to 80% less VRAM, using custom kernels and 4-bit quantisation. It's become the default starting point for solo builders.

Tinker, from Thinking Machines Lab, is a managed LoRA API. It handles GPU scheduling and checkpointing but still lets you control the actual training algorithm. It launched in October 2025 and supports both SFT and RFT.

Overmind is a model training platform. It runs LoRA fine-tuning on your agent traces.

Reinforcement fine-tuning (RFT)

This used to just be called "RL," or RLHF if you wanted to impress someone.

SFT needs you to write the correct answer. RFT only needs you to score it. You define a reward, hand it the model's output, and the model learns to produce completions that score higher across many attempts.

Reach for this when good is easier to recognise than to demonstrate. I can't write the perfect customer support reply off the top of my head, but I can tell you whether one resolved the ticket, stayed on brand, and didn't promise something we don't offer. RFT is also the natural fit for agents: tool calls, multi-step retrieval, anything where success depends on the whole trajectory and not any single message.

GRPO (Group Relative Policy Optimization) is the algorithm doing most of the RFT work right now. It came out of DeepSeekMath and got famous when it trained DeepSeek-R1. The trick is sampling a group of responses to the same prompt, then scoring each one against the group average instead of some absolute scale. That kills the need for a separate critic model, which is what made older methods like PPO so expensive to run. DAPO and Dr.GRPO are newer variants that patch specific instabilities GRPO runs into on long chain-of-thought outputs.

Distillation

Distillation is copying a big model's behaviour into a small one. Run your inputs through the big model, collect what it says, then train a smaller model to say the same things. You lose a little quality and gain a lot of speed, plus a much smaller bill.

This is helpful when a frontier model already nails your task but is too slow or too expensive to run at real volume. It's also a big part of why small language models have taken off this past year. Teams distill a specialist model instead of shipping the 400B-parameter original into production.

Picking one

Once you know your task, the decision is mostly mechanical.

Method	What you must supply	When to pick it	Data volume	Relative cost
Supervised fine-tuning (SFT)	Input/output pairs you already know are right	You can write the ideal output yourself	A few thousand examples for a narrow task	Low
Reinforcement fine-tuning (RFT, usually GRPO)	A reward function or judge that can score an attempt	You can recognise a good output but can't write one, or success spans a whole agent trajectory	Prompts plus a scorer, no written answers needed	Highest, you sample many completions per prompt
Distillation	A teacher model that already does the task, plus your inputs	A frontier model nails the task but is too slow or too expensive at volume	As many teacher outputs as you can afford to generate	Medium, teacher inference then a cheap SFT run
LoRA / QLoRA	Whichever method above, plus one GPU	Almost always, unless you specifically need full-weight training	Same as the method it wraps	Lowest, a 7B run over a few thousand examples often costs a few hundred dollars

The same tree in words.

Your situation	Method
A big model already does the job, but it's too slow or too costly	Distillation
You can write the ideal output	Supervised fine-tuning
You can only judge the output, not write it	Reinforcement fine-tuning with GRPO
Any of the three, on one GPU	Run it through LoRA or QLoRA

Data is still the hard part

Every method above needs the same raw material underneath. SFT needs labelled pairs. RFT needs a reward or a judge that can rank attempts. Distillation needs a teacher and your inputs. Pick the wrong method and you lose some efficiency. Show up with bad data and none of them work. No algorithm fixes that for you.

Fine-tuning has gotten cheap. A LoRA run on a 7B model over a few thousand examples often costs a few hundred dollars. But cheap training doesn't fix bad data, and none of these tools hand you a set of real runs labelled, or ranked, against your own definition of good. That's a separate problem, and it's the one that eats the time. We cover it in how do you turn traces into a training dataset and how to train your agent.

FAQ

Is LoRA a replacement for fine-tuning?

No. LoRA is a way to run fine-tuning or RFT cheaper, not a different goal. You still pick SFT or RFT first, then decide whether to run it through LoRA.

When should I use reinforcement fine-tuning instead of SFT?

When you can judge a good output but can't write one yourself, or you're training an agent where success depends on a whole multi-step trajectory rather than one response.

Is distillation cheaper than fine-tuning a small model from scratch?

Usually, if a large model already performs well on your task. You're paying for inference calls to the teacher model instead of building the target behaviour from labelled data by hand.

What is GRPO and why does everyone use it now?

Group Relative Policy Optimization scores a batch of responses against each other instead of an absolute baseline, which removes the need for a separate critic model. It's cheaper to run than older RL methods like PPO, which is most of why it's become the default for RFT.

Overmind is the model training platform for AI teams. It turns your production traces into specialised models you own. Get started.

"So, you have observability. Now what?"

Tyler Edwards — Tue, 01 Sep 2026 13:46:47 +0000

By Tyler Edwards, co-founder and CEO at Overmind. If you run an agent in production, odds are you already have traces piling up in Langfuse or LangSmith. This piece is about the part nobody instruments: what those traces are for.

Observing your agent is table stakes.

In the not-so-distant past there was a brief period when picking an LLM meant picking a vendor. You wrote against their SDK, you paid their bill, and when their servers fell over on a Tuesday afternoon so did yours. Switching was easy enough but did require Ctrl+F'ing your way through the codebase - or asking your coding agent of choice to perform this insultingly simple task for you instead.

Then on the sixth day came the LLM gateway! One endpoint, dozens of models behind it, swap them in and out faster than the UK swaps Prime Ministers (and for international readers here is the context on that one). The gateway is, on its own terms, a genuinely nice piece of engineering, and there is now a number of teams shipping model routers with various bells and whistles: Portkey, LiteLLM, OpenRouter, TrueFoundry, Kong, Cloudflare.

Sitting alongside the gateway, usually paid for separately and instrumented by a different team three weeks later, is the LLM observability stack. Langfuse, Helicone, Arize Phoenix, Braintrust, LangSmith, Datadog now too. Pick your favourite, they mostly do the same thing. They capture every prompt and every completion, stitch them into traces, and present you with dashboards.

Between the two, your AI product is being watched harder than a Premier League VAR decision.

What Is Observability?

LiteLLM shipped to solve a pretty annoying problem: every provider's API was almost-but-not-quite the same. Portkey and OpenRouter commercialised the idea, TrueFoundry and Bifrost followed, and the existing API-gateway crowd (Kong, Cloudflare) extended their products to cover LLMs the moment it became clear LLMs were here to stay whether anyone liked it or not.

The observability side has a similar shape. Langfuse, Helicone, Arize Phoenix, Braintrust all turned up in 2023 to build native LLM observability from scratch. The companies that already did ML observability - Fiddler, WhyLabs, Galileo - pivoted with the arrival of ChatGPT and have spent the last three years insisting they were actually always about LLMs really anyway.

By 2026 both categories had matured enough to start being acquired. Portkey went to Palo Alto Networks in April for somewhere around $120-140M. Helicone went to Mintlify the month before. Langfuse was rolled into ClickHouse's Series D in January. Cisco picked up Galileo and folded it into Splunk.

All Data, No Insight

So you now have hundreds of thousands, possibly millions, of traces composting nicely in a database somewhere. Every prompt, every completion, every tool call, every retry. The gateway has them as individual requests. The observability tool has them stitched into trajectories. Between the two, your AI product is the most thoroughly documented thing in your engineering org.

The gateway, by design, treats each call as a self-contained unit - the only way to route, cache, fall back and bill at scale. It doesn't have a concept of the trajectory because it doesn't need one. As far as the gateway is concerned, each call was a separate transaction.

The observability tool does have a concept of the trajectory. It will stitch the spans together, draw you a lovely waterfall diagram, and tell you exactly how the agent arrived at whatever it arrived at. What it won't tell you is whether what it arrived at was any good - because good is a judgement, and the observability tool is, as its name suggests, there to observe.

Every call along the way can pass every check either tool runs. A 200 from the provider, the right shape of completion, no PII, latency inside SLO, and cost on budget. The agent can still invent a company policy, ignored an instruction from two steps earlier, and make three perfectly correct-looking calls on its way to deleting a production database - and then gleefully reported its action back to you in the same way a dog might excitedly drop a dead bird at your feet.

Where Observability Fails

The observability vendors aren't naive about any of this. Most of them ship some version of trace review - LangSmith has annotation queues, Braintrust has human review alongside LLM-as-judge scoring, Langfuse has dataset workflows, Arize Phoenix ships evals out of the box. Teams use them. Engineers were spending afternoons grading traces and writing scoring functions and tagging failure modes, and the tools make this less painful than it would otherwise be.

What you get at the end of all that effort is a graded slice of traces. A few thousand examples marked good or bad, often with structured scores attached, sometimes with notes. This is useful - for catching regressions, for spotting drift, for arguing with your model provider about why their new release broke something. It is not, on its own, a training or eval dataset, and it is several steps short of a model that has actually improved.

LLMs are intelligently dumb, they will learn an approximation of what you teach them, as such you need to teach them exactly what you want them to do. A graded trace is an observation. A dataset is a curated, balanced, deduplicated set of examples organised around the behaviours you want the model to learn. The difference between them is curation logic, clustering, sampling strategy, and a clear view of what the model is supposed to get better at - none of which the observability tool produces.

And even once you have the dataset, the pipeline from there to a better agent has its own shape. You need to fine-tune a model or train a LORA adapter, eval-gate the result against a held-out slice, deploy the new weights behind your routing layer, and watch the next week of traces to see whether the change actually moved the agent's behaviour or just shuffled the failure modes around. Each of those steps is its own piece of infrastructure.

Bridging The Gap

Some of the vendors have started shipping bolt-ons aimed at closing parts of this. Portkey's Autonomous Fine-Tuning wires gateway logs into a provider's training API and hands off the dataset. LangChain's Engine edits prompts and code from production traces. Braintrust's Loop generates evaluators and datasets from natural-language descriptions of failure modes. Each is a real move in the right direction, and each stops one step short of the model itself. The dataset gets handed off and the trail goes cold.

This is the gap between what's possible and what's shipped. The technology to take a production trace, score it, cluster it, curate it into a dataset, fine-tune a model on it, eval-gate the result and deploy it back through your routing layer - all of that exists, in pieces, sold by different companies with different pricing pages. What doesn't exist yet is a single layer that does the whole loop. You can probably see where i’m going with this but lets not jump ahead.

The Opportunity

Cursor is the example everyone reaches for, and for good reason. They trained their Composer model on production agent traces using online reinforcement learning, and it now beats the frontier at its size on cost and performance. The model that ships to Cursor users is the model trained on what Cursor users actually do. The labelling encodes the judgement. The fine-tune encodes the labelling. The product encodes the fine-tune. It compounds to create a very powerful product.

Doing this used to require a research team and the patience of an academic institution. It doesn't anymore. Open-weights bases - Llama, Qwen, Mistral, DeepSeek - are now genuinely competitive on scoped tasks. A LoRA fine-tune on a few thousand labelled trajectories runs for low hundreds of dollars on Modal or Together or your own GPUs if you're feeling brave. Unsloth and Tinker have made the mechanics broadly accessible to anyone willing to spend a weekend with them, with the small caveat that you do still currently have to be a bit of a genius to use them properly.

What this adds up to is something most teams haven't fully reckoned with yet. The data is sitting in the gateway. The traces are sitting in the observability tool. The curation can be automated. The fine-tuning is cheap. The deployment is a config change. The only thing standing between any reasonably-sized team and a continuously-improving fleet of specialist models trained on their own production traffic is whether anyone has joined the pieces up.

Overmind

And we arrive at the aforementioned point. We built Overmind on the same bet we wrote about in Think Smaller: the teams who win the next phase of agentic AI will be the ones who own their models, not the ones who rent the most powerful ones, and certainly not the ones who own the most expensive plumbing in front of those rented models.

Overmind is the layer that turns the data your gateway and observability stack are already collecting into models that improve because of it. It plugs into the trace store you've already got, runs the curation, builds the dataset, fine-tunes the model, eval-gates the result and deploys the new weights behind your existing routing layer. The gateway keeps doing what it's good at. The observability stack keeps doing what it's good at. Overmind does the bit in between that nobody else is doing.

Keep your gateway. Keep your observability stack. Just stop pretending they're enough on their own.

FAQ

What's the difference between an LLM gateway and LLM observability? Gateways route and rate-limit calls between your app and model providers. Observability tools record traces and let you inspect them. Neither curates the data or trains models.

Why isn't observability enough for production agents? Observability shows you what happened. It doesn't tell you whether the trajectory was good, and it doesn't turn that judgement into model improvements. That gap is the bottleneck.

What does Overmind do that observability tools don't? Overmind sits on top of your trace store, curates the data into training-grade datasets, fine-tunes specialist models, evaluates them, and deploys the new weights behind your existing routing layer.

Tyler Edwards is Co-Founder and CEO of Overmind. He writes about agent infrastructure, fine-tuning, and what it takes to ship AI that actually improves in production. Connect on LinkedIn.

Stay in-the-loop

Join our newsletter to follow as we build supervision for super-intelligence.

If your traces should end as a model you own rather than a dashboard, that loop is what Overmind does: audited datasets from production traces, fine-tuning, a benchmark against the model you run today, and serving. There is a live Langfuse import if your traces already live there.