What Is Fine-Tuning?
A pre-trained model learns from billions of tokens of general text. It develops broad language understanding. Fine-tuning takes that base model and trains it further on a smaller, task-specific dataset.
The result: a model that performs well on your specific use case, whether that is medical diagnosis, legal summarization, customer support, or code generation.
Fine-tuning is not training from scratch. You preserve general knowledge and adapt behavior.
How a Neural Network Learns
Before fine-tuning makes sense, you need to understand what happens inside a neural network during training.
A neural network is a system of layers. Each layer contains nodes (neurons). Each connection between nodes has a weight. These weights determine what the network outputs given any input.
At the start, weights are random. Training adjusts them so the network produces correct outputs.
Here is the full training loop:
Training Data
↓
Forward Pass
↓
Prediction
↓
Loss Calculation
↓
Backpropagation
↓
Gradient Calculation
↓
Weight Update
↓
Repeat
↓
Trained Model
Phase 1: Forward Pass
The model receives an input (a sentence, an image, a token). Data flows forward through every layer. Each neuron applies a mathematical function and passes its result to the next layer.
At the end, the model produces a prediction. For a language model, this is a probability distribution over the next token.
No learning happens here. This phase only produces output.
Phase 2: Loss Calculation
The model's prediction is compared to the correct answer. A loss function measures how wrong the prediction is.
Common loss functions:
- Cross-entropy loss: used for classification and language modeling
- Mean squared error: used for regression tasks
A high loss means the model is far off. A loss near zero means the prediction was close to correct.
The loss is a single number. It summarizes the error for one batch of data.
Phase 3: Backpropagation
This is where learning begins.
Backpropagation works backward through the network. It calculates how much each weight contributed to the total loss. The math tool used here is the chain rule from calculus.
Think of it as blame assignment. Each weight receives a gradient: a number that says "changing this weight by X changes the loss by Y."
A large gradient means that weight had a big effect on the error. A small gradient means it had little effect.
Phase 4: Gradient Calculation
After backpropagation, every weight in the network has a gradient.
The gradient is a vector. It points in the direction of steepest increase in loss. To reduce the loss, you move in the opposite direction.
Phase 5: Weight Update (Optimization)
An optimizer uses the gradients to update every weight.
The most basic optimizer is Stochastic Gradient Descent (SGD):
new_weight = old_weight - (learning_rate × gradient)
The learning rate controls the step size. Too large: the model overshoots. Too small: training takes too long.
Modern optimizers like Adam adjust the learning rate automatically for each weight. Adam tracks the history of gradients and adapts accordingly. It converges faster than basic SGD in most cases.
The Full Loop
One pass through the data is one epoch. Training repeats this loop thousands of times across many epochs. With each pass, the weights improve. The loss decreases. The model gets better at the task.
Types of Fine-Tuning
There are three main approaches. They differ in what gets updated during training.
1. Full Fine-Tuning
All weights in the model are updated.
You take the pre-trained model and run the training loop on your dataset. Every parameter is fair game. The optimizer adjusts all of them.
Advantages:
- Maximum flexibility
- Best performance ceiling for large datasets
Disadvantages:
- Requires significant GPU memory (storing the model, gradients, and optimizer states for billions of parameters)
- Expensive and slow
- Risk of catastrophic forgetting: the model overwrites general knowledge with task-specific patterns
- A 7B parameter model at full precision requires roughly 28 GB of GPU memory for weights alone, plus optimizer states
Full fine-tuning makes sense when you have large datasets and substantial compute.
2. PEFT (Parameter-Efficient Fine-Tuning)
PEFT is a family of methods. The core idea: freeze most of the model and only train a small number of parameters.
You add new, trainable components to the frozen model. Only those new components are updated. The original weights stay fixed.
This reduces:
- GPU memory required (no gradients for frozen weights)
- Training time
- Risk of catastrophic forgetting
PEFT methods include LoRA, prefix tuning, prompt tuning, and adapter layers. LoRA is the most widely used.
3. LoRA (Low-Rank Adaptation)
LoRA is the dominant PEFT method in 2024 and 2025.
The insight behind LoRA: weight updates during fine-tuning tend to have low intrinsic rank. You do not need to update a full weight matrix. You approximate the update with two much smaller matrices.
How it works:
For a weight matrix W of size (m × n), instead of updating W directly, LoRA adds two matrices:
- Matrix A: size (m × r)
- Matrix B: size (r × n)
Where r is the rank, typically 4, 8, 16, or 64. You only train A and B. The product AB approximates the full weight update.
During inference, the update is merged back: W_new = W + AB
Parameter count comparison:
A weight matrix of size 4096 × 4096 has 16,777,216 parameters.
LoRA with rank 8 adds two matrices: (4096 × 8) + (8 × 4096) = 65,536 parameters.
That is a 256x reduction in trainable parameters for that layer.
Advantages:
- Trains on consumer GPUs (a 7B model with LoRA fits on 8-16 GB VRAM)
- Multiple LoRA adapters for different tasks, swapped at inference time
- The base model stays frozen and reusable
Disadvantages:
- Slightly below full fine-tuning on performance in some benchmarks
- Requires choosing rank (r) and which layers to target
4. QLoRA (Quantized LoRA)
QLoRA combines quantization with LoRA. It pushes LoRA further in terms of memory efficiency.
What quantization does:
Standard model weights use 16-bit or 32-bit floating point numbers. Quantization converts them to lower precision: 8-bit, 4-bit, or even 3-bit integers.
A 7B model in 16-bit uses roughly 14 GB of memory. The same model in 4-bit uses roughly 3.5 GB.
QLoRA workflow:
- Load the base model in 4-bit precision (NF4 format, designed for neural network weights)
- Freeze all quantized weights
- Add LoRA adapters in 16-bit precision
- Train only the LoRA adapters
The gradients flow through the quantized model, through the LoRA adapters. Only the adapters are updated.
Double quantization: QLoRA applies a second round of quantization to the quantization constants themselves, saving an additional 0.5 GB on a 7B model.
Advantages:
- Fine-tune a 70B model on a single 48 GB GPU
- Fine-tune a 7B model on a single 8 GB consumer GPU
- Performance within 1-2% of full fine-tuning on most benchmarks
Disadvantages:
- Slower than LoRA (dequantization overhead during forward/backward pass)
- Quantization introduces approximation errors
Comparison Table
| Method | Trainable Params | VRAM (7B model) | Performance | Use Case |
|---|---|---|---|---|
| Full Fine-Tuning | 100% | 80+ GB | Highest | Large datasets, max compute |
| PEFT (general) | 0.1 - 1% | Varies | Good | Resource-constrained |
| LoRA | 0.1 - 1% | 16-24 GB | Near full | Most fine-tuning tasks |
| QLoRA | 0.1 - 1% | 6-10 GB | Near LoRA | Consumer GPU fine-tuning |
Choosing the Right Method
Start with QLoRA if you are on a single GPU with less than 24 GB VRAM. It lets you work with models up to 13B or 30B parameters.
Use LoRA if you have 24-40 GB VRAM and want faster training than QLoRA.
Use full fine-tuning if you have a multi-GPU setup, a large high-quality dataset (100k+ samples), and need maximum task performance.
Key Hyperparameters in Fine-Tuning
Learning rate: Typically much lower for fine-tuning than pre-training. Common range: 1e-5 to 3e-4. Too high and you destroy pre-trained representations.
Rank (r) in LoRA: Higher rank captures more complex updates but uses more memory. Start with 8 or 16.
LoRA alpha: Scales the LoRA update. Common setting: alpha = 2 × rank.
Epochs: Fine-tuning usually needs far fewer epochs than pre-training. 1 to 5 epochs on domain-specific data is typical.
Batch size: Larger batches give smoother gradients. Use gradient accumulation if your GPU limits batch size.
What Makes Good Fine-Tuning Data
Data quality matters more than quantity.
- 1,000 high-quality examples often outperform 100,000 noisy ones
- Format your data to match the task: instruction-response pairs for chat models, completions for base models
- Balance your dataset across categories to avoid the model over-indexing on one topic
- Remove duplicates; they waste training compute and bias the model
Summary
Neural networks learn by repeating a loop: forward pass, loss calculation, backpropagation, gradient calculation, weight update. Fine-tuning runs this loop on task-specific data starting from a pre-trained model.
Full fine-tuning updates everything. PEFT methods freeze most weights. LoRA approximates weight updates with low-rank matrices. QLoRA adds quantization to push memory requirements lower.
Your choice depends on your GPU, your dataset size, and your performance target.
Top comments (0)