How to Fine-Tune an LLM with Unsloth Studio

#ai #llm

Fine-tuning is the process of further training a language model on new data so it learns new information, behaviors, or style — updating the model's own weights, rather than just showing it information at the moment you ask a question.

There are two common ways to get an LLM to work with new information: RAG and fine-tuning. RAG (Retrieval-Augmented Generation) retrieves relevant information and feeds it to the model as context at the time of the query. Fine-tuning instead trains the model on new examples so that knowledge becomes part of the model itself.

In this video, I start a new series on fine-tuning, where I go over everything you need to know to fine-tune a model using Unsloth Studio.

Prepare Dataset

First, we need to source our data. For this video, I manually created a PDF with facts about the just-concluded 2026 FIFA World Cup. You could also use the Wikipedia page, but be mindful of noise in that data — you'll need to manually clean it so you don't end up training your model on noise instead of facts.

After cleaning your PDF, you can head to Claude or ChatGPT and ask for a multi-turn conversational dataset in the ChatML format:

[
  {
    "messages": [
      {"role": "system", "content": "..."},
      {"role": "user", "content": "..."},
      {"role": "assistant", "content": "..."}
    ]
  },
  {
    "messages": [
      {"role": "system", "content": "..."},
      {"role": "user", "content": "..."},
      {"role": "assistant", "content": "..."}
    ]
  }
]

Or take advantage of Unsloth Studio's Data Recipes and generate a ChatML dataset right from the application. I explain how to do this in depth in the video — the prompt and response schema I used are below.

Response Schema (JSON Array)

{
  "type": "array",
  "minItems": 5,
  "items": {
    "type": "object",
    "properties": {
      "role": {
        "type": "string",
        "enum": ["system", "user", "assistant"]
      },
      "content": {
        "type": "string"
      }
    },
    "required": ["role", "content"]
  }
}

Prompt

Generate a conversation from the chunk below: a system message, then exactly two user/assistant exchanges.

Start with a system message setting the assistant's role as a helpful assistant knowledgeable about the 2026 FIFA World Cup. Then the user asks a factual question about something in the chunk, and the assistant answers accurately. The user then asks a follow-up question that builds on the assistant's answer — asking for more detail, a related fact, or clarification — and the assistant answers that too.

Rules:
- Never invent facts not in the chunk. If the chunk is too narrow for a genuine follow-up, find a second distinct fact from the same chunk to ask about instead — do not fabricate details to fill the follow-up.
- Answers must be self-contained (no "the text says...").
- Use full names, not pronouns.
- After the system message, roles must alternate: user, assistant, user, assistant.

Output only a JSON array matching this schema — no wrapper object, no extra text:
[{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]

Chunk:
'{{chunk_text}}'

I used the unsloth/Qwen2.5-Coder-3B-Instruct-GGUF model for this step, since it reliably returned a valid JSON array. I tried the unsloth/gemma-4-E2B-it-GGUF model first, but it couldn't handle the JSON array output correctly.

Base Model

I used the unsloth/Llama-3.1-8B-Instruct model for two reasons:

It needs about 8.6GB of GPU memory, which fits comfortably within my machine's 32GB of unified memory (which serves as VRAM on Apple Silicon).
It's built for conversational chat, since it ships with a chat template out of the box.

I initially tried the base version, unsloth/Llama-3.1-8B, and ran into this error:

This page from Unsloth's documentation explains why: base models don't come with a chat template at all, while Instruct models do.

You can still fine-tune a base model for conversational chat, but you'll need to manually create a chat template for it first. Unsloth Studio doesn't yet support this out of the box — there's an open GitHub issue tracking it as a feature request, so hopefully it lands soon.

Hyperparameters

For hyperparameter configuration, check Unsloth's LoRA Hyperparameters Guide for guidance on getting optimal values for your training run.

One of the most important settings is eval_steps. Without it, evaluation never runs alongside training. I set mine to 0.1 in the video, meaning evaluation runs every 10% of the way through training. This gives you an eval loss graph, which shows how your model is actually performing — not just how well it fits its own training data.

Here's how to read it:

If the graph stays flat and never comes down, that's underfitting — the model isn't learning the patterns in your dataset.
If it comes down and then goes back up, that's overfitting — the model has stopped generalizing and started memorizing the training data instead.
What you want is the eval loss graph trending smoothly downward, alongside the training loss graph.

If you found this useful, drop the video a like and subscribe to the channel for more content.

Thanks, happy coding!

DEV Community