I’ve heard about fine-tuning language models for a long time, but it always felt like something only big teams with expensive GPUs could do.
Turns out, you can do it for free using Google Colab. Here’s what I tried.
The Problem I Was Trying to Solve
In past We built an analytics chatbot. Users can ask questions like:
“How has our membership grown quarter by quarter?”
The app needs to figure out:
- Can this question be answered from the data?
- Does it need a chart or just a number?
- If it needs a chart, which type? (line, bar, pie, etc.)
At first, We used a general language model for this. But it was often wrong.
I tried improving it by adding more instructions and context, and it helped a bit. But the results stopped improving after a point. and it felt like couldn’t be pushed further.
So I thought:
What if I train a smaller model just for this task?
The Setup
I used:
- A tool that makes training faster and cheaper
- A small open model (around 2GB in size)
- Google Colab (free GPU)
For training, I used 418 examples.
Each example had:
- A user question
- The correct structured answer
Example:
Input:
How has our membership grown quarter by quarter since 2022?
Output:
A structured response saying:
- The question can be answered
- A chart is needed
- The chart type should be “line”
Training took about 20 minutes.
The Results
This part surprised me the most:
| Before | After | |
|---|---|---|
| Pass | 15 | 43 |
| Fail | 41 | 17 |
| No Output | 9 | 5 |
| Accuracy | 23% | 66% |
The same model improved from 23% to 66% accuracy.
Just by showing it ~400 examples of what I wanted.
What Didn’t Go Well
I ran into a few issues along the way.
Most of them were due to incorrect settings, which depend on the model you choose.
Sometimes the model also failed to load when not done the right way.
After fixing these, everything worked fine.
Exporting the Model
In the end, I converted the model into a format that can run locally.
Now it can be used in tools like local AI apps without needing heavy setup.
Code
You can check the full code here:
👉 https://github.com/prathameshks/LLM-Finetuning
Key Takeaways
- Training a small model for a specific task works really well
- You don’t need thousands of examples — 400+ was enough
- You can do this on free tools like Google Colab
- Small improvements in data can give big results
I’m planning to improve the dataset and see how far I can push the accuracy.
But for a first attempt, going from 23% to 66% with a small model feels like a solid win 🚀
Top comments (0)