DEV Community

Cover image for Ten fine-tuning mistakes I see students make (and made myself)
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Ten fine-tuning mistakes I see students make (and made myself)

#finetuning #mistakes #students #guide

I built FineTune Studio after making most of these mistakes on my own runs. They are not exotic. They are the difference between "I fine-tuned a model" on a resume and "I fine-tuned a model and here is what improved by how much".

The mistakes

  1. No baseline. Not running the base model on the same evaluation set first. Without it, you cannot know whether the fine-tune did anything. Base-versus-tuned is the whole point.
  2. Training on unvalidated data. Malformed records, duplicates, and template mismatches. Run the pre-training checklist first.
  3. Leaking eval into training. Splitting after deduplication is not optional.
  4. Chasing training loss. It goes down whether or not the model is getting better. Watch eval loss, then read outputs.
  5. Too many epochs on too little data. A few hundred examples for ten epochs is memorisation with a progress bar.
  6. Choosing the biggest model that fits. A 1–3B model fine-tuned well beats a 7B model fine-tuned badly, and ships on hardware you have.
  7. Fine-tuning for facts. Knowledge belongs in retrieval; see RAG vs fine-tuning.
  8. Ignoring the chat template. Training with one format and serving with another produces a model that seems to have forgotten everything.
  9. Not saving checkpoints. The best model was at step 600; you only kept step 1200.
  10. No deployment plan. An adapter on a laptop is not a result anyone else can use. Decide how it will be served before you start.

The habit that prevents most of them

Write the evaluation before the training script: the held-out set, the rubric, and the baseline numbers. Everything else becomes an experiment against a fixed target instead of a hope. That habit, more than any hyperparameter, is what I try to teach in campus sessions on fine-tuning.

Make these mistakes once, on a small model, on a weekend. Then never again.

About Pranjul Rathour

Pranjul Rathour, GenAI engineer from Kanpur, in a white turtleneck and black jacket, looking to the side
Pranjul Rathour — GenAI engineer, Kanpur

Pranjul Rathour speaking into a microphone on stage at a MeetKats event
Speaking at a MeetKats event

Pranjul Rathour holding a microphone while answering a question during a session
Taking questions during a session

Pranjul Rathour in a grey suit at Dr. Virendra Swarup Institute of Computer Studies (VSICS), Kanpur
At VSICS, Kanpur

Pranjul Rathour in front of an Integral Startup Foundation hackathon backdrop
At an Integral Startup Foundation hackathon

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-06.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)