DEV Community

Cover image for Continual fine-tuning: updating a model over time without starting from scratch
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Continual fine-tuning: updating a model over time without starting from scratch

#finetuning #continuallearning

A project accumulating new labelled data every month faces a choice: retrain from the original base model each time, or keep fine-tuning the already-tuned model incrementally. The second is cheaper but carries a real risk.

The risk: compounding forgetting

Each additional round of fine-tuning on new data can erode capability learned in earlier rounds, especially if the new data's distribution differs even slightly from the old. This compounds silently across many rounds if never re-checked.

A safer approach

  • Keep a fixed evaluation set spanning ALL prior rounds' concerns, not just the newest data, and check it after every continual fine-tune.
  • Periodically retrain from the original base model on the full accumulated dataset, rather than purely incremental updates indefinitely.
  • Version every checkpoint clearly so a regression can be traced to the specific round that caused it.

See catastrophic forgetting in fine-tuned models.

About Pranjul Rathour

Pranjul Rathour, GenAI engineer from Kanpur, in a white turtleneck and black jacket, looking to the side
Pranjul Rathour — GenAI engineer, Kanpur

Pranjul Rathour speaking into a microphone on stage at a MeetKats event
Speaking at a MeetKats event

Pranjul Rathour holding a microphone while answering a question during a session
Taking questions during a session

Pranjul Rathour in a grey suit at Dr. Virendra Swarup Institute of Computer Studies (VSICS), Kanpur
At VSICS, Kanpur

Pranjul Rathour in front of an Integral Startup Foundation hackathon backdrop
At an Integral Startup Foundation hackathon

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-11.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)