DEV Community

Cover image for Catastrophic Forgetting, Measured: I Fine-Tuned a Small LLM on One New Character and Its Old Stories Went From Perplexity 9 to 156
AI Explore
AI Explore

Posted on

Catastrophic Forgetting, Measured: I Fine-Tuned a Small LLM on One New Character and Its Old Stories Went From Perplexity 9 to 156

Part 4 of ten. Each post is a short, plain-words tour of one lesson from e=mc², a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site.

The most common thing anyone does with a language model is teach it something new, and almost nobody measures what that costs. Last month I took the 5.9-million-parameter storyteller from Lesson 01 of e=mc², the one that writes bedtime stories about Momo the monkey, and taught it a brand-new character. It learned her in about fifteen seconds. In the same fifteen seconds it became roughly seventeen times worse at the stories it already knew. That is catastrophic forgetting, and the point of Lesson 04 is to turn the scary name into a number you can watch move.

Where we are

Lessons 01 to 03 built Momo from random dials, read his attention heads, and watched a model start reciting its training stories. This lesson starts the second stage: taking a model that already works and changing it. The start is Lesson 01's public weights, perplexity 9.09 on 200 stories the model never saw. Perplexity is "between how many words is Momo still choosing". Lower is better.

A new student in Ella's class

Momo is the model, a head full of 5.9 million dials. Ella the elephant is the teacher and the memory: she holds the stories, reads them aloud, and tells Momo how wrong each guess was. In Lesson 01 she read him about 23,000 bedtime stories.

Now a new student joins her class. Zuri is a young zebra with a world of her own: a watering hole, a flat acacia tree, a little giraffe named Kofi, a baby hippo named Nia, and lessons about waiting your turn and telling the truth. I generated 400 unique Zuri stories (Momo never appears in them), kept 300 for teaching, 36,812 tokens in all, and held back 100 as a quiz.

Teaching Momo her stories is fine-tuning. Ella does not start again from random dials. She keeps teaching the same Momo, a few more nights, with new stories: 300 nights, each a batch of 20 windows of 256 tokens, turning the dials a third as far per step as Lesson 01 did, because trained dials should be nudged rather than thrown around.

The one idea: nothing gets deleted

Here is the thing I most want you to take away. When Momo forgets his old stories, no memory is erased. There is no drawer labelled "Lily and her balloon" that gets emptied to make room for Zuri.

There are only the 5.9 million dials, and the same dials do every job. Every Zuri lesson turns them a little toward Zuri. If no lesson ever asks them to keep telling the old stories, nothing turns them back. Night by night they drift, and the old stories get harder to read. Not because they were removed, but because the thing that used to read them has been quietly rebuilt for another purpose.

"Forgetting" is simply the name for the loss on old text going up. That is why it can be a number.

Ella's fix: replay

Once you see the mechanism, the fix is obvious. If the problem is that nothing asks the dials to keep the old job, then ask. In every batch of twenty windows, slip in two from the old training stories. That is replay, and it is the only difference between the two main runs in the lesson:

# One batch of 20 windows of 256 tokens. Run A: replay_windows = 0. Run B: replay_windows = 2 (10%).
xb, yb = windows(zuri_ids, BATCH - replay_windows, gen) # the new character's stories
if replay_windows:
 xr, yr = windows(replay_ids, replay_windows, gen) # Lesson 01's own training text
 xb, yb = torch.cat([xb, xr]), torch.cat([yb, yr])
_, loss = model(xb.to(device), yb.to(device)) # Lesson 01's model and loss, unchanged
Enter fullscreen mode Exit fullscreen mode

Run A: twenty new windows a night, no old ones. Run B: eighteen new, two old. Same seed, same steps, same everything else.

The numbers, honestly

Three quizzes, each watching one thing. (a) Lesson 01's hidden quiz, 200 TinyStories: does general storytelling survive? (b) 100 fresh Momo stories: does he keep his own character? (c) the 100 held-out Zuri stories: did he learn the new one? A cell compares every quiz story to every training story and stops the notebook on any overlap. All three counts were zero.

  • Before: 9.09 on the old quiz, 1.24 on Momo, 68.32 on Zuri. He has never heard of her.
  • Run A, Zuri only, 300 steps: 156.08 / 17.35 / 1.17. Her quiz fell from 68.32 to 1.20 within 50 steps. The old quiz went the other way: 53 at step 50, 87 at step 100, 156.08 at step 300 and still climbing. The Momo quiz rose fourteenfold.
  • Run B, two old windows in every twenty: 11.75 / 1.39 / 1.15. He learned Zuri just as well and kept nearly everything else. Ten percent replay removed 98.2% of Run A's perplexity rise on the old quiz and 99.1% on the Momo quiz.
  • 14 to 15 seconds per run on an Apple Silicon Mac with the GPU lightly shared.

Two honest footnotes. First, replay is not free of loss: Run B still finished 29% above where it started, 11.75 against 9.09, and even 25% replay, at 10.42, did not get all the way back. Second, the numbers near 1 are not understanding. The Zuri and Momo quizzes are new combinations of the training stories' sentences, and 300 stories read about forty times each are easy for 5.9 million dials to store. A perplexity of 1.15 means he recites those sentences well; one Zuri story the fine-tuned model wrote shares an unbroken run of 287 characters with a training story.

The full table, with the 5% and 25% replay rows, and a replay slider you can drag are on the lesson page.

What a number cannot tell you

The best finding came from reading, not measuring. Run B's Momo quiz, 1.39, says Momo stories are still easy to read. Then I gave it "Once upon a time, a little monkey", and it continued with

a little monkey named Zuri

and told a Zuri lesson. Given "Lily had a red balloon.", Run A's model dropped Lily at once and told a Zuri lesson about a baby bird. When the model has to choose a name after "a little monkey", the freshest lessons win. The quiz number was fine. The story was not. One number never tells you that; a sample does.

Why this is bigger than a zebra

None of this is new science. Replay and methods that protect important weights are the standard answers, and the papers are linked from the lesson. What is rare is to watch it happen in a model you can retrain in fifteen seconds, before and after side by side. Every fine-tune of every large model pays this bill somewhere.

Run B's dials are saved as tiny-storyteller-v2 in the lesson's Hugging Face repo: Momo who knows Zuri, old quiz at 11.75 instead of 156, because two old windows rode in every batch. It is the model the next lesson starts from.

Want to actually build it?

This post is the postcard. The lesson is the trip. Lesson 04 of e=mc² on theaivibe.org walks you through every piece with widgets you can poke, a notebook that runs on a free GPU in minutes, and the honest numbers at every step. No account, no paywall, nothing to install.

Tomorrow: Lesson 05 starts from tiny-storyteller-v2 and teaches it to follow a request instead of only continuing a story, which is instruction tuning from scratch.

Top comments (0)