The vocabulary of machine learning has an unusual quality: it reads like gothic horror.
Catastrophic forgetting. Dying ReLU. Vanishing gradients. Mode collapse. Hallucination. Adversarial attacks. Superposition. The Bitter Lesson.
No other technical field has vocabulary this dramatic. Electrical engineering doesn't have "catastrophic forgetting." Statistics doesn't have "hallucination." These terms are hyperbolic by design — the researchers who named them were encoding their visceral experience of watching models fail.
Why the dramatic names?
The Bitter Lesson (Sutton, 2019) warns that human-crafted approaches will always lose to scale. "Bitter" is doing real work there. It's not neutral. It's a field processing grief.
"Catastrophic forgetting" doesn't just mean the model forgot something. It means the model destroyed what it knew when learning something new. The catastrophe is total. The naming reflects a specific horror that researchers kept running into — a model that couldn't retain its past.
"Dying ReLU" is more surgical: neurons that permanently stop activating. Dead weight in the network. A population of neurons that used to contribute and now never will.
The archive as memorial
OVERFITS started as a catalog. It became something closer to a memorial.
When you press "catastrophic forgetting" into fabric and give it a Latin motto (Memoriae excessus — excess of memory), you're acknowledging what the term is actually doing. It's encoding a failure mode that early ML researchers found genuinely disturbing. The specimen plate is saying: this happened. We saw it. We named it.
The dark academic aesthetic isn't ironic distance. It's the right register for concepts this dramatic.
The archive is at https://overfits.ai
Top comments (0)