You have a copy machine. You copy a document. The copy is slightly blurry. You copy the copy. It is blurrier. You copy it again. After ten generations, it is unrecognizable. This is model collapse. AI models are now training on data generated by previous AI models. The internet is filling with synthetic content. The next generation of models will train on that synthetic content. And the generation after that will train on the synthetic content of the synthetic content. The signal is degrading.
We are entering a dangerous feedback loop. AI is eating its own tail. And the result may be a slow, creeping decline in quality.
The Problem: The Internet Is Becoming Synthetic
Human-generated content is being diluted.
The Shift:
In 2020, most text was human-written.
In 2025, a significant fraction is AI-generated.
In 2030, most text may be AI-generated.
The Consequence:
Future models will train on data that is statistically similar to their own outputs.
The diversity of the training data will decrease.
The models will become more homogeneous.
A Contrarian Take: The Internet Was Always Synthetic.
We worry about AI-generated content. But human-generated content is also "synthetic" in a sense. It is filtered, curated, and biased.
The problem is not synthesis. The problem is degeneration. If the synthetic content is high quality, the feedback loop can be positive. If it is low quality, the feedback loop is negative.
The Mechanism of Model Collapse
Model collapse occurs through a series of generations.
Generation 1:
Trained on human-generated data.
Produces synthetic data.
Generation 2:
Trained on a mix of human and synthetic data.
Produces more synthetic data.
Generation 3:
Trained mostly on synthetic data.
Produces low-quality, repetitive output.
Generation N:
The model collapses into a narrow, degenerate state.
It loses nuance, diversity, and creativity.
A Contrarian Take: Model Collapse Is Not Inevitable.
Model collapse is a risk, not a certainty. It depends on the quality of the synthetic data. If the synthetic data is carefully curated, the feedback loop can be managed.
The problem is not synthetic data. The problem is unfiltered synthetic data.
The Degeneration Patterns
What actually degenerates?
- Diversity:
The model becomes less creative.
It produces similar outputs.
It loses the ability to generate surprising combinations.
- Nuance:
The model becomes less subtle.
It defaults to the average.
It loses the ability to capture edge cases.
- Factual Accuracy:
The model becomes less accurate.
It amplifies errors.
It hallucinates more.
- Language Quality:
The model becomes less fluent.
It uses simpler vocabulary.
It loses stylistic variety.
A Contrarian Take: Degeneration Is Not Uniform.
Some aspects degenerate faster than others. Language quality may degrade slowly. Diversity may degrade quickly.
The rate of degeneration depends on the model architecture, the training data, and the training regime.
Case Study: The LLaMA Experiment
Researchers trained a model on a dataset that was progressively more synthetic.
The Setup:
Generation 1: Trained on human data.
Generation 2: Trained on 50% human, 50% synthetic.
Generation 3: Trained on 90% synthetic.
The Results:
Generation 2 was slightly worse than Generation 1.
Generation 3 was significantly worse.
The model became repetitive and dull.
The Conclusion:
Synthetic data is not a substitute for human data.
The feedback loop is dangerous.
A Contrarian Take: The Experiment Was Flawed.
The researchers used low-quality synthetic data. They did not curate it. They did not filter it.
A well-curated synthetic dataset might produce better results. The experiment is a warning, not a verdict.
What You Can Do
You are not training a model. But you are consuming AI content.
- Support Human Content:
Read human-written articles.
Watch human-made videos.
Support human creators.
- Be Skeptical of Synthetic Content:
Ask: "Is this AI-generated?"
Be aware of the limitations.
- Demand Transparency:
Ask: "Is this content synthetic?"
Support labeling of AI-generated content.
- Advocate for Curation:
Synthetic data is not inherently bad.
It needs to be curated.
The Last Generation
The last generation is not the model. It is you.
You ask: "What is the future of AI?"
The model says: "The future depends on the choices we make today."
You realize: The future is not predetermined. It is a choice.
If the internet becomes mostly synthetic, how will you decide what to trust?
Top comments (0)