Can AI Teach Itself, or Does It Just Eat Itself?

Models that learn from their own answers can compound into something smarter or quietly rot. The difference comes down to one thing, and a cheap checker tips it the wrong way.

Train a model on its own output and one of two things happens. It gets better, the way a student improves by grinding practice problems and checking the answers. Or it gets worse, slowly, feeding on its own mistakes until the distribution caves in. Both happen, both are common, and the interesting part is the line between them.

We went looking for that line, and the first thing it told us was uncomfortable. A weak checker is not a small help. It is a net harm. In our runs, a verifier that was right only half the time did not just fail to improve the model. It dragged the model below where it started. Some signal is not better than none. Below a certain quality, the signal is the problem.

This is a field guide to that line: when a self-training loop compounds, when it collapses, and the failure modes that stay hidden in your average-accuracy number until it is too late. Fair warning, it goes deep. We built a controlled experiment, ran it, and every chart here is our own data rather than a redrawn version of someone else's. The payoff is a rule you can act on, but the point of the piece is to show you why it holds, not just to assert it, so settle in.

The half of the story everyone tells

You have heard the scary half. Train models on model-generated data and they degrade. The tails of the distribution thin out, the rare cases vanish, variance shrinks, and after a few rounds you have a confident, fluent, narrower model that has quietly forgotten things its parent knew. "AI eats itself" makes a good headline, and it does happen.

The half nobody puts next to it is that self-training also works, constantly, in production. Bootstrapping a model on its own correct solutions, filtering generations through a reward signal, having a model improve by checking its own work against a test suite. These are not exotic. They are how a lot of current capability gets built. The same basic move, a model learning from its own output, sometimes makes it stronger and sometimes rots it.

Both being true is not a contradiction. It means there is a variable doing the deciding, and the doom stories leave it out.

The variable is information, not effort

Here is the way to think about it that makes both outcomes obvious in hindsight.

A model trained on its own output cannot, by itself, learn anything it did not already contain. Generating more of your own answers and training on them is a closed loop. If nothing enters that loop from outside, you are reshuffling what you already had, and every imperfect step is slightly lossy, so the loop drifts downhill. That is collapse.

A loop compounds only when something enters it from outside the model's current distribution. There are three ways to let information in:

• a verifier, a checker that can tell a correct answer from a wrong one, so you keep your good outputs and drop your bad ones
• a real-data anchor, some genuine examples kept in the training mix, that the model cannot talk itself out of
• environment feedback, where the world tells you whether the action worked

These look like different techniques. They are the same thing: a channel that carries information the model does not already have. And the reach of the loop is capped by how much real information that channel actually carries. We will call that its fidelity. The whole "compound versus collapse" question reduces to one thing, whether your channel lets in enough real signal to outrun the lossiness of the loop.

I think this is the most useful lens on self-training there is, because it makes both outcomes obvious instead of surprising, and it tells you exactly the experiment to run. Hold everything fixed, change only the channel, and watch what happens.

The setup

We trained a small transformer from scratch to do two-digit arithmetic, addition, subtraction and multiplication. Small and from scratch matters: arithmetic has a free, perfect ground truth, so we can measure quality exactly and we are not smuggling in knowledge from some pretraining corpus. The model is the only thing learning, and we control its whole world.

We get it to a starting accuracy of about 80%, then run self-training loops that differ in one thing only, the information channel:

• verifier(q): keep the model's own answers that a verifier marks correct, where the verifier is right a fraction q of the time. q = 1.0 is a perfect checker; q = 0.5 is a coin flip.
• anchor(r): no verifier at all, but mix a fraction r of real, correct examples into each round.
• self-judge: the model decides for itself which of its answers to trust, by majority vote across several tries. No outside truth.
• replace: train on all of its own output, unfiltered, with no real data. The pure closed loop.

Then we just watch accuracy over rounds, and measure one more thing each round: the fraction of the training labels that were actually correct, however that correctness was sourced. That is the channel's measured fidelity.

The whole thing, the code and the data behind every chart below, is on GitHub, so you can rerun any of it yourself: github.com/Production-Grade/self-training-information-channel.

Result one: the source doesn't matter, the fidelity does

Plot final accuracy against measured fidelity and every channel lands on one curve.

A verifier and a real-data anchor are not two different strategies with two different ceilings. They are interchangeable ways of buying the same thing. A verifier that lets through 90%-correct data and an anchor that produces 90%-correct data land in the same place. The closed loop with no channel sits at the bottom, collapsing. Accuracy tracks fidelity, and it does not care where the fidelity came from.

You can even read off the exchange rate between the two. In our runs:

• a verifier right 75% of the time bought what roughly 38% real data bought
• a verifier right 50% of the time bought what roughly 10% real data bought

And the rate is convex, which is the practical surprise hiding in it: a little real data goes a long way. A small anchor of genuine examples does most of the work of a decent verifier, and it never needs you to build the verifier.

Result two: a weak verifier is worse than nothing

Now the uncomfortable one. Because the channels all sit on a fidelity curve, a low-fidelity channel does not just help less. It can push you below where you started.

Our model began at about 80%. A perfect verifier compounded it up to around 88% and then plateaued. A 75%-accurate verifier still helped, settling around 83%. But a coin-flip verifier, right half the time, dragged the model down to 78%, below where it started. It would have been better to do nothing.

This is the thing the "any signal helps" instinct gets wrong. There is a break-even fidelity. Above it, the loop compounds. Below it, the loop is actively corrosive, because you are now training on a set that is barely better than random, and the loop's own lossiness finishes the job. A cheap reward model or a sloppy LLM-as-judge that you reach for because "it is better than nothing" can be the reason your model gets worse. Below break-even, your verifier is not a safety net. It is the hole.

Result three: this is what collapse actually looks like

The pure closed loop, training on its own unfiltered output, does not fail all at once. It spirals.

Each round, the model generates its training data, that data is slightly worse than the last round's, it trains on the worse data and gets slightly worse, so next round's data is worse again. We watched the fidelity of the self-generated training set fall from 77% to 69% over a handful of rounds, with accuracy sliding alongside it from 78% to 71%. There is no cliff. It is a slow, self-reinforcing decline, which is exactly why it is dangerous: by the time it is obvious, several rounds of it are baked in.

And it does not take from everywhere evenly. It eats your hardest cases first. When we split the test set into easy and hard problems, the collapsing model held onto the easy ones (75% accuracy) while the hard ones rotted faster (68%). Your average accuracy, dominated by the easy majority, looks merely soft. Underneath it, the model's competence on the long tail is falling out from under you, and the single number on your dashboard will not show it until it has spread.

Result four: you cannot grade your own homework upward

The self-judge channel is the one people hope works, because it needs nothing from outside. The model proposes answers, the model picks which to trust, the model trains on its own picks.

It went nowhere. It held steady at almost exactly its starting competence and never climbed past it. That is the ceiling you would expect once you see the loop for what it is. A model judging its own work cannot inject information it does not have. Its judgement is only as good as it already is, so it can keep itself honest, but it cannot pull itself up. Self-consistency is a stabiliser, not an engine.

The operating rules

If you are running, or about to run, a self-training or synthetic-data loop, the whole thing collapses to a short checklist.

1. Name your channel. What is letting real information into this loop: a verifier, retained real data, an environment? If the honest answer is "nothing," you are in the closed loop and you will collapse. Add a channel before you add iterations.
2. Measure its fidelity, and stay above break-even. Do not assume a weak checker helps. A verifier below the break-even line makes your model worse than leaving it alone. If you cannot establish that your verifier clears the bar, a small real-data anchor is the safer buy.
3. Accumulate, do not replace. Keep real data in the mix every round. A little goes a long way, and it is the cheapest insurance against the spiral there is.
4. Watch coverage, not just the average. Collapse hides in your hardest cases first. Track accuracy on the long tail, or on a frozen hard set, because the headline number will lie to you for several rounds.

What actually travels

The numbers here come from a deliberately small model on a clean, synthetic task, where the ground truth is free and the loop has no confounds. Treat them as illustrative, not as constants to paste into a 70B run. What survives the jump in scale is not the exact 80% or the precise break-even point, it is the shape: the same closed loop, the same three channels, the same floor below which a checker starts to hurt instead of help.

And the shape is enough to act on. Whether your signal comes from a verifier, a slice of real data, or a real environment, you are running the same loop, and its fate is set by how much real information you put into it each round. I'm not certain where the break-even lands at frontier scale, and I'd be wary of anyone who quotes you a universal number, but I think the direction is the part that is safe to build on: there is a floor, and a cheap checker can sit under it.

Two things are worth holding on to. Compounding is bounded, so do not expect even a perfect channel to climb forever; ours plateaued after a few rounds and yours will too. The failure mode is not bounded, and it is quiet. The model that hurts you is not the one that obviously breaks. It is the one getting slowly, confidently narrower while your average accuracy says everything is fine.

So before you point a model at its own output, answer one question. What is letting new information into this loop, and is it good enough to be worth it? If you cannot answer, you are not self-improving. You are recycling, and recycling loses a little on every pass.