Four Kinds of Recursive Self-Improvement: What Exactly Is Improving Itself?

Short answer: "Recursive self-improvement" (RSI) now covers at least four different things. The useful question is not whether a system improves itself, but what it improves: the harness around a model, the model's weights, the evaluator that scores it, or the research process that builds it. Each level ships a different artifact, fails in a different way, and needs a different kind of proof.

A 2026 survey of roughly 1,250 RSI papers (arXiv 2607.07663) organizes the field along two axes: what the system improves, and how closed the loop is. This post applies the first axis and turns it into a comparison you can use when reading the next "self-improving AI" claim.

The comparison table

	Harness-level RSI	Model-level RSI	Evaluator-level RSI	Research-level RSI
What changes	Prompts, tools, memory, control flow, agent code	The model's weights	The reward model / judge	Training methods, algorithms, experiments
Analogy	Rewriting an employee's manual	The employee gets smarter	The examiner gets sharper	The lab improves how it does research
Examples	Google's RRSI, Darwin Gödel Machine	TTRL, R-Zero, Self-Rewarding LMs, STaR, Darwin-RSI	Self-improving LLM judges and reward models	AlphaEvolve, autonomous post-training loops
What ships	A better configuration; the model is unchanged	A new model file	A better scorer	A new method or algorithm
Portability	The harness must travel with the model	The model alone carries the gain	The scorer must be used with it	The method must be re-applied
Main cost	Many calls to a strong external model (proposer, critic)	GPU training, once per round	Moderate	Very high
Typical failure	Overfits the evolve set (memorizes tasks)	Reinforces its own mistakes ("bad teacher")	Reward hacking: the scorer gets fooled	Hard to verify at all
Proof that convinces	Gains on benchmarks never used in selection	Gains on held-out benchmarks, paired tests	Agreement with independent human or ground-truth checks	Independent reproduction

1. Harness-level RSI: improve everything around the model

The model is frozen. An outer loop proposes edits to prompts, tools, memory and control flow, evaluates them, and keeps the winners.

Google's RRSI (Regularized RSI of Agent Harnesses, arXiv 2609.24972) is the clearest recent example. With a frozen policy it reports Terminal-Bench 2.1 going from 74.2 to 80.2, and gains of +2 to +5 points on benchmarks that never entered selection (SWE-bench Verified, GDPval, APEX-Agents, Frontier-Eng). With a weaker frozen model the in-distribution gain is larger (64.6 → 78.7).

The interesting part is not the loop but the regularizers that stop it from overfitting:

• Noise-adjusted floor: a gain inside evaluation variance is not a gain.
• Leakage critic: candidates are screened for benchmark-specific logic before evaluation.
• Edit history: a falsified hypothesis is not proposed again.
• Cost rule: extra inference tokens must be paid for by measured gain.
• Pruning: components that stop helping are removed.

These rules generalize well beyond harnesses. Any self-improvement loop without them will eventually report gains that are noise or leakage.

2. Model-level RSI: the model itself improves

Here the weights change. The model solves problems, a check decides which of its own solutions were good, and it trains on those. Repeat.

The key design question is where the training signal comes from. In the label-free family (TTRL, R-Zero and our own Darwin-RSI), the model learns only from its own solutions; no human-written solutions or reasoning traces are used, and correctness is checked automatically — for example by agreement across the model's samples or by executing code.

Results in this family are real but modest per round:

System	Setup	Reported gain
R-Zero (ICLR 2026)	Qwen3-4B-Base, challenger/solver co-evolution	+6.49 math, +7.54 general reasoning
Darwin-27B-RSI	27B, two rounds of self-improvement	GPQA Diamond 72.85 → 78.09 (single sample), 79.80 → 83.59 (majority@16), same protocol for both

The characteristic failure is the bad teacher: if the check that selects "good" solutions is weak, the model trains on its own errors and gets confidently worse. The quality of the check is the ceiling of the whole loop.

Why this level matters in practice: the gain ships inside the model file. Anyone who downloads the model gets the improvement without the loop, the harness, or the checker.

3. Evaluator-level RSI: improve the judge

Every other loop depends on a scorer. If the scorer improves, every loop that uses it improves too. If the scorer is fooled, every loop optimizes the wrong thing.

This level is under-discussed and under-measured. The practical test is simple: does the improved judge agree more with independent ground truth on items it never trained on? If you cannot answer that, you have a reward-hacking risk, not an evaluator improvement.

4. Research-level RSI: improve how improvement is done

The system modifies the research process itself: search over algorithms, training recipes, or entire post-training pipelines. AlphaEvolve is the best-known example of evaluator-guided evolution applied to algorithms and AI training infrastructure. Autonomous post-training loops that run for weeks without a human are now reported on public leaderboards.

This is the most ambitious level and the hardest to verify. Independent reproduction is the only convincing evidence.

How the levels combine

The levels are complementary, not competing. A harness-level loop can run on top of a model-level RSI model. A better evaluator strengthens both. A reasonable roadmap for a lab is:

1. Model-level RSI to get a stronger base that ships as a file.
2. Harness-level RSI on top of it for task-specific gains.
3. Invest in the evaluator, because it is the ceiling of both.

A checklist for reading any "self-improving AI" claim

1. Which level? Harness, model, evaluator, or research process.
2. Where does the training signal come from? Human-written solutions, public answer keys, or the model's own solutions with an automatic check.
3. Is the gain outside the noise band? Look for paired tests or confidence intervals, not a single delta.
4. Was the evaluation set used for selection? If yes, discount the number; look for held-out results.
5. What ships? A config, a model file, a scorer, or a method.
6. What does it cost per round? Tokens for harness loops, GPU hours for model loops.

FAQ

What is recursive self-improvement (RSI) in AI?
A loop in which an AI system improves some part of itself and the improved version runs the next round. The part being improved can be the harness, the model weights, the evaluator, or the research process.

What is the difference between harness-level and model-level RSI?
Harness-level RSI changes the prompts, tools and workflow around a frozen model. Model-level RSI changes the model's weights, so the improvement ships inside the model file.

Is Google's RRSI model-level RSI?
No. RRSI keeps the model frozen and evolves the agent harness. It is harness-level RSI.

Can a model improve without human-written solutions?
Yes. Label-free methods train only on the model's own solutions and use an automatic correctness check, such as agreement across samples or code execution. The check is the limiting factor.

How do you know a self-improvement gain is real?
Measure on benchmarks that were never used for selection, use paired statistical tests, and treat any gain inside the evaluation noise band as zero.

---

References: RRSI code · RRSI paper · RSI survey, arXiv 2607.07663 · R-Zero · Darwin-27B-RSI model card