DEV Community

euk ela
euk ela

Posted on

Meta Let the Model Write Its Own Memory Prompts. It Beat Humans by 11 Points — Then Distillation Gave Back 13

Meta FAIR just published a method called Self-Improving Memory, and its headline result is a small paradox: let an agent discover and optimize its own memory-management prompts, and it beats the best human-written harness — 56.0 to 44.6 on the Continual Learning Bench. Then distill that same skill into the model's weights, and the score falls back to 43.1, below the human baseline.

A skill that lives in the prompt is not the same as a skill that lives in the weights. This post walks through the method, the numbers, and the gap between the two.

The method in two steps: search the prompt, then distill it

Self-Improving Memory is a two-stage method. The base model is Kimi-K2.6 — a 1T-parameter MoE with 32B active parameters and a 256K context window. The meta-agent and the task agent are the same model.

Step 1: discover memory prompts. The agent works with file-based memory and manages two operations: LOAD (read information it needs into context) and OFFLOAD (write information it no longer needs back to the file). The meta-agent keeps generating revised versions of the memory prompt, tests each one on real tasks, and keeps the versions that score higher. This is a search process, not hand-tuning.

Step 2: distill into the weights. The behavior traces produced by the optimized prompt are used for supervised fine-tuning (SFT), with a rank-64 LoRA adapter added to the attention projection layers. The goal is for the model to reproduce the same memory-management behavior without depending on the prompt.

Evaluation uses the Continual Learning Bench, with three tasks: Database (SQL queries), Codebase (fixing bugs in a repository), and Spectrum (radio spectrum scanning). The test subtasks include deliberately designed phase shifts, so the bench measures generalization to new tasks of the same kind — not memorization.

The score table: where it wins, where it loses

Score Database Codebase Spectrum Average
Best human-written harness 47.9 43.7 42.1 44.6
Model-discovered optimized prompt 53.8 64.3 49.9 56.0
Distilled model (generic prompt) 37.7 45.1 46.5 43.1

Three things are worth unpacking.

First, the optimized prompt wins on all three tasks, by 11.4 points on average. The biggest jump is Codebase: 64.3, a full 20.6 points above the human-written version. Bug fixing is a long-chain task, and it lives or dies on memory management — which context to keep, which to write back to file. The prompt's wording decides the outcome.

Second, the distilled model averages 43.1 — 12.9 points below the optimized prompt, and slightly below the human baseline of 44.6. SFT can learn to "do what the prompt says," but it has not fully learned why the prompt says it. The skill is in the prompt; it did not fully become the model's skill.

Third, the search itself is noisy: of 44 generated revisions, 37 failed to beat the best score so far. Optimization here is not a steady climb — it is a large pile of failed attempts with a few effective improvements sifted out. The method burns search volume, not inspiration.

One more comparison for context: the distilled model averages 43.1 across 10 matched seeds, while a baseline trained with SFT on generic trajectories scores only 37.6. Distillation does help — it just does not catch the prompt version.

Three takeaways for agent developers

1. Memory management deserves first-class optimization. No matter how large the context window gets, it cannot hold an entire task history. LOAD and OFFLOAD strategy is becoming the second engineering surface that decides whether an agent succeeds — after tool use.

2. Prompt optimization and weight distillation are two different roads. Right now, a good memory prompt is the better deal — 56.0 versus 43.1. Get the prompt version running reliably first, and only then consider distilling. Do not reverse the order.

3. Search noise is part of the cost. A 37-out-of-44 revision failure rate means that if you want to reproduce this method, your budget has to include a large volume of dead-end searches. Without the search volume, there are no effective improvements.

Honest limitations

The bench is controlled, not real. All three tasks use deliberately designed phase shifts to measure controlled generalization — not the memory decay of a real long-running deployment. In production, task distribution drifts in messier, less regular ways.

The distillation drop is unresolved. The 43.1-versus-56.0 gap shows that the current SFT-plus-LoRA approach does not retain the full benefit of prompt optimization. The method is still in its "the prompt works better" stage.

Reproducing it is expensive. The base model is a 1T-parameter Kimi-K2.6, and serving it takes a multi-GPU setup (one deployment guide suggests starting at 8x H200). Small teams will struggle to reproduce it as-is and can mostly borrow the method's ideas.

All numbers come from Meta's own blog and paper (Figures 4 and 5), with no independent third-party reproduction yet. The relative ordering is credible; the absolute values will have to wait for replication.

The question

If the prompt version of an agent beats the distilled version by 13 points, which would you choose: keep maintaining an increasingly complex set of memory prompts, or accept the score drop for a cleaner weights-only version? And when you run long-task agents today, how do you manage memory — stuff everything into context, or do you already have your own LOAD/OFFLOAD rules? Let me know in the comments.

Top comments (0)