Remembers Everything, Learns Nothing

The agent broke a rule it had written down

The agent had the rule. You can find it in its memory file, line 1,140 of about 1,800: do not reformat the config, the deploy is strict about its indentation. You wrote it there three weeks ago, the last time it broke the deploy. This morning the agent reformatted the config. The deploy broke. It remembered the rule and broke it anyway.

Anyone who has run an agent for more than a week has met some version of this. You added memory. You did the responsible thing, gave the agent a place to keep what it learned, and then watched it keep the wrong things, or keep the right things somewhere it never looks. Run 100 came out no sharper than run 1. The store filled up and the behaviour stayed exactly where it was.

It remembered too much, and it kept all of it in one pile, where 1,800 lines bury every instruction equally. The rule loaded. It just loaded as one line among the hundreds it had needed once and never again.

Memory is several different things wearing one name. Store them in one place and you get a slow agent, a swollen context, and a file full of rules that quietly disagree.

That is the whole problem. The fix has two parts: sort what the agent keeps by how often it changes, and put a loop in charge of what gets to stay.

Memory is four things, sorted by how often each changes

Pull the pile apart with one question: how often does this change? The answer tells you where each piece belongs, because the four kinds of memory want four different homes.

At one end sit the things that almost never change. The agent’s standing rules, its constitution, the few constraints that hold on every task. Those belong in one short, always-loaded file, the kind Claude Code keeps in CLAUDE.md and Codex keeps in AGENTS.md, and short is the load-bearing word. A fresh session can burn a real slice of its budget loading its own instructions before you have typed a thing, so a line earns its place in that file by one test: would the agent get this wrong without it, on most tasks? The config rule passes, which is why it belonged here, in the fifty lines the agent reads every time, not on line 1,140 of a log it barely skims.

A step along are the things that change now and then, and only for certain tasks. Workflows, procedures, the steps for cutting a release. Those become skills, each in its own small file the agent loads only when the task calls for it, so you can keep fifty of them and pay for none until the one you need comes up.

Further along is what changes every single run. What the agent did, what broke, the fix it landed on. That is raw trajectory, and it goes in an append-only log, a plain learnings.md you only ever add to and compress later, never editing in place.

At the far end sits the big reference body that changes rarely but in bulk. Documentation, old code, whatever corpus the agent searches. That lives in an external store it queries once, early, under tight hygiene. Facts live there. Lessons live a layer up, in the log.

Sort memory by how often it changes, and each kind lands where the agent will look for it. Mix them, and the rule you need every time sits buried among everything you needed once.

This is the part most “give your agent a memory” guides get right and then quietly undo, by letting all four drain back into one file. The whole trick is keeping them apart. Hold the four kinds separate and each stays usable. Let them merge and you slide back into the single pile, config rule and all.

The loop is the part that compounds

Separation stops the rot. It does not, on its own, make the agent better. A tidy store is still a store, and a store only remembers. None of what follows pays off on one-off work, where a wrap-up is pure overhead; the loop earns its keep only when the same failure keeps coming back. Improvement comes from a loop that runs on top of the store, and the loop has four moves.

It starts with a wrap-up. After every task, the agent appends a few plain lines to the log:

## 2026-06-05  deploy auth changes
did:       edited config.yaml, ran the deploy
failed:    deploy rejected the file, the parser choked on the new indentation
fix:       restored the original formatting, deploy passed
next time: do not reformat config.yaml, the deploy is strict about indentation

The last line is the one that earns its keep. That next time is the promotable lesson, the single thing that might change what the agent does on its next run. You wire this up with one standing rule in the always-loaded file: after each task, append a wrap-up to learnings.md in this shape. The model will mostly remember on its own, and a Claude Code stop hook makes it certain, running the wrap-up the moment the agent finishes. No wrap-up, no raw material, and everything downstream starves.

Then an evaluation runs. On a schedule, you re-run a set of tasks drawn from real past failures and check whether the agent still handles them. This is the step that turns “it feels worse lately” into a logged, specific entry you can act on.

Consolidation comes next. Once a week, a pass compresses the log, archives the dead lines, and holds the active file to something an agent can read in one sitting, a few hundred lines rather than a few thousand. Without it, the append-only log becomes the 1,800-line pile again by a slower route.

Promotion is where it pays off. When the same next time line shows up three times or more, it graduates. It stops being a log entry and becomes its own skill, a file at .claude/skills/deploy-prep/SKILL.md:

name: deploy-prep
description: Use before any deploy. Stops the config.yaml reformatting that has broken the deploy three times.
Before any deploy, leave config.yaml formatting untouched; the deploy is strict about indentation.
Run `make check-config`, then deploy.

That description line is the load-bearing part. The agent reads it every session and pulls the skill in only when a deploy comes up, so the rule stays out of the way until the moment it matters. The buried log line is now a procedure the agent runs without being told, and the three redundant entries get cut. The break stops happening.

A skill is still context, though. The agent reads it and usually obeys, and for most lessons usually is the right bar. For a failure that is cheap to trigger and expensive to suffer, promote it one more step, out of memory and into enforcement: a pre-deploy check that fails loudly, or a Claude Code hook that blocks the edit before it lands. The make check-config line in that skill is the seed of it. Memory tells the agent what to do. A hook makes the wrong move impossible.

A store remembers. A loop improves. The difference is whether a failure the agent logged ever becomes a procedure it runs without being asked again.

Notice what compounds. The store only grows. The loop is the part that keeps finding the failures that recur and turning them into procedures, and three of its four moves throw things away or move them up the stack. Only the first one adds.

The evaluation is what separates learning from theatre

One of those four moves is doing more work than the rest, and it is the one almost everyone skips. Run the loop without the evaluation step and the wrap-up notes are self-reported and unchecked. The agent writes “fixed the config issue” and nothing on earth confirms it. The log fills with confident receipts for work that may not hold. You get a beautiful record of intentions and no idea whether the agent is improving or quietly getting worse.