- Book: RAG Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Give Claude Opus 4.7 a folder it can keep notes in between sessions, and its score on a database question task rises from 0.383 to 0.700. Tell it to record only its failures, and it loses much of that boost. Both results are in The components of agent memory, a post Anthropic's engineering team published on November 6, 2026.
The post is research, not a launch: it announces no new model or API feature (Anthropic). It does link two recipes, a dreaming cookbook and a knowledge-wiki quickstart, and it builds on the Dreams endpoint, which is in research preview for Claude Managed Agents. If you run long-lived agents, or let Claude Code keep a MEMORY.md, the findings map onto choices you already make.
What Anthropic measured
The main benchmark is the database task from CL-Bench 1.0. In Anthropic's words, the task "asks Claude to answer a series of questions by looking up information in a database." Each question arrives in a fresh session, and the only thing that carries over is a memory directory that persists between sessions (Anthropic).
Mean reward, stateless baseline against a memory store, as reported in Anthropic's post:
| Model | No memory | Memory store |
|---|---|---|
| Opus 5.5 | 0.655 | 0.838 |
| Sonnet 5.5 | 0.644 | 0.833 |
| Opus 5 | 0.564 | 0.828 |
| Opus 4.7 | 0.383 | 0.700 |
| Sonnet 5 | 0.306 | 0.610 |
| Sonnet 4.6 | 0.330 | 0.364 |
Sonnet 4.6 is the outlier. The post says it "often didn't read files from its memory directory and failed to test its theories." Prompting it to read its memory files before answering, and then to test its theories, "nearly doubled its benchmark score," to 0.641 (Anthropic; ai-tldr summary).
A memory directory the model never opens is just disk usage. If an older model seems not to benefit from memory, check that it reads its notes before you drop the idea.
Four layers, and the top one stays small
The post describes memory as a hierarchy (Anthropic):
- Transcripts: "a searchable event log" of the agent's sessions.
- Memory notes: "salient observations written by Claude."
- Representations: structures such as wikis, "a collection of pages written and maintained by Claude."
- Injected context: material "loaded into every session."
Four layers of agent memory, as Anthropic describes them. Only the small top layer loads into every session.
The cost sits at the top. "Injected memories use up valuable tokens, and an incorrect note can be carried into every session," the post warns, adding that "Loading a note also doesn't guarantee that Claude acts on it." Its conclusion: "the top layer of the memory hierarchy has to stay relatively small" (Anthropic).
Claude Code already caps that layer. Its memory docs put auto memory in ~/.claude/projects/<project>/memory/, with a MEMORY.md index, and state: "The first 200 lines of MEMORY.md, or the first 25KB, whichever comes first, are loaded at the start of every conversation." Claude moves detail into separate topic files and reads them when it needs them (Claude Code docs).
Say what memory is for, not how to use it
The finding that cuts against instinct is about steering. A rule like "only record what went wrong" looks like good engineering. In Anthropic's runs it backfired: told to record only failures, "both Opus 5 and Opus 4.7 lost much of the performance boost that memory gave them" (Anthropic).
The post sums it up as "it helps to tell Claude what memory is for, but not how to use it," and adds: "For stronger models, giving them freedom to judge salience was more effective than setting fixed rules in advance." With frontier models, Anthropic says it typically gives "simple guidance to create memory notes" when a person asks for one or when something is clearly worth remembering, and leaves that call to Claude (Anthropic).
The Pokémon Red runs in the same post show the shift across generations. Sonnet 3.7 wrote 413 memory files across 225 folders and earned one badge. Fable 5 wrote nine files in two folders plus a global.md, and earned four (Anthropic). In that comparison, fewer notes went with more progress.
The catch is that the right amount of steering depends on the model: Sonnet 4.6 needed explicit instructions just to read its memory. Test your own memory prompt on your own tasks instead of copying anyone else's.
Dreaming: an offline pass that fixes the notes
Notes go wrong. The post says "Claude can make incorrect predictions or write rules to memory that are wrong from the start." Anthropic's answer is dreaming, "reflecting on logs to fix mistakes or produce new notes and representations," and "Dreaming gives Claude a way to test its notes against the original transcripts" (Anthropic).
In the Pokémon run, dreaming corrected a wrong location note, and the agent reached Cerulean City at action 1,935. The runs with uncorrected memory and with no memory never got there (Anthropic).
The wiki experiment is the one closest to retrieval work. Claude built a wiki over a Stainless corpus of "about 350,000 files, roughly 420M tokens" and answered 25 graded questions. Wiki alone rose from 13% to 48% of graded criteria as dreaming went from 2 to 128 sessions. With search, "Claude's aggregate score rose from 54% to 61% as that wiki grew," against 53% for search alone, and the pair "saved about 20,000 tokens per question." The costs are in the post too. On a separate legal-brief task, "the wiki grew larger than the documents themselves," and dreaming in general "requires compute" (Anthropic).
Dreaming, in Anthropic's sense: an idle-time pass that checks notes against transcripts and rewrites the wrong ones.
On Managed Agents, dreaming is an API call. The Dreams docs describe an asynchronous job that reads a memory store plus 1 to 100 past sessions and writes a new, reorganized store: "The input store is never modified, so you can review the output and discard it if you don't like the result." Dreams need the dreaming-2026-04-21 beta header (the SDK sets it), "typically take minutes to a few hours," and bill at standard token rates for the model you pick (Dreams docs).
This is the Python call from those docs, reflowed to fit. store_id and the session IDs come from your own Managed Agents runs:
import anthropic
client = anthropic.Anthropic()
dream = client.beta.dreams.create(
inputs=[
{"type": "memory_store",
"memory_store_id": store_id},
{"type": "sessions",
"session_ids": [session_a, session_b]},
],
model="claude-opus-5-5",
instructions=(
"Focus on coding-style preferences; "
"ignore one-off debugging notes."
),
)
print(dream.id)
I haven't run the Dreams API or the cookbooks myself.
What to change in your own agent setup
- Keep what loads into every session short. In Claude Code that is the top of
MEMORY.mdplus your CLAUDE.md files, and the docs say to "target under 200 lines per CLAUDE.md file" (Claude Code docs). - Tell the agent what its memory is for, drop rules about what to save, and compare scores on your own tasks.
- Check that the model reads its notes. In Claude Code, messages like "Recalled 2 memories" mean Claude is reading from the memory directory, and
/memoryopens the auto memory folder so you can see what is in it (Claude Code docs). - Audit notes against what happened. Auto memory files "are plain markdown you can edit or delete at any time" (Claude Code docs), and a wrong one reaches every session until someone fixes it.
- If you dream, aim it at what changes. The post suggests to "focus dreaming on information that's likely to change" and to "carefully tune the cadence of dreaming" (Anthropic).
I build Hermes IDE, so weigh this with that in mind. Hermes has a top layer of its own: per its README, "facts and pinned files saved for a project reach every later session's agent, also after a restart." Anthropic's rule applies there too. Keep pins few and correct, because each later session's agent gets them.
How much does your agent write to memory today, and when did you last open MEMORY.md and read it?
If this was useful
I build Hermes IDE: free, source-available (BSL 1.1), on macOS, Windows and Linux, for running Claude Code, Codex and other agents side by side, each on its own branch. It's on GitHub, and it isn't affiliated with Nous Research, Anthropic or OpenAI.
The wiki result is a retrieval result: a structured layer on top of search beat search alone and spent fewer tokens per question. The RAG Pocket Guide covers the search half of that, from chunking to reranking, so the pages your agent pulls are the right ones.
Top comments (0)