DEV Community

Alex Georghiou
Alex Georghiou

Posted on

The Context Window Is Not a Memory Strategy

The Context Window Is Not a Memory Strategy

When an agent forgets something important, the first instinct is often to give it more context. A longer context window can help, but it is not a substitute for a memory design.

Different information has different lifetimes. The current user's request belongs in working context. A verified account preference may belong in a durable store. A retrieved reference document should be fetched when relevant. An old tool trace usually belongs in neither place.

Stuffing all of these into every prompt increases cost and can reduce clarity. It also makes security harder: a stale document may override fresh policy, or a previous user's data may leak through an incorrectly scoped summary.

I model memory as explicitly managed state with source, timestamp, owner, scope, and deletion behavior. Updates must be deliberate. A model should not silently turn an unverified statement from a retrieved page into a permanent fact about a user.

When context is compacted, preserve decisions and open questions, not every sentence. When information can be re-retrieved reliably, store a reference instead of an enormous copy.

A context window is a transport for relevant information at inference time. Memory is a product and data-governance decision. Confusing them creates systems that are expensive and difficult to trust.

Top comments (0)