I've been reading through dev communities lately, and this exact topic keeps showing up in different forms: context windows hit a million tokens, so is memory even necessary anymore. Enough people are arguing both sides that I wanted to actually dig into it and put together my own take instead of just picking whichever post I read most recently.
Just How Big Are They Now?
A few years ago, a few thousand tokens felt generous. Now, 1 million is the baseline. Meta released the 10-million-token Llama 4 Scout last year, and a startup called Magic built a 100-million-token model (LTM-2-mini). That means about 10 million lines of code or 750 novels can fit into a single prompt. At this point, you have to wonder what is left for a separate memory system to do.
Camp 1: It's Already Enough
Fabio Akita pointed out that looking at the leaked Claude Code source code, Anthropic's own coding agent doesn't use a vector DB at all; it just uses the file system and grep. By his math, a 200,000-token query costs about $0.63 including caching, which is cheaper long-term than maintaining a vector DB pipeline. Meta made the same bet, marketing the Llama 4 Scout as capable of holding years of chat history "without a vector store."
It's an attractive argument, and honestly, the cost aspect is the strongest part of this camp. But I think the question this camp is answering is narrower than reality. "Do we need a vector DB?" and "Do we need memory?" are different questions. Akita's point is really about search complexity, using grep instead of embeddings, not about whether state needs to persist between sessions. Meta's marketing also conveniently skips over the fact that a session with 10-million-token chat history eventually ends, and the next session starts at zero.
Camp 2: Memory Solves a Completely Different Problem
Mem0 compares the context window to RAM rather than storage: the moment a session ends, everything inside vanishes. Redis puts it more sharply: agents don't fail because a single invocation lacks space, but because they lack continuity, they can't carry what they learned in one session over to the next. No window size fixes this. Even a 100-million-token model completely forgets you the moment you open a fresh chat.
The most reliable evidence here is Chroma's "context rot" research. Testing 18 models (including GPT-4.1, Claude 4, Gemini 2.5, Qwen3), they found that performance degrades as inputs get longer, well before hitting the model's actual limits. A single piece of irrelevant distractor info noticeably drops accuracy. In some tests, a scrambled mess of info actually performed better than an organized one. This is what genuinely convinced me: fitting into a window and a model actually utilizing it well are two different things, and sellers of larger windows have plenty of incentive to blur that boundary.
And Camp 3: The Window Is Simply Too Small
Factory.ai points out that current 1-million to 2-million-token models are already smaller than their enterprise customers' codebases. CloudGeometry goes a step further, arguing that even if you gave them 100 million tokens tomorrow, it wouldn't be enough because codebases are graph structures while context windows are linear, no matter how large they get, that structural topology disappears.
What's fascinating is that Factory and CloudGeometry draw the exact same fact, "therefore, we need better search," while Magic and Meta go with, "therefore, let's build bigger windows." Same observation, opposite prescriptions. It's also worth noting that the "bigger window" crowd usually sells models, while the "better search" crowd usually builds things on top of other people's models.
My Conclusion
Context windows need to keep growing, I'm not arguing against that. What I disagree with is treating "bigger windows will fix memory problems" as a given. These are two separate investments, and relying entirely on one to substitute for the other doesn't work. Why? Because a bigger window solves exactly one problem: the fitting problem.
It does nothing for session amnesia, accuracy loss from irrelevant tokens, or the cost of reprocessing the exact same context on every single invocation. Memory has to evolve on its own to solve these three, and scaling the context window won't do that work for it. So the real paradigm isn't "context versus memory." Rather, while context handles one problem, memory must solve the remaining three through its own evolution, not by waiting for someone else's context window to get bigger.
The Strongest Counter-Argument to My Point
I don't want to give memory a free pass, so let's briefly counter my own argument. Memory poisoning is a real attack where someone sneaks mundane content into an agent's long-term memory to plant false info, which the agent then retrieves and trusts in a completely unrelated situation later on. MINJA, presented at NeurIPS 2025, achieved this with just a few mundane queries.
Add in silent failures (memory retrieves the wrong thing, yet the model speaks with absolute confidence and no errors) and stale information (studies measure that forcing answers yields stale info 15 to 40 percent of the time), and you have genuine reasons to doubt anyone saying, "just throw memory at it."
Yet my answer doesn't change, and here is why. Silent failures and stale info are mistakes, not malicious acts. They are mostly fixed by displaying confidence scores and recency on retrieved memories, and automatically invalidating old info when new data arrives. This isn't a design flaw, it's engineering maturity, and it's precisely what improves as memory technology matures. Security, however, never fully goes away, just like defense and offense evolving together, much like anti-spam filtering never truly ends. But saying "this requires ongoing work" is different from saying "memory is a dead end." It's simply a trade-off you must accept the moment you ask an agent to persist anything, via memory or otherwise.
Wrapping Up
So let's sum it up: if your agent needs to remember anything past the current session, waiting for context windows to get bigger is betting on the wrong lever.
Change my mind. What is actually breaking in your production environment right now, massive contexts, RAG, structured memory, or something entirely different?
Top comments (0)