Graph‑based retrieval can cut memory‑operation token usage to zero while still returning correct answers. Zero‑Mem shows that a deterministic graph of past interactions supplies all the evidence an LLM needs, so no intermediate generation step is required even for long‑running agents.
Before this work, most LLM agents stored history in flat text buffers or vector stores and refreshed it with additional model calls, incurring recurring token and latency costs. Retrieval‑augmented generation and chain‑of‑thought memory pipelines all invoke the language model to encode, summarize, or re‑write past turns before answering new questions.
Zero‑Mem eliminates all memory‑operation token consumption. “With an identical final‑QA reader and an equivalent context budget, Zero‑Mem eliminates memory‑operation token consumption and reduces latency by 57.6% relative to the most time‑efficient baseline.” [1] The system builds an entity–context graph and a temporal hierarchy once, then routes queries through these structures without ever sending intermediate traces through the model.
Zero‑Mem cuts memory‑operation latency by 57.6 % compared with the fastest existing baseline. “With the same final‑QA reader and context budget, it reduces memory‑operation time cost by 57.6% relative to the fastest compared baseline.” [1] Because only the final question answer triggers the LLM, inference time shrinks proportionally to the saved intermediate passes.
The result does not settle how graph construction scales to millions of interaction steps or whether deterministic retrieval can handle highly ambiguous queries that benefit from generative reasoning. The paper reports competitive performance but leaves open whether zero‑token memory can match specialized fine‑tuned readers on niche domains, suggesting a need for broader evaluation across diverse knowledge‑intensive tasks.
If structured agent memory truly needs no intermediate tokens, the default architecture for long‑context agents should shift to a graph‑based retriever feeding a single final reader. Existing benchmarks that count total LLM tokens and latency must be rerun with Zero‑Mem’s pipeline to expose the real cost savings of eliminating memory‑operation generation altogether.
Top comments (0)