Add persistent memory to a LlamaIndex agent (Memory, memory blocks and a retriever)

Disclosure: I'm Mohammed, founder of MemorySync. I drafted this article with AI assistance and reviewed it for accuracy against the official MemorySync and LlamaIndex docs. The code is adapted from the docs and I haven't run it.

LlamaIndex agents already have memory. Pass a Memory object to agent.run(..., memory=memory) and the agent gets a short-term chat buffer plus optional long-term "memory blocks". Two things surprise people when they try to make it remember a user across sessions:

1. By default the Memory class uses an in-memory SQLite database, so the buffer is gone when the process restarts (LlamaIndex memory docs).
2. Memory blocks only receive messages when the short-term buffer overflows. The defaults are token_limit=30000 and chat_history_token_ratio=0.7, so nothing is flushed to long-term memory until the chat history passes about 21,000 tokens. Most real conversations never get that long.

This post fixes both with MemorySync's LlamaIndex integration, llamaindex-memorysync. MemorySync is a hosted memory layer, and I'm building it. LlamaIndex behaviour comes from the official memory guide linked above. Everything about MemorySync comes from its LlamaIndex Memory guide. The code is adapted from those docs. I haven't run it for this post, so check it against your versions.

What you'll set up

1. A drop-in Memory that sends each user message for fact extraction as it arrives and recalls relevant facts on later runs, for the same user, in any thread.
2. Optional: the recall block on its own, composed with blocks you already use.
3. Optional: memories as a LlamaIndex retriever, a query engine or an agent tool.
4. Optional: five memory tools and three helpers.

Install

pip install llamaindex-memorysync

Per the guide it needs llama-index-core 0.13+ and Python 3.10+. PyPI currently lists 1.1.0 with llama-index-core>=0.13,<0.15. Set MEMORYSYNC_API_KEY in the environment, or pass api_key, and keep it server-side. This is Python only. The guide points TypeScript users to the Vercel AI SDK or Mastra integrations.

Step 1: Swap in MemorySyncMemory

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
from llamaindex_memorysync import MemorySyncMemory

memory = MemorySyncMemory.from_defaults(
    user_id="customer-7",    # required: scopes memory to one end user
    session_id="thread-42",  # groups the stored facts by thread
)

agent = FunctionAgent(tools=[...], llm=OpenAI(model="your-model"))

# First conversation
await agent.run("I'm vegetarian and I fly aisle.", memory=memory)

# A later run for the same user
response = await agent.run(
    "Book my trip: flight plus a dinner spot.", memory=memory
)

MemorySyncMemory is a subclass of LlamaIndex's Memory, not a wrapper, so token_limit, insert_method and any extra blocks still work. What changes, per the guide:

• Each user message is sent to MemorySync when it's added to memory (on aput), not when the buffer overflows. Short conversations persist too.
• Only user messages are sent. Assistant replies, system prompts and tool traffic stay in the short-term buffer.
• recall=False or persist=False turns either half off.

The guide also spells out failure handling. The short-term buffer write happens first and its errors propagate. The MemorySync write comes after and reports through on_error instead of failing the turn. If recall fails, no memory block is rendered and the chat history still flows.

Note that the short-term buffer is still LlamaIndex's own. If you also want the raw transcript to survive restarts, LlamaIndex's docs show passing async_database_uri (for example a Postgres URI) or your own chat_store to Memory.from_defaults. I'm assuming those pass through, since the guide says every standard Memory kwarg does, but I haven't tested it.

Step 2: Who owns the memory

user_id is the memory owner. Take it from your authenticated user record, not from the request body and not from a session id. A new user_id is a new, empty memory. session_id only groups facts by thread, so the same user gets the same memories in a new thread.

Step 3 (optional): Compose the block yourself

If you already run your own Memory with blocks like FactExtractionMemoryBlock or VectorMemoryBlock, add the recall block next to them:

from llama_index.core.memory import Memory
from llamaindex_memorysync import MemorySyncMemoryBlock

memory = Memory.from_defaults(
    session_id="thread-42",
    memory_blocks=[
        MemorySyncMemoryBlock(user_id="customer-7"),
        # ...alongside any other blocks you already run
    ],
)

Recall renders into LlamaIndex's own memory template. Keep the flush rule in mind, though: in this setup the block only receives the user messages the buffer flushes into it. If you want every user message stored right away, use MemorySyncMemory from Step 1. The guide says a message sent on both paths is recognised server-side and extracted once.

Truncation is the other difference. LlamaIndex's docs say that when blocks plus chat history exceed token_limit, blocks are truncated in priority order, and by default a truncated block is removed entirely. The guide says MemorySyncMemoryBlock drops its lowest-value lines instead, so the top memories stay in the prompt.

Step 4 (optional): Memories as a retriever

from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.tools import RetrieverTool
from llamaindex_memorysync import MemorySyncRetriever

retriever = MemorySyncRetriever(user_id="customer-7", similarity_top_k=5)

nodes = retriever.retrieve("dietary preferences")  # scored nodes

engine = RetrieverQueryEngine.from_args(retriever=retriever, llm=llm)
answer = engine.query("What do we know about this customer's diet?")

tool = RetrieverTool.from_defaults(
    retriever=retriever,
    name="memory_search",
    description="Search everything known about this user.",
)

It's a standard BaseRetriever (sync and async), with the memory id and user in node metadata. similarity_top_k accepts 1 to 50. Unlike the memory classes, it raises on API failure instead of returning an empty list, so a RAG pipeline can tell "no memories" from "the service errored".

Step 5 (optional): Tools and helpers

from llama_index.core.agent.workflow import FunctionAgent
from llamaindex_memorysync import create_memorysync_tools

agent = FunctionAgent(
    tools=create_memorysync_tools(user_id="customer-7"),
    llm=llm,
    system_prompt="Use the memory tools to remember durable facts.",
)

# For agents you don't fully trust: search and list only
create_memorysync_tools(user_id="customer-7", read_only=True)

The tools are add_memory, search_memory, list_memories, update_memory (tags and importance only; memory text can't be changed) and delete_memory. They return readable error strings instead of raising.

For code outside an agent, there are sync helpers:

from llamaindex_memorysync import get_memory_context, save_turn, search_memories

context = get_memory_context("what should I cook?", user_id="customer-7")  # "" for a new user
hits = search_memories("dietary preferences", user_id="customer-7")

# Explicit persistence. This one raises on failure.
save_turn(user_id="customer-7", user="I'm vegetarian",
          assistant="Noted!", session_id="thread-42")

save_turn sends only the user message. API errors raise MemorySyncAPIError, which carries the HTTP status and server detail.

Troubleshooting

• Nothing is remembered after a short chat. You're probably using a plain block inside Memory, which only sees flushed messages. Switch to MemorySyncMemory.
• A new thread knows nothing. Check that user_id is identical in both runs. session_id can differ.
• The memory block disappears in long chats. That's LlamaIndex's default truncation for other blocks. Check their priority values; priority=0 blocks are always kept.
• Version errors. The package targets llama-index-core 0.13 to 0.14.

Sources

• MemorySync LlamaIndex Memory guide: docs.memorysync.io/guides/llamaindex
• LlamaIndex memory guide: developers.llamaindex.ai/python/framework/module_guides/deploying/agents/memory/
• PyPI: llamaindex-memorysync 1.1.0

Try it

If you want to see the save, reload and recall loop before wiring up an agent, the MemorySync playground runs it in your browser: memorysync.io/try.