How to Build Long-Term Memory for LLMs (RAG + FAISS Tutorial)

Devendra Parihar — Sun, 01 Feb 2026 08:43:39 +0000

Build a memory system that lets LLMs remember user preferences across conversations using Python, LangChain, FAISS, and SQLite.
tags: ai, python, langchain, machinelearning

Have you ever had a deep, meaningful conversation with an AI, only to come back the next day and find it has forgotten everything about you? It's the "50 First Dates" problem of modern AI. While Large Language Models (LLMs) are incredibly smart, they suffer from a severe case of amnesia. Once the context window closes, the memory is gone.

In this post, I'll walk you through how I built a Long-Term Memory System for LLM agents that allows them to extract, store, and recall personalized information across conversations. We'll use LangChain, OpenAI, FAISS for vector search, and SQLite for persistent storage.

The Problem: Context Window vs. Long-Term Memory

LLMs have a "context window" — a limited amount of text they can process at once. You can stuff user history into this window, but it gets expensive and eventually runs out of space. Plus, it's inefficient to re-read the entire history of every conversation just to know the user's name or favorite programming language.

We need a system that acts like a human brain:

Short-term memory: The current conversation.
Long-term memory: Important facts stored away and retrieved only when relevant.

The Solution: RAG + Semantic Search

We're building a specialized Retrieval-Augmented Generation (RAG) pipeline. Instead of retrieving generic documents, we are retrieving personal memories about the user.

Key Components

Memory Extractor: An LLM agent that "listens" to the chat and identifies facts worth saving.
Vector Store (FAISS): Stores the "meaning" (embedding) of the memory for fuzzy search.
SQL Database: Stores the structured data (content, timestamp, category) for reliability.
Retrieval System: Fetches relevant memories based on the current user query.

Step 1: Defining a Memory

First, we need a structure. A memory isn't just text; it has metadata.

from dataclasses import dataclass
from typing import List, Dict, Optional

@dataclass
class Memory:
    id: str
    content: str
    category: str  # e.g., 'tools', 'personal', 'work'
    importance: float  # 0.0 to 1.0
    timestamp: str
    embedding: Optional[List[float]] = None
    metadata: Optional[Dict] = None

Step 2: Extracting Memories with LangChain

We don't want to save everything. "Hello" is not a memory. "I use VS Code for Python development" is.

We use LangChain and a carefully crafted system prompt to extract structured data.

# memory_system.py (Simplified)
prompt = ChatPromptTemplate.from_messages([
    ("system", """You are an expert at extracting factual information.
    Focus on preferences, tools, personal info, habits.
    Return a list of memories with an importance score (0-1).
    """),
    ("human", "Message: {message}")
])

# Output is structured JSON
# User: "I mostly code in Python but use Rust for side projects."
# Result: [
#   {"content": "Codes primarily in Python", "category": "skills", "importance": 0.9},
#   {"content": "Uses Rust for side projects", "category": "skills", "importance": 0.7}
# ]

Step 3: The "Brain" (Vector Store + Database)

We use a hybrid storage approach.

Why FAISS? We need to answer questions like "What tools do I use?" even if the memory is recorded as "I work with NeoVim." Keyword search fails here, but Vector Search understands that NeoVim is a tool.

Why SQLite? Vectors are great for search but bad for reading. We need a reliable place to store the actual text, timestamps, and IDs to handle updates and deletions.

class VectorStore:
    def __init__(self, openai_api_key):
        self.embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
        self.index = faiss.IndexFlatIP(1536)  # Inner Product for Cosine Similarity

    def add_memory(self, text):
        vector = self.embeddings.embed_query(text)
        self.index.add(np.array([vector]))

Step 4: Connecting the Dots

The main loop handles the flow:

User sends message.
System extracts facts (if any).
System checks for updates (Did the user say "Actually, I switched to Java"?).
System retrieves relevant history based on the current message.
LLM generates response using the retrieved memories as context.

def answer_with_memory(self, question):
    # 1. Search vector DB for similar memories
    relevant_memories = self.vector_store.search_similar(question)

    # 2. Construct context
    context = "\n".join([m.content for m in relevant_memories])

    # 3. Ask LLM
    prompt = f"Based on these memories:\n{context}\n\nAnswer: {question}"
    return self.llm.invoke(prompt)

Live Demo

I built a Streamlit app to visualize this "brain". You can see the memories forming in real-time, search through them, and even see how the system categorizes your life.

👉 Try the Live Demo

Why This Matters

This isn't just about remembering names. It's about Personalization.

A coding assistant that remembers your preferred libraries.
A tutor that remembers what you struggled with last week.
A therapist bot that remembers your long-term goals.

Future Improvements

Graph Database: Linking memories (e.g., "Paris" is related to "France").
Local LLMs: Running Llama 3 for privacy.
Time Decay: Slowly "forgetting" unimportant memories over time.

Check out the Code

The full code is available on GitHub. It includes the complete detailed implementation of the memory extractor, vector store management, and the Streamlit UI.

Devparihar5 / llm-long-term-memory

A sophisticated memory storage and retrieval system that provides LLMs with persistent, searchable long-term memory capabilities. This system can extract, store, update, and retrieve memories from conversations, enabling AI agents to maintain context across multiple sessions.

🧠 LLM Long-Term Memory

Give your AI agents persistent, searchable long-term memory with pluggable storage backends.

A sophisticated memory storage and retrieval system that provides LLMs with persistent, searchable long-term memory capabilities. This system can extract, store, update, and retrieve memories from conversations, enabling AI agents to maintain context across multiple sessions.

✨ Features

🧠 Intelligent Memory Extraction - Automatically extracts factual information from conversations using OpenAI GPT
🔍 Semantic Search - Vector-based similarity search using OpenAI embeddings and FAISS
💾 Pluggable Storage Backends - SQLite, PostgreSQL, MongoDB, and Redis support
🔄 Memory Management - Add, update, and delete memories with conflict resolution
📊 Category Organization - Automatic categorization of memories
⚡ Importance Scoring - Weighted importance system for memory prioritization
🔗 LangChain Integration - Built with LangChain for robust LLM interactions

📦 Installation

# Basic installation (SQLite backend)
pip install llm-long-term-memory
# With PostgreSQL support
pip install llm-long-term-memory[postgresql]

#

…

View on GitHub

How do you handle state in your LLM apps? Drop a comment below! 👇

Originally published on Medium

As A very beginner open-source contributor

Devendra Parihar — Sun, 05 Nov 2023 17:29:15 +0000

Introduction:
As the leaves turned from green to fiery shades of red and gold, a new chapter unfolded in my life as a software developer. This year, I embarked on my very first Hacktoberfest journey, and it was nothing short of an adventure into the world of open source. In this post, I want to share my experience, the lessons I learned, and the excitement that came with contributing to the open-source community.

The Road to Hacktoberfest:
Like many, I had heard about Hacktoberfest in the past, but I had never actively participated. This year, I decided to change that and take the plunge. The first step was signing up on the Hacktoberfest website, which was incredibly easy and straightforward. Next, I had to decide which projects I wanted to contribute to.

Choosing the Right Projects:
One of the most daunting aspects of participating in Hacktoberfest is deciding which projects to contribute to. With countless repositories available on platforms like GitHub, it can be overwhelming. I decided to focus on projects that aligned with my interests and skills, ensuring that my contributions would be meaningful and valuable.

Getting My Hands Dirty:
Once I had selected a few projects, I started digging into their codebases. It was time to get my hands dirty! The feeling of diving into an unknown codebase was both exciting and intimidating. I faced new challenges and picked up new skills along the way. Reading and understanding the project's documentation and codebase was a vital first step.

Contributions and Pull Requests:
I started with small bug fixes and documentation improvements. This allowed me to get a feel for the contribution process while also making a positive impact on the projects. I learned how to create a pull request (PR), submit it for review, and engage with maintainers and other contributors.

The Thrill of Collaboration:
One of the most rewarding aspects of Hacktoberfest was the sense of community and collaboration. I had the opportunity to interact with maintainers and other contributors from around the world. Their feedback and guidance were invaluable, and it was heartwarming to see how open source brings people together.

Learning and Growing:
Hacktoberfest provided a unique learning experience. I not only contributed to open-source projects but also enhanced my coding skills, learned new technologies, and improved my collaboration and communication abilities.

Completion and Reflection:
As the month of October drew to a close, I had successfully completed my Hacktoberfest challenge by making four meaningful contributions. The feeling of accomplishment was incredibly satisfying. It was a moment to reflect on the journey, the knowledge gained, and the relationships built within the open-source community.

Conclusion:
Hacktoberfest was a thrilling and educational experience that I will treasure for a long time. It's a fantastic opportunity for both beginners and seasoned developers to contribute to open source, learn, and connect with like-minded individuals. If you've never participated in Hacktoberfest, I highly recommend giving it a try next year. You won't just be contributing to projects; you'll be contributing to your growth as a developer and to the vibrant world of open source.

So, here's to my first Hacktoberfest, and here's to the countless adventures that await in the world of open source! 🎉🐱‍💻 #Hacktoberfest #OpenSource #CodingCommunity

DEV Community: Devendra Parihar

How to Build Long-Term Memory for LLMs (RAG + FAISS Tutorial)

The Problem: Context Window vs. Long-Term Memory

The Solution: RAG + Semantic Search

Key Components

Step 1: Defining a Memory

Step 2: Extracting Memories with LangChain

Step 3: The "Brain" (Vector Store + Database)

Step 4: Connecting the Dots

Live Demo

Why This Matters

Future Improvements

Check out the Code

Devparihar5 / llm-long-term-memory

A sophisticated memory storage and retrieval system that provides LLMs with persistent, searchable long-term memory capabilities. This system can extract, store, update, and retrieve memories from conversations, enabling AI agents to maintain context across multiple sessions.

🧠 LLM Long-Term Memory

✨ Features

📦 Installation

As A very beginner open-source contributor