How to Self-Host mem0 with a Local Vector Store

If you've used mem0 and discovered it calls OpenAI by default, the fix is three configuration values. mem0's Memory() class is cloud-by-default. It calls OpenAI for fact extraction, calls OpenAI again for embeddings, and writes vectors to a Qdrant instance at /tmp/qdrant. Replacing all three takes a single Memory.from_config() call, and this tutorial shows the exact config using Ollama for the Large Language Model (LLM) and embeddings, and Actian VectorAI DB as the vector store.

Local experimentation tolerates an occasional cloud call. A production deployment in an air-gapped network, an on-premises environment, or any infrastructure where conversation data cannot leave the machine does not, since a single external Application Programming Interface (API) call is a hard failure there, not a minor exception. That's the distinction this tutorial is built around: a stack that is "mostly local" but still makes one cloud call is still a cloud deployment in those environments.

If you've already looked at replacing CrewAI's default memory backend, you'll recognize the decision. Replace the default memory infrastructure with components that match your deployment requirements, while keeping the application logic unchanged. By the end of this guide, you'll have a reusable configuration that runs entirely offline.

Prerequisites

Have these running before the first step:

• Ollama installed and running locally

• Docker installed and running

• VectorAI DB set up locally

• Python 3.10 or higher

What mem0 Uses by Default and What to Override

Three components phone out or write to a default location, and all three are replaceable through config:

• LLM: gpt-5-mini from OpenAI, used for fact extraction from conversation turns. mem0's own package description states this directly: "Mem0 requires an LLM to function, with gpt-5-mini from OpenAI as the default."

• Embedder: text-embedding-3-small from OpenAI, which produces 1536-dimensional embeddings by default.

• Vector store: Qdrant at /tmp/qdrant, which stores and retrieves the embeddings.

Written out, the default config looks like this.

# ILLUSTRATION ONLY. This is what Memory() gives you implicitly. Do not run.
default_config = {
    "llm": {
        "provider": "openai",
        "config": {"model": "gpt-5-mini"},
    },
    "embedder": {
        "provider": "openai",
        "config": {"model": "text-embedding-3-small"},  # 1536 dims
    },
    "vector_store": {
        "provider": "qdrant",
        "config": {"path": "/tmp/qdrant"},
    },
}

The rest of this tutorial swaps each slot for a local alternative.

Figure 1: Default cloud components and their local replacements

Step 1: Install Dependencies

Create and activate a virtual environment.

python -m venv .venv
source .venv/bin/activate

Install mem0 together with the dependencies required for Ollama and the VectorAI DB integration.

pip install mem0ai==2.0.11 langchain-actian-vectorai langchain-ollama

The VectorAI DB langchain-actian-vectorai includes actian_vectorai as a transitive dependency, so you do not need to add it to this command separately. Once installation completes, confirm mem0 reports version 2.0.11:

python -c "import mem0; print(mem0.__version__)"

Step 2: Start VectorAI DB

Pull the image and start the container with a volume mount, so the collection survives a restart instead of resetting every time the container stops:

docker pull actian/vectorai:latest
docker run -d --name vectorai \
  -v ./local_data:/var/lib/actian-vectorai \
  -p 6573-6575:6573-6575 \
  -e ACTIAN_VECTORAI_ACCEPT_EULA=YES \
  actian/vectorai:latest

The -p 6573-6575:6573-6575 maps all three ports the container exposes in one range, rather than three separate -p flags: 6573 for the REST API, 6574 for the gRPC API VectorAIClient connects to, and 6575 for the local web UI.

Confirm the container is up with the Python SDK, which is how VectorAI DB verifies a fresh install:

from actian_vectorai import VectorAIClient

with VectorAIClient("localhost:6574") as client:
    info = client.health_check()
    print(f"Connected to {info['title']} v{info['version']}")

Step 3: Pull the Ollama Models

Two pulls give you the extraction LLM and the embedding model this tutorial uses.

ollama pull qwen3:4b
ollama pull nomic-embed-text

The qwen3:4b handles fact extraction. nomic-embed-text produces 768-dimensional embeddings. If you substitute a different embedding model, update the dimension value in Step 4 to match its output; otherwise, the first insert will fail.

Step 4: Build the Local Config and Initialize mem0

The config dict has three top-level keys covering llm, embedder, and vector_store. Each key specifies a provider and its config dict. The llm and embedder sections use mem0's ollama provider. The vector_store section uses mem0's langchain provider and takes an initialized ActianVectorAIVectorStore as the client.

The embedding model is configured in two places, and each one does different work. mem0's embedder section embeds the queries you pass to search(). The OllamaEmbeddings instance passed to ActianVectorAIVectorStore embeds the text mem0 writes on add(), since inserts go through the LangChain store rather than through mem0's own embedder. Both must point at the same model, or query vectors and stored vectors end up at different dimensions and retrieval breaks silently.

import os

# REQUIRED. Set before importing mem0. mem0 reads this once, at package
# import time (mem0/memory/telemetry.py, line 14: MEM0_TELEMETRY =
# os.environ.get("MEM0_TELEMETRY", "True")). Setting it after the import
# has no effect.
os.environ["MEM0_TELEMETRY"] = "false"

from actian_vectorai import VectorAIClient, VectorParams, Distance
from actian_vectorai.exceptions import CollectionExistsError
from langchain_actian_vectorai import ActianVectorAIVectorStore
from langchain_ollama import OllamaEmbeddings
from mem0 import Memory

# 1. Connect to VectorAI DB and create the collection.
#    Size 768 matches nomic-embed-text. Change both together, or neither.
client = VectorAIClient("localhost:6574")
client.connect()

try:
    client.collections.create(
        "mem0_memories",
        vectors_config=VectorParams(size=768, distance=Distance.Cosine),
    )
except CollectionExistsError:
    pass  # Collection already exists from a previous run.

# 2. Build the LangChain vector store backed by VectorAI DB.
#    OllamaEmbeddings keeps the store's own embedding path local too.
vector_store = ActianVectorAIVectorStore(
    client=client,
    collection_name="mem0_memories",
    embedding=OllamaEmbeddings(model="nomic-embed-text"),
)

# 3. Override all three cloud defaults in one config.
config = {
    "llm": {
        "provider": "ollama",
        "config": {
            "model": "qwen3:4b",
            "temperature": 0,  # 0 favors consistent, literal fact extraction
            "ollama_base_url": "http://localhost:11434",
        },
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "nomic-embed-text",
            "ollama_base_url": "http://localhost:11434",
            "embedding_dims": 768,
        },
    },
    "vector_store": {
        "provider": "langchain",
        "config": {
            "client": vector_store,
        },
    },
}

mem = Memory.from_config(config)
print("mem0 initialized. LLM, embedder, and vector store are all local.")

The CollectionExistsError comes from actian_vectorai.exceptions. It catches exactly one condition: the collection already existing. Anything else, a refused connection, a rejected credential, or a dimension conflict, still raises and stops the script where you can see it, instead of being swallowed by a bare except Exception.

Run the script. The success message confirms initialization. If any provider string or key is rejected, recheck it against the mem0 vector store config reference.

Step 5: Store and Retrieve Memories

Three operations exercise the full stack: store a memory with mem.add(), retrieve it with mem.search(), and understand one limitation of mem.get_all() under the langchain provider.

The add() still takes user_id as a direct keyword argument. search() and get_all() do not; both reject a top-level user_id and require filters={"user_id": ...} instead, and raise ValueError if you pass it the old way. This is a real asymmetry in the 2.0.11 API surface.

# Store a memory from a conversation turn.
result = mem.add(
    [
        {"role": "user", "content": "I'm vegetarian and I'm allergic to nuts."},
        {"role": "assistant", "content": "Noted, I'll remember that."},
    ],
    user_id="alice",
)
print(result)
# Expected shape:
# {'results': [
#   {'id': '...', 'memory': 'Is vegetarian', 'event': 'ADD'},
#   {'id': '...', 'memory': 'Is allergic to nuts', 'event': 'ADD'},
# ]}

# Retrieve memories relevant to a query for the same user.
# search() takes filters, not a direct user_id kwarg.
hits = mem.search("What should I avoid eating?", filters={"user_id": "alice"})
for hit in hits["results"]:
    print(hit["memory"], "| score:", round(hit["score"], 4))
# Scores are illustrative and vary by model and run.
# Expect the allergy fact to rank higher than the vegetarian fact.

# get_all() also takes filters, not a direct user_id kwarg. Do not run this
# under the Langchain provider yet -- see the note below.
# all_memories = mem.get_all(filters={"user_id": "alice"})

The first two calls succeed, and the facts that add() stored appear in the search() results. The get_all() call is the exception. The mem0 docs confirm that the LangChain vector store provider does not support get_all or delete_all because LangChain's interface lacks a standardized bulk listing across backends. Expect an error or empty behavior rather than a listing.

Use search() with a broad query as the retrieval path, or query the VectorAI DB collection directly through client.points.scroll() when you need a full inventory.

Step 6: Run the Network-Blocked Verification

Telemetry suppression is handled in Step 4, in the os.environ["MEM0_TELEMETRY"] line before the mem0 import. What's left to verify is that nothing else calls out either.

Run an insert with the network blocked. Disable external network access by disconnecting the interface, using a firewall rule, or running inside a network-isolated container, then run the add() call again:

docker network create --internal isolated

Run your script inside a container attached only to that network, with Ollama and VectorAI DB reachable on the host or the same network. If the insert completes without error while external egress is blocked, extraction, embedding, and storage are all local.

When This Pattern Fits and When It Falls Short

Use this stack when data locality is the requirement:

• Air-gapped deployments

• Privacy-sensitive workloads, such as healthcare and legal

• Edge devices where cloud API calls are unavailable

• Cost-sensitive deployments processing high memory-operation volumes, where per-call API fees compound

Stick with the cloud defaults when locality is not a constraint:

• Development and prototyping, which move faster without managing local model infrastructure

• Workloads where memory precision is the product and a larger cloud model's extraction quality matters more than where it runs

• Teams without the capacity to operate Ollama and a database, who may prefer paying per call

Wrapping up

You replaced mem0's three cloud dependencies, the LLM, the embedder, and the vector store, with Ollama and VectorAI DB through a single Memory.from_config() call, then verified the stack with the network blocked. If your deployment is heading on-premises or to the edge, read how on-premises agent architecture differs from cloud deployments next.

Explore the VectorAI DB Community Edition, the VectorAI DB documentation for additional vector store configuration options, and the Actian Discord for help from other people running this stack.

Frequently Asked Questions

Does this work with a different local LLM?

Yes. Change model in the llm section to any model pulled in Ollama, such as llama3.2:3b or a larger qwen3 variant. Nothing else changes. Extraction quality is not equivalent across models, though. Fact extraction depends on instruction following, and smaller models extract fewer and noisier facts. If memories look incomplete, move up a model size before changing anything else.

What if I want to use a different embedding model?

Change two values together. Set the new model name in the embedder section and in OllamaEmbeddings, and set the collection dimension to the model's output size when creating the collection. nomic-embed-text is 768 dimensions and text-embedding-3-small is 1536, so a swap between them without updating the dimension guarantees a mismatch. The consequence of a mismatch is a DataException on the first insert, because VectorAI DB locks the vector dimension at collection creation. Delete and recreate the collection when switching models.

Is mem0 really fully offline after this setup?

Fully offline here means no runtime calls leave the machine, including telemetry, which Step 4 suppresses. Extraction runs in Ollama, embedding runs in Ollama, and storage is VectorAI DB on localhost. Run the network-blocked insert test from Step 6 rather than trusting the config. Model pulls and pip installs still need a network, so provision those before going offline.

What is the difference between the open-source mem0 library and the mem0 Platform?

This tutorial covers the open-source library, the mem0ai package you self-host. The mem0 Platform is the managed product with a hosted API, its own client, dashboards, and usage-based pricing, and it cannot be pointed at your local vector store. If data locality brought you here, the open-source library is the only fit. Choose the Platform when you want managed memory without operating infrastructure and cloud processing is acceptable.

Common Problems

DataException on the first insert.

The embedding dimension does not match the collection dimension in almost every case. Check the collection's configured size in VectorAI DB, check your embedding model's output size, and make them equal. mem0's self-hosting guide calls this line "easy to miss and hard to debug without," and it is the single most common failure in this tutorial. If you changed embedding models after creating the collection, delete the collection and recreate it at the new size, since you can't change dimensions after creation.

Ollama connection refused.

Confirm Ollama is running with ollama list, which fails fast if the server is down. Then confirm the port with curl http://localhost:11434, which returns "Ollama is running". Finally, check that ollama_base_url in both the llm and embedder config sections matches that address exactly, including the scheme. A missing http:// or a stale port from another machine's config are the two usual culprits.

mem0 is still calling OpenAI despite the config override.

Check which constructor you used. Memory() ignores your config dict entirely and uses the cloud defaults. Only Memory.from_config(config) applies overrides. Also confirm all three sections are present in the config, because overriding only the LLM still leaves the embedder calling OpenAI.

mem0 is still sending telemetry.

Placement is the issue. MEM0_TELEMETRY is read once, at package import time, so setting it after from mem0 import Memory has no effect. Move the os.environ line above every mem0 import in the process entry point, and check for other modules that import mem0 earlier in the import chain. To confirm it is taking effect, print os.environ.get("MEM0_TELEMETRY") immediately before the mem0 import and verify it shows false.