Building NovaMem: The Local-First, Open-Source Vector Database for AI Agents

Adhvaidh Abu — Sat, 20 Dec 2025 15:33:10 +0000

Modern AI systems don’t just generate text—they remember, retrieve, and reason over information. This capability, known as RAG (Retrieval-Augmented Generation), is the backbone of useful AI agents.

But when I explored building AI memory systems, I noticed a glaring gap in the ecosystem.

Most vector databases today are massive, cloud-dependent, and expensive. They hide the internal logic behind proprietary APIs. If you are building a local AI agent or learning how embeddings work, spinning up a cloud cluster feels like overkill.

So, I built NovaMem—a local, open-source AI memory engine that runs fully offline and generates embeddings automatically using Ollama.

The Problem: "Why do I need an API key to remember text?"

If you want semantic search or AI memory today, the standard stack usually looks like this:

Cloud Vector DB (Pinecone, Weaviate Cloud)
OpenAI API Key (for embeddings)
Network latency
Monthly subscriptions

For enterprise scale, this is fine. But for indie hackers, students, and privacy-sensitive projects, this introduces unnecessary friction. I wanted a solution that was:

Local: Runs on your machine.
transparent: Open-source code you can actually read.
Self-Contained: Handles embedding generation internally

What is NovaMem?

NovaMem is a local AI memory and vector database.

Unlike a standard database that stores text based on keywords, NovaMem stores the meaning of text using vector embeddings. This allows your application to search by semantic similarity.

In simple terms: NovaMem helps computers remember ideas, not just words.

The Tech Stack
I chose a hybrid architecture to balance performance with ease of use:

Rust Core: Handles the heavy lifting—vector storage, similarity calculations, and memory safety.
Go HTTP API: Provides a clean, developer-friendly interface for integration.
Ollama Integration: Automatically converts text to vectors locally.

How NovaMem Works
The architecture is designed to be simple but effective. Unlike other databases where you have to generate embeddings before inserting data, NovaMem handles that pipeline for you.

The Insert Flow When you send text to NovaMem, it doesn't just save the string.

NovaMem accepts the JSON payload.

It calls your local Ollama instance.

Ollama converts the text into a vector embedding.

NovaMem stores the Text + Vector + Metadata in the Rust backend.

The Search Flow When you query the database:

Your query text is converted into an embedding.

NovaMem performs a vector similarity search against stored data.

Results are ranked by "meaning" rather than exact keyword matches.

Developer Experience: Auto-Embedding in Action
One of the main goals of NovaMem was to remove the "embedding boilerplate" code developers usually have to write.

Here is how you insert a memory. Note that you don't need to provide the vector array manually:

curl -X POST http://localhost:8080/insert \
-H "Content-Type: application/json" \
-d '{
"text": "NovaMem remembers locally using Ollama",
"metadata": { "agent": "demo", "timestamp": "2024-05-20" }
}'

NovaMem automatically:

Talks to Ollama.
Generates the vector.
Persists the data.

Roadmap & Future Vision

NovaMem is currently in the early stages, but the vision is to create the go-to memory layer for local AI agents. The roadmap includes:

Metadata-Aware Search: Filtering results based on JSON tags (e.g., "Find memories from agent_1").

Configurable Backends: Support for different embedding models beyond default Ollama models.

Embedding Versioning: Handling changes in model dimensions gracefully.

Performance: Further optimizations in the Rust core for larger datasets.

Conclusion
NovaMem started as a learning project to understand how vector databases work under the hood. It has evolved into a tool that I believe empowers the next generation of local AI development.

It is not meant to replace massive distributed systems for enterprise data. It is meant to be the brain for your local agents.

I believe infrastructure should be transparent, understandable, and free to learn from. NovaMem is licensed under Apache-2.0 and is open to contributions.

Check out the code, star the repo, and let's build local AI together.

🔗 GitHub: https://github.com/adhvaidhloi/NovaMem

DEV Community: Adhvaidh Abu

Building NovaMem: The Local-First, Open-Source Vector Database for AI Agents