Why I Built a Vector Database Around Disk Instead of RAM

Volatil — Sun, 30 Aug 2026 16:35:40 +0000

I run my AI stack locally: the LLM, the embedding model, and the vector store behind RAG and the agent's memory. Everything shares one machine's RAM. The vector store's share is the part that grows. An embedded vector store keeps its entire index in memory, its memory usage scales with the number of vectors stored, and that growth competes directly with the model for RAM.

So I built AsterVec, a vector database that keeps its index on disk and runs within a configurable memory budget.

Two data structures with opposite personalities

HNSW is the standard vector index, and it is built for memory. Embedded vector engines like Chroma keeps the whole index in the process heap. Traditional server-based vector databases like pgvector and Elasticsearch stay fast only while it fits in memory: once the index outgrows RAM, searches and updates alike degrade into random disk access, and both slow down sharply.

So an on-disk index needs its own layout, and designing one starts
with noticing that an HNSW index is really two data structures with
opposite storage needs:

	Vectors	Graph edges
Shape	fixed size, same for every node	varies from node to node
Writes	each embedding written once	neighbour list keeps updating
Reads	many per query in bulk	one small list per hop
Wants	a packed page layout	a flexible structure

Most engines interleave the two, storing each vector next to its edge list. A single layout then has to serve both access patterns, and it cannot be good at both. The compromise shows up on writes: recording one new edge can rewrite a large block of vector data that did not change.

This is the conflict AsterVec is built around removing. The index is split into two stores on disk, each shaped for its own half:

Graph edges go to a graph-oriented LSM-tree which turns small scattered writes into sequential writes on disk.
Vectors go to a page-based array with a locality-aware cache, so vectors that are searched together tend to be read together.
RAM holds only the upper navigation layers and the caches. That is the entire resident footprint, and it is bounded.

The benchmarks

I compared AsterVec with Chroma and LanceDB, because they are also embedded vector databases. All three ran on the same machine on the
SIFT dataset, with identical HNSW parameters and out-of-box settings. At 100K vectors, the working set is mostly cached, so AsterVec is effectively in memory too:

AsterVec serves 5.2x Chroma's queries per second and 7.3x its inserts per second, in half the memory, with recall on par.

At a million vectors, AsterVec is genuinely serving from disk:

Queries run in roughly 0.3 GB of engine memory against Chroma's 1.2 GB, query speed stays in the same ballpark, and recall is slightly ahead. Comparable speed at a fraction of the memory is exactly the trade-off I wanted: on a machine that also runs an LLM, that memory is needed elsewhere.

Try it on your workload

If memory is a constraint for your local agent or desktop RAG, or you simply want your vector store to use less of it, AsterVec is built for that problem: pip install aster-vec. And if you just want a lightweight embedded vector database, it is a good fit for that too. I would like to know how it behaves on your workload: open an issue if something breaks, and star the repo if you find it useful. It is a young project, and feedback from real workloads helps the most.

github.com/NTU-Siqiang-Group/AsterVec

DEV Community: Volatil

Why I Built a Vector Database Around Disk Instead of RAM

Two data structures with opposite personalities

The benchmarks

Try it on your workload