I Built the SQLite of Vector Search in ~120KB of Pure C & SIMD: 3,000x Faster Cold Starts and Zero Dependencies for AI Agents

⚡ I Built the SQLite of Vector Search in ~120KB of Pure C & SIMD

If you're building LLM agents, local RAG systems, or CLI tools in Python today, you've likely faced the vector database dependency nightmare.

To store a few thousand embeddings from a conversation history or document chunks, standard tutorials tell you to pip install chromadb or install FAISS.

Here is what happens under the hood:

• Over 35 transitive dependencies are downloaded into your virtual environment (pydantic, onnxruntime, tokenizers, fastapi, duckdb, uvicorn, grpcio).
• The distribution package balloons past 150 MB+.
• Cold Start Penalty: Just typing import chromadb takes 1.5 to 2.5 seconds before a single line of your actual code executes. If you run a CLI tool, a serverless AWS Lambda function, or an ephemeral Docker container, you pay this 2-second tax every single run.
• The Small-to-Medium Scale Trap: Over 95% of AI agent workloads store between 50 and 50,000 vectors (conversation turns, tool history, session memory). At this scale, hierarchical graph traversal (HNSW) incurs massive pointer chasing, cache thrashing, and non-deterministic approximate recall.

I asked myself: Why isn't there an SQLite equivalent for vector search?

A single, self-contained binary file. Zero external dependencies. Sub-millisecond import time. Single-file persistence (.nvec).

So I built NanoVector.

---

⚖️ The Numbers: NanoVector vs ChromaDB vs FAISS

Benchmarked on an Intel/AMD x86-64 CPU (AVX2+FMA) with standard 384-dimensional embeddings (all-MiniLM-L6-v2 / sentence-transformers):

Metric / Feature	NanoVector ⚡	ChromaDB 🐢	FAISS ⚖️
Wheel Download Size	38 KB (~120 KB unpacked)	~120 MB+	~50 MB+
External Dependencies	0 (Zero)	35+ packages	OpenMP, BLAS
Python Cold Import Time	0.6 ms (🚀 3,000x faster)	1,850 ms	~120 ms
Search Latency ($N=2,000$, 384D)	0.13 ms (7,478 QPS)	8.2 ms	0.22 ms
Batch Ingestion Throughput	1,414,000 vectors/sec	~25,000 vectors/sec	~400,000 vectors/sec
Persistence Model	Single .nvec binary file	Multi-dir SQLite + DuckDB	Custom binary
Zero-Copy NumPy Buffer	Yes (Python Buffer Protocol)	No (copies memory)	Partial
GIL Released During Search	Yes (Py_BEGIN_ALLOW_THREADS)	Partial	Partial

---

🛠️ How It Works: Bare-Metal Architecture

1. Direct 8-Wide AVX2 & ARM NEON Vector Kernels

Instead of relying on heavy linear algebra libraries (OpenBLAS, MKL) that incur function call dispatch overhead, NanoVector uses handcrafted SIMD kernels:

• AVX2+FMA (x86-64): Processes 8 float32 elements per vector register cycle with 4-way loop unrolling (32 floats per iteration) directly in CPU L1/L2 cache.
• ARM NEON (Apple Silicon M-series & Linux AArch64): Uses 128-bit float32x4_t registers with fused multiply-accumulates (vmlaq_f32).
• FASM x86-64 Edition: Includes a standalone pure assembly implementation written in Flat Assembler for zero-C environments.

       Query Vector Q (1 x D)              Database Vector Matrix (N x D)
     [ q0 q1 q2 q3 q4 q5 q6 q7 ]           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 0
                                     x     [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector 1
                                           [ . . . . . . . . . . . . ]
                                           [ d0 d1 d2 d3 d4 d5 d6 d7 ] -> Vector N
                      │                                   │
                      └─────────────────┬─────────────────┘
                                        ▼
                   AVX2 Dot Product Accumulator (ymm0-ymm3)
                         Exact Cosine / L2 / IP Score

2. The 100% Deterministic Brute-Force Advantage

At scale ($N < 50,000$), modern CPUs with 256-bit SIMD can compute dot products across the entire database in less than 0.2 milliseconds.
Approximate Nearest Neighbor (ANN) algorithms like HNSW or IVF trade off accuracy for speed, but at $N < 50k$, the graph traversal overhead and random pointer jumps actually make HNSW slower than sequential SIMD streaming from L2 cache!

NanoVector provides 100% exact, deterministic recall with zero approximation errors.

3. The .nvec Single-File Storage Format

Like SQLite's single .db file, NanoVector serializes the vector matrix, vector IDs, and optional JSON metadata strings into a compact, atomic .nvec file:

[Header: 32 bytes]  -> Magic 'NVEC', Version, Metric, Dim, Count
[Vectors: N * D * 4] -> Contiguous 32-byte aligned IEEE-754 floats
[String Offsets]    -> ID & Metadata index table
[Strings Data]      -> Packed UTF-8 strings

Saving and reloading takes under 1 millisecond.

---

🚀 Quickstart: 30 Seconds to Long-Term Agent Memory

Install via pip:

pip install nanovector

1. Basic Vector Search

import nanovector
import numpy as np

# Initialize index (384D for sentence-transformers, 768D for BERT, 1536D for OpenAI)
index = nanovector.Index(dim=384, metric="cosine")

# Add embeddings with metadata
vec = np.random.randn(384).astype(np.float32)
index.add("doc_1", vec, metadata='{"title": "NanoVector Launch", "author": "eminsk"}')

# Search top-k
query = np.random.randn(384).astype(np.float32)
results = index.search(query, top_k=5)

for r in results:
    print(f"ID: {r.id} | Score: {r.score:.4f} | Meta: {r.metadata}")

# Save to a single atomic file
index.save("memory.nvec")

# Reload instantly
loaded = nanovector.load("memory.nvec")
print(f"Loaded {len(loaded)} vectors in {loaded.dim}D!")

2. Autonomous AI Agent Episodic Memory Pattern

Here is how you give an LLM agent persistent memory without external database infrastructure:

import os
import json
import nanovector
import numpy as np

class AgentMemory:
    def __init__(self, filepath="agent_brain.nvec", dim=384):
        self.filepath = filepath
        self.index = nanovector.load(filepath) if os.path.exists(filepath) else nanovector.Index(dim=dim, metric="cosine")

    def remember(self, turn_id: str, embedding: np.ndarray, user_prompt: str, assistant_reply: str):
        meta = json.dumps({"prompt": user_prompt, "reply": assistant_reply})
        self.index.add(turn_id, embedding, metadata=meta)
        self.index.save(self.filepath)

    def recall(self, query_embedding: np.ndarray, top_k=3):
        return self.index.search(query_embedding, top_k=top_k)

# Usage in your agent loop
brain = AgentMemory()
# Recalls relevant past experiences in 0.15 milliseconds!
memories = brain.recall(current_task_embedding, top_k=3)

---

🌐 Open Source & Community

• GitHub: https://github.com/eminsk/nanovector
• PyPI: https://pypi.org/project/nanovector/
• Interactive Google Colab Notebook: Open in Colab

If you're tired of 200MB Docker images and 2-second cold imports for simple vector operations, give NanoVector a spin. ⭐ Star the project on GitHub if you believe in lightweight, bare-metal software!