Building Production-Ready Vector Search for the Browser with Rust and WebAssembly

Matteo Panzeri — Tue, 16 Dec 2025 03:22:11 +0000

The Problem: Vector Search at the Edge

If you've worked with AI applications, you've probably used vector databases like Pinecone, Weaviate, or Qdrant. They're excellent for server-side deployments, but what happens when you need vector search:

In a browser — without sending user data to external servers?
Offline — where network connectivity isn't guaranteed?
At the edge — where latency to cloud services is unacceptable?

This was the problem I set out to solve with EdgeVec.

What is EdgeVec?

EdgeVec is an embedded vector database built in Rust that compiles to WebAssembly. It runs sub-millisecond nearest neighbor search directly in browsers, Node.js, and edge devices.

Full disclosure: I built this as a solo developer with LLM assistance (Claude). The AI helped with architecture review, boilerplate code, test generation, and documentation. I handled core algorithm decisions, performance optimization, and quality verification. This collaboration model accelerated development significantly.

Key numbers:

329µs search latency at 100k vectors (768 dimensions)
227 KB gzipped bundle size
3.6x memory compression via scalar quantization
24x faster than voy (the fastest pure-WASM alternative)

import init, { EdgeVec, EdgeVecConfig } from 'edgevec';

await init();
const config = new EdgeVecConfig(128);  // 128 dimensions
const index = new EdgeVec(config);

// Insert vectors
const embedding = new Float32Array(128).fill(0.1);
const id = index.insert(embedding);

// Search
const results = index.search(embedding, 10);
console.log(results);  // [{ id: 0, score: 0.0 }, ...]

// Persist to IndexedDB
await index.save("my-vectors");

Why WebAssembly?

I chose WebAssembly (WASM) for three reasons:

1. Performance

JavaScript is fast for most tasks, but vector similarity search involves millions of floating-point operations. WASM runs at near-native speed, and Rust's zero-cost abstractions mean no runtime overhead.

2. Portability

The same compiled WASM module runs in:

Chrome, Firefox, Safari, Edge
Node.js
Deno
Cloudflare Workers
Any WASM runtime

One codebase, deploy anywhere.

3. Security

WASM runs in a sandboxed environment with no access to the file system or network by default. User embeddings never leave the device unless you explicitly send them.

The Architecture

EdgeVec uses three core components:

HNSW Index (Hierarchical Navigable Small World)

HNSW is a graph-based algorithm for approximate nearest neighbor search with O(log n) query complexity. Each vector becomes a node in a multi-layer graph:

Layer 0: Contains all vectors, dense connections
Higher layers: Contain fewer vectors, act as "express lanes"

Search starts at the top layer and descends, following the greedy path to the nearest neighbor.

SQ8 Quantization

Full-precision vectors use 4 bytes per dimension. For 768-dimensional embeddings (like OpenAI's), that's ~3 KB per vector.

EdgeVec's scalar quantization (SQ8) compresses each dimension to 1 byte:

3.6x memory reduction
Minimal recall loss (~1-2% at k=10)
SIMD-accelerated distance calculations

IndexedDB Persistence

In browsers, EdgeVec stores data in IndexedDB — the same storage API used by offline-first web apps. Your vectors survive page refreshes and browser restarts.

// Save
await index.save("my-index");

// Load
const loaded = await EdgeVec.load("my-index");

What's New in v0.4.0

Version 0.4.0 focuses on production readiness:

Documentation Suite

Tutorial: Step-by-step getting started guide
Performance Tuning: HNSW parameter optimization
Troubleshooting: Top 10 common errors and solutions
Integration Guide: Works with transformers.js, TensorFlow.js, OpenAI API

Migration Guides

Detailed instructions for migrating from:

hnswlib (Python/C++)
FAISS (Python)
Pinecone (Cloud)

Quality Infrastructure

15 chaos tests: Empty index, max dimensions (4096), all deleted, etc.
Load tests: 100k vector stress tests
P99 latency tracking: Percentile benchmarks in CI
Regression detection: 10% threshold enforcement

Performance Comparison

Benchmarked on AMD Ryzen 7 5700U, 16GB RAM:

Library	Search P50	Type	Notes
EdgeVec	0.20ms	WASM	Fastest pure-WASM
hnswlib-node	0.05ms	Native C++	Requires compilation
voy	4.78ms	WASM	KD-tree algorithm

EdgeVec is 24x faster than voy while both run as pure WASM. Native bindings (hnswlib-node) are faster but don't work in browsers and require a C++ toolchain.

Development Philosophy

LLMs are tools, like IDEs and linters. The key is understanding what you're building and verifying the output. Every piece of generated code went through manual review, unit tests, integration tests, and benchmark validation.

Use Cases

Offline-First Semantic Search

Build search features that work without internet:

// User searches their local notes
const queryEmbedding = await embed("meeting notes from last week");
const results = index.search(queryEmbedding, 5);

Privacy-Preserving RAG

Run retrieval-augmented generation without sending documents to external APIs:

// All embeddings stay on device
const context = index.search(questionEmbedding, 3);
const answer = await localLLM.generate(question, context);

Browser-Based AI Assistants

Add semantic memory to chatbots running entirely client-side:

// Store conversation embeddings
index.insert(await embed(userMessage));

// Retrieve relevant context
const relevant = index.search(await embed(newMessage), 5);

Getting Started

Installation

npm install edgevec

Rust Usage

use edgevec::{HnswConfig, HnswIndex, VectorStorage};

let config = HnswConfig::new(128);
let mut storage = VectorStorage::new(&config, None);
let mut index = HnswIndex::new(config, &storage)?;

let id = index.insert(&vec![1.0; 128], &mut storage)?;
let results = index.search(&vec![1.0; 128], 10, &storage)?;

With OpenAI Embeddings

import init, { EdgeVec, EdgeVecConfig } from 'edgevec';
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function getEmbedding(text) {
    const response = await openai.embeddings.create({
        model: "text-embedding-3-small",
        input: text,
    });
    return new Float32Array(response.data[0].embedding);
}

await init();
const index = new EdgeVec(new EdgeVecConfig(1536));

// Index your documents
const docs = ["Hello world", "Vector search is cool"];
for (const doc of docs) {
    index.insert(await getEmbedding(doc));
}

// Search
const query = await getEmbedding("greeting");
const results = index.search(query, 5);

Future Roadmap (v0.5.0+)

ARM/NEON optimization: Verified cross-platform SIMD
Mobile support: Formalized iOS Safari and Android Chrome testing
Enhanced metadata: Native metadata storage alongside vectors
CLI tools: Optional developer command-line interface

Conclusion

EdgeVec proves that high-performance vector search doesn't require cloud infrastructure. With Rust and WebAssembly, we can bring AI capabilities directly to users' devices — faster, more private, and more reliable.

If you're building offline-first AI applications, privacy-preserving search, or browser-based ML tools, give EdgeVec a try. It's MIT/Apache-2.0 dual-licensed, and I welcome contributions and feedback.

Questions? Drop a comment below or open an issue on GitHub. I'm happy to discuss the implementation details or help troubleshoot integration issues.

References

The Rise of WASM in 2024 - Dev.to
Top 5 Vector Databases in 2024 - Medium
How to Get the Right Vector Embeddings - Milvus

Building a Sub-Millisecond Vector Database in Rust/WASM

Matteo Panzeri — Fri, 12 Dec 2025 22:41:48 +0000

I recently built EdgeVec, a high-performance vector database that runs entirely in the browser. Here's how I achieved sub-millisecond search times with WebAssembly.

The Challenge

Vector databases are everywhere in AI applications - they power semantic search, RAG systems, and recommendation engines. But they typically require a server. I wanted to explore: can we get comparable performance entirely in the browser?

Performance Results

Scale	Float32	Quantized (SQ8)
10k vectors	203 µs	88 µs
50k vectors	480 µs	167 µs
100k vectors	572 µs	329 µs

This is comparable to server-side solutions, but running entirely client-side.

Technical Approach

1. HNSW Algorithm

I implemented Hierarchical Navigable Small World graphs - the same algorithm used by production vector databases like Weaviate and Qdrant.

2. Scalar Quantization

Instead of storing 32-bit floats (768 dimensions × 4 bytes = 3KB per vector), I compress to 8-bit integers. This gives 3.6x memory savings with minimal accuracy loss.

3. SIMD Optimization

Using Rust's portable SIMD, I vectorize distance calculations:

AVX2 on native (x86_64)
simd128 on WASM (where available)

4. WASM Compilation

Built with wasm-pack, the final bundle is just 148 KB gzipped - small enough for any web app.

Use Cases

Where does client-side vector search make sense?

Privacy: Embeddings never leave the device
Latency: Zero network round-trip
Offline: Works without internet
Cost: No server bills

Perfect for browser extensions, local-first apps, and privacy-preserving RAG.

Try It

import init, { EdgeVec, EdgeVecConfig } from 'edgevec';

await init();
const config = new EdgeVecConfig(768);
const index = new EdgeVec(config);
index.insert(new Float32Array(768).fill(0.1));
const results = index.search(query, 10);
// results: [{ id: 0, score: 0.0 }, ...]

GitHub: https://github.com/matte1782/edgevec
npm: npm install edgevec

This is an alpha release - feedback welcome!

DEV Community: Matteo Panzeri

Building Production-Ready Vector Search for the Browser with Rust and WebAssembly

The Problem: Vector Search at the Edge

What is EdgeVec?

Why WebAssembly?

1. Performance

2. Portability

3. Security

The Architecture

HNSW Index (Hierarchical Navigable Small World)

SQ8 Quantization

IndexedDB Persistence

What's New in v0.4.0

Documentation Suite

Migration Guides

Quality Infrastructure

Performance Comparison

Development Philosophy

Use Cases

Offline-First Semantic Search

Privacy-Preserving RAG

Browser-Based AI Assistants

Getting Started

Installation

Rust Usage

With OpenAI Embeddings

Future Roadmap (v0.5.0+)

Links

Conclusion

References

Building a Sub-Millisecond Vector Database in Rust/WASM

The Challenge

Performance Results

Technical Approach

1. HNSW Algorithm

2. Scalar Quantization

3. SIMD Optimization

4. WASM Compilation

Use Cases

Try It