Sho Tanaka (tsho)

Posted on Aug 8 • Originally published at implicit-none.com

pgvector vs LanceDB: A Reproducible Vector Search Benchmark (100k 1536-dim)

#database #performance #postgres #ai

Originally published at implicit-none.com.

Introduction

Whenever you build a vector database — for a recommendation engine, RAG, or anything else — the same question comes up: should vector search ride along in the Postgres you already run (pgvector), or live in a dedicated embedding store (like LanceDB)?

As far as I could find, reproducible same-data, same-query measurements are rare. So this post benchmarks two open-source options — pgvector 0.8.6 and LanceDB 0.36.0 — on 100k real OpenAI embeddings of DBpedia (1536-dim) across four axes:

Ingest + index build (speed and disk)
The recall-latency curve (comparisons only make sense at equal recall, so both systems' search parameters are swept into curves)
Concurrent load (1 thread vs 8 threads)
Filtered search (WHERE + vector — unavoidable in real RAG)

The headline results:

LanceDB ingests 20× faster and uses 1/3 the disk
Single-threaded queries: LanceDB is ~2× faster at equal recall
But at 8 concurrent clients, pgvector flips it and wins by 1.8× (server-side process parallelism vs embedded + GIL)
Filtered search is the big discovery: pgvector's recall can collapse to 0.07 or jump to exact depending on what the Postgres planner silently decides; LanceDB's prefilter is boringly consistent

The architectural difference, first

These two aren't rivals on the same field — they're differently shaped tools, and that shapes how to read the numbers:

	pgvector	LanceDB
Form	PostgreSQL extension (client/server)	Embedded library (in-process)
Invocation	SQL over TCP	Python function calls
Data format	Postgres heap tables	Lance (columnar, Arrow-native)
Transactions	Full Postgres ACID	Optimistic concurrency

pgvector's latencies include connection and protocol overhead; LanceDB's don't. That's not unfair — it's how you'd actually use each one, and this benchmark takes that stance.

One terminology note: LanceDB is the database layer built on top of the Lance format (a columnar format + engine developed under its own org). The storage and vector-index implementations live in the lance core; what this article measures is the lancedb package usage on top of it (using the format directly via pylance is a topic for the upcoming Lance vs Parquet article).

Environment & setup

Item	Value
Machine	Apple M5 Pro / 48 GB RAM / macOS 26.4
pgvector	PostgreSQL 18.4 + pgvector 0.8.6 (official Docker image, capped at 8 CPUs / 6 GB, shared_buffers 3GB)
LanceDB	0.36.0 (Python 3.14, in-process)
Data	DBpedia OpenAI embeddings, 100,000 × 1536-dim, L2-normalized
Queries	1,000 held-out vectors (never inserted), k=10, cosine
Ground truth	Exact top-10 by full scan (recomputed per filter condition)
Indexes	pgvector: HNSW (m=16, ef_construction=64) / LanceDB: IVF_HNSW_SQ (m=16, ef_construction=64)
Filter columns	Boolean columns at 1% / 10% selectivity (deterministic by row index)

All measurements after warmup, fixed seeds, every setting auto-saved into the result JSONs (code on GitHub).

Result 1: Ingest & index build — a LanceDB landslide

Metric	pgvector	LanceDB	Ratio
Ingest (100k)	31.3 s (3,190 vec/s)	1.4 s (71,474 vec/s)	22.4×
Index build	60.3 s	5.2 s	11.6×
Total disk	2.48 GB	0.78 GB	3.2×

pgvector's ingest uses the standard COPY (text format), which pays the structural cost of textifying 1536-dim vectors through the SQL layer. LanceDB writes Arrow tables essentially as-is. The disk gap comes from Lance's columnar format plus the SQ (scalar quantization) index.

If your pipeline regenerates or appends embeddings daily, this 20× gap is wall-clock time you feel.

Result 2: The recall-latency curve — LanceDB 2× faster single-threaded

The core comparison. Since equal-recall comparison is the only fair method for ANN benchmarks, both systems' parameters are swept into curves:

recall@10 band	pgvector p50	LanceDB p50
~0.95	2.50 ms (ef=40)	1.30 ms (nprobes=8)
~0.97–0.98	2.86 ms (ef=80)	1.58 ms (np=8, refine=2)
~0.99	3.42 ms (ef=160)	1.86 ms (np=16, refine=4)

LanceDB is roughly 2× faster across the whole range — which, per the architecture note, includes pgvector's loopback TCP + SQL overhead.

Gotcha: LanceDB's IVF_HNSW_SQ plateaus around recall 0.95 on its own (scalar-quantization error). If you need high recall, refine_factor (re-ranking with full-precision vectors) is mandatory — every ≥0.97 row above uses it. pgvector's HNSW is unquantized, so cranking ef_search takes it toward recall 1.0 with no extra knobs.

Result 3: Concurrency — pgvector flips it at 8 threads

	1-thread QPS	8-thread QPS	Scaling
pgvector (ef=80, recall 0.98)	350	2,376	6.8×
LanceDB (np=8, rf=2, recall 0.97)	611	1,338	2.2×

The engine that was 2× faster alone loses by 1.8× under concurrency. The cause is structural:

pgvector: each connection gets its own Postgres backend process — 8 cores fully used (6.8× scaling)
LanceDB: the search kernel is Rust, but it runs inside a Python process, so thread concurrency is constrained by the GIL (2.2×)

If you're serving high QPS behind a web server, this reversal matters. LanceDB could do better with multiprocessing or a free-threaded (3.13+) Python build — but these numbers are the reality of using it from stock Python.

Result 4: Filtered search — pgvector is "planner-dependent", LanceDB is steady

Combining WHERE category = ... with vector search is mandatory in real RAG, and this is where the two personalities diverge most:

LanceDB (prefilter): recall 0.97–1.0 at p50 2.3–7.5 ms at both selectivities — consistently predictable.

pgvector: results are governed by which plan the Postgres planner picks. Verified by recording EXPLAIN for every case:

1% selectivity: the planner skips HNSW entirely and runs an exact scan + sort → recall 1.0 at ~5 ms. With only 1,000 candidate rows, that's the right call
10% selectivity: the plan flips non-monotonically with ef_search — HNSW at ef=10 and ef=80–320 (fast, recall 0.09–0.99), exact at ef=20–40 (recall 1.0 but 28 ms)
hnsw.iterative_scan = relaxed_order (new in 0.8) lifts the HNSW-path recall from 0.76 to 0.97 at ef=80. Non-negotiable if you do filtered search on pgvector
The most important gotcha: right after a bulk load (before ANALYZE), the planner has no statistics and picks HNSW + post-filter even at 1% selectivity — recall collapsed to 0.071. The identical query returns recall 1.0 after ANALYZE. Always ANALYZE after bulk-loading into pgvector

In short: pgvector's filtered search lets the planner silently decide between fast and exact, and demands tuning literacy (ANALYZE, iterative_scan, reading plans). LanceDB has none of that complexity — but also none of the planner's cleverness (like falling back to exact at 1%).

Which to pick

Two axes — performance and operations:

Situation	Pick	Why
Postgres already deployed, high QPS	pgvector	1.8× at 8 threads; ACID, backups, and permissions come free
ML pipeline with frequent embedding regeneration	LanceDB	20× ingest, Arrow/pandas-native, zero servers
Filter + vector compound queries dominate	LanceDB (or pgvector + tuning literacy)	predictable prefilter; pgvector needs ANALYZE/iterative_scan
Single-request latency first (agent memory etc.)	LanceDB	~2× faster at equal recall, in-process, no network
Many millions of rows, ops team = DBAs	pgvector	reuse the whole Postgres operational stack

Summary

Measured pgvector 0.8.6 vs LanceDB 0.36.0 on identical data (100k × 1536-dim), identical queries, recall-matched, across four axes
LanceDB: 22.4× ingest, 11.6× index build, 1/3.2 disk — decisive for write-heavy workloads
LanceDB ~2× faster single-threaded at equal recall; pgvector wins 1.8× at 8 concurrent clients. "One client or many" is the fork in the road
Filtered search is pgvector's minefield: recall 0.07 without ANALYZE, non-monotonic plan flips with ef_search — both measured, both fixed by iterative_scan = relaxed_order + ANALYZE
LanceDB plateaus at recall ~0.95 without refine_factor (SQ quantization) — always add it for high-precision requirements

Future work

The 1M-scale rerun (memory and index-build gaps should widen further)
Lance vs Parquet: format-level read/write comparison (part of this blog's data engineering series)
Unquantized HNSW head-to-head: LanceDB's flat-family indexes vs pgvector

Benchmark code

All scripts (Docker Compose, data prep, ground-truth generation, both benchmarks, chart generation) and the measured JSONs are at tsho/tsho-lab. Reproduce with docker compose up plus three scripts.

DEV Community