LanceDB Review: Promising Object-Storage Vector Search, Production Benchmarks Still Unverified

Why We Brought This Tool Into Our Lab

We brought LanceDB into the lab to test a specific architectural proposition: could a file-format-first vector database remove a separately operated vector service from a multimodal AI stack?

That proposition matters. A typical retrieval system accumulates several operational surfaces: an embedding pipeline, blob storage for source media, a vector database, metadata storage, backup jobs, and synchronization logic between them. Every additional managed service adds network hops, credentials, billing dimensions, failure modes, and another consistency boundary.

LanceDB takes a materially different approach. Its open-source implementation runs as an embedded library rather than requiring a long-lived database server. The table lives in Lance-formatted storage, while the application process supplies the query compute. The same programming model can point at a local directory or an object-store URI.

That makes the architecture attractive for:

• Batch enrichment pipelines already reading from object storage.
• Single-tenant retrieval systems with predictable concurrency.
• Multimodal datasets where vectors and associated columns should remain together.
• Developer tools that need local search without Docker Compose or a control plane.
• Teams trying to avoid a managed vector-database bill before traffic justifies one.

It also moves responsibility back onto the engineering team. Index maintenance, append behavior, object-store request latency, version retention, compaction, caching, concurrency control, and deployment topology become part of the application design. “No server” does not mean “no operations.”

We used three acceptance baselines. First, the LanceDB Enterprise benchmark gave us a vendor-controlled reference point. For our Enterprise-only reference baseline, we used the warmed-cache profile for one million 1,536-dimensional vectors: P50 25 ms, P90 26 ms, P99 35 ms, and maximum 49 ms. We treated those figures as directional, not as expectations for the embedded object-storage path.

Second, we selected the multi-vector needle-in-a-haystack tutorial as a prospective baseline for late-interaction retrieval, but we did not evaluate its workflow. Third, we built our test plan around the vector-search contract: exact search as ground truth, IVF_PQ and IVF_HNSW_FLAT as ANN candidates, matching distance metrics, and full-vector refinement where applicable.

Our pass criteria were deliberately harder than “the quickstart returned neighbors.” We required:

1. Recall@10 and latency for IVF_PQ and HNSW-family search at fixed recall targets.
2. Index build time normalized per one million vectors.
3. Recall and latency after repeated appends.
4. Explicit index-rebuild timing.
5. Version cleanup and compaction timing.
6. Local-disk versus S3 or GCS latency.
7. A cost model that included compute, storage, object requests, and engineering ownership.

We use these acceptance criteria for infrastructure reviewed in our tools collection. LanceDB’s architecture warranted evaluation, but we could not complete the benchmark.

Hands-On Walkthrough: Setup, Execution & Output

The advertised installation requires only a few commands:

python -m venv .venv
source .venv/bin/activate
pip install lancedb

For repeatable work, we pinned the dependency set instead of allowing the latest transitive versions to drift:

pip install \
  lancedb==0.17.0 \
  pyarrow==18.1.0 \
  numpy==1.26.4 \
  pydantic==2.10.6

No database process, port, API key, or local container is required for embedded mode. A local connection is simply a filesystem path:

import lancedb

db = lancedb.connect("./data/lancedb")

The equivalent object-storage topology points the connection at a bucket prefix:

import lancedb

# Credentials are supplied through the workload identity or environment.
s3_db = lancedb.connect("s3://effloow-lab/lancedb-benchmark")
gcs_db = lancedb.connect("gs://effloow-lab/lancedb-benchmark")

We would not use static administrator keys for this. For a future object-storage test, we would restrict the workload identity to the benchmark prefix and only the object operations needed to read, write, list, and delete table artifacts. We did not provision or verify such an identity in our network-isolated sandbox, and no object-storage test ran. Bucket versioning and lifecycle rules also needed separate review because database-level cleanup and provider-level object retention are not the same thing.

Our proposed production harness would generate deterministic normalized vectors, create an exact-search ground truth, build each ANN index independently, and compare returned IDs at k=10. The important point was to compare indexes at the same recall target, not merely at their default settings.

This is the reduced local harness we prepared:

import json
import shutil
import time
from pathlib import Path

import lancedb
import numpy as np
import pyarrow as pa

ROWS = 100_000
DIMENSIONS = 128
QUERIES = 200
K = 10
SEED = 20261016

root = Path("./lancedb-lab")
shutil.rmtree(root, ignore_errors=True)

rng = np.random.default_rng(SEED)
vectors = rng.normal(size=(ROWS, DIMENSIONS)).astype("float32")
vectors /= np.linalg.norm(vectors, axis=1, keepdims=True)

query_ids = rng.choice(ROWS, size=QUERIES, replace=False)
queries = vectors[query_ids]

schema = pa.schema([
    pa.field("id", pa.int64()),
    pa.field("modality", pa.string()),
    pa.field("vector", pa.list_(pa.float32(), DIMENSIONS)),
])

data = pa.Table.from_pydict(
    {
        "id": np.arange(ROWS, dtype=np.int64),
        "modality": np.where(np.arange(ROWS) % 2 == 0, "image", "text"),
        "vector": vectors.tolist(),
    },
    schema=schema,
)

db = lancedb.connect(str(root))
table = db.create_table("items", data=data)

def run_query(query, exact=False, refine=None):
    search = (
        table.search(query, vector_column_name="vector")
        .distance_type("dot")
        .limit(K)
        .select(["id"])
    )
    if exact:
        search = search.bypass_vector_index()
    if refine is not None:
        search = search.refine_factor(refine)
    return search.to_list()

def benchmark(exact=False, refine=None):
    latencies_ms = []
    results = []

    for query in queries:
        started = time.perf_counter()
        rows = run_query(query, exact=exact, refine=refine)
        latencies_ms.append((time.perf_counter() - started) * 1000)
        results.append([row["id"] for row in rows])

    return results, {
        "p50_ms": float(np.percentile(latencies_ms, 50)),
        "p95_ms": float(np.percentile(latencies_ms, 95)),
        "p99_ms": float(np.percentile(latencies_ms, 99)),
    }

ground_truth, exact_latency = benchmark(exact=True)

started = time.perf_counter()
table.create_index(
    vector_column_name="vector",
    index_type="IVF_PQ",
    metric="dot",
    num_partitions=256,
    num_sub_vectors=16,
    replace=True,
)
build_seconds = time.perf_counter() - started

approx, ann_latency = benchmark(refine=10)
recall_at_10 = np.mean([
    len(set(expected) & set(actual)) / K
    for expected, actual in zip(ground_truth, approx)
])

print(json.dumps({
    "rows": ROWS,
    "dimensions": DIMENSIONS,
    "index": "IVF_PQ",
    "build_seconds": build_seconds,
    "recall_at_10": recall_at_10,
    "exact_latency": exact_latency,
    "ann_latency": ann_latency,
}, indent=2))

This is an unexecuted example; we have not verified that it runs with the pinned dependencies. Our actual isolated attempt targeted a separate optional filtering-contract test: a small deterministic table, exhaustive search without an index, and comparisons of default prefiltering, explicit postfiltering, and a nonrestrictive-filter control. Dependency preparation exceeded the environment's artifact limit before that test executed. The dependency log included:

Downloading lancedb-0.17.0 ... 28.2 MB
Downloading pyarrow-18.1.0 ... 38.6 MB
Downloading numpy-1.26.4 ... 13.9 MB
Downloading pylance-0.20.0 ... 31.9 MB
Successfully downloaded lancedb pyarrow numpy pydantic pydantic-core pylance
wheel bundle exceeds 100 MiB

That distinction matters. The dependency download resolved wheels for the target platform, but the artifact limit prevented execution of our optional local filtering test. ANN index builds, append loops, cleanup, and remote-storage measurements were outside that attempted test's scope. We have no local filtering observations or measured recall, latency, memory, compaction, or cost-per-million results to publish.

We will not turn package download logs into a database benchmark.

Installation Failure and What We Could Not Verify

The immediate failure was not a LanceDB query exception. It was a build-artifact policy in our isolated runner. The resolved wheel bundle crossed 100 MiB, largely because the stack included LanceDB, PyArrow, NumPy, and the native Lance dependency.

That is not evidence that a normal workstation or production image cannot install LanceDB. It is evidence that “one-line install” does not imply a tiny deployment artifact. Teams using Lambda-style packaging, restricted CI runners, or systems that mirror every wheel into an internal registry need to account for native and Arrow dependencies in capacity planning.

Our workaround is straightforward: build a pinned container image in a runner with an adequate artifact allowance, record its digest, and execute the benchmark from that immutable image. We would also separate dependency caching from benchmark storage so a cold package installation cannot contaminate timing.

Separately from the blocked filtering test, our production evaluation still needs answers to the following questions.

Index freshness after appends

An ANN index is not useful merely because it exists. We needed to know how newly appended rows participate in search, when index coverage degrades, and whether rebuilds create latency or storage spikes. Our planned sequence was an initial 100,000-row index followed by ten 10,000-row appends, with recall and latency measured after every batch.

We obtained no valid measurements for that sequence. Teams evaluating append-heavy chat memory, event ingestion, or catalog updates should treat rebuild cadence as an explicit operational requirement until their own workload proves otherwise.

Distance metric mismatch

LanceDB’s search contract requires the query metric to match the index metric. A mismatch can prevent index use and force exhaustive search. That is a dangerous performance failure because the query can remain functionally correct while becoming much more expensive.

We made .distance_type("dot") explicit in the harness because the vectors were normalized and the index used dot. We would enforce this in a shared query wrapper and add a latency regression test. Relying on SDK defaults across version upgrades is not a production control.

Quantized distance is not exact distance

With IVF_PQ, the returned _distance can come from the compressed index representation. refine_factor(1) recomputes full-vector distances for the existing candidate set; a factor above one expands and reranks candidates, usually trading more work for better recall.

That means a comparison of default IVF_PQ against exact search is incomplete. A useful comparison must show how nprobes and refinement affect recall and tail latency. We planned to select configurations at recall@10 targets of 0.90, 0.95, and 0.99. We did not complete tests at any of those recall targets, so we cannot declare a winner between IVF_PQ and the HNSW-family option.

Object storage changes the latency model

Local NVMe and S3 or GCS are not interchangeable backends. Object storage introduces request latency, request charges, cache dependence, and different behavior for many small reads. A warmed process may hide that difference; a restart, autoscaling event, or low-frequency tenant may expose it.

The Enterprise reference numbers do not answer this question because they describe a warmed, provisioned Enterprise environment. They are not measurements from our embedded Python process reading an object-store table. Our remote-storage column remains blank.

Version cleanup and compaction remain blank as well. Until we measure both wall-clock time and temporary storage amplification, we would not place LanceDB into an append-heavy production path with a tight storage budget.

Scale, Latency & Cost vs. Alternatives

The only verified latency figures available to our review are the Enterprise reference results. We recorded them as context, not as outcomes from our embedded test:

Workload	Dataset	Cache condition	P50	P90	P99
Enterprise vector search	1M vectors, 1,536 dimensions	Warmed	25 ms	26 ms	35 ms
Enterprise selective metadata filter	15M vectors, 256 dimensions	Warmed	30 ms	39 ms	50 ms
Enterprise broader metadata filter	15M vectors, 256 dimensions	Warmed	65 ms	76 ms	100 ms

The broader filter result is especially useful because it shows how workload shape can dominate a headline database number. Selectivity, index configuration, cache warmth, concurrency, and cluster sizing all matter.

Our own required measurement matrix ended here:

Acceptance test	Status	Production conclusion
IVF_PQ recall and latency	Not executed	No approval
IVF_HNSW_FLAT recall and latency	Not executed	No index winner
Fixed 0.90, 0.95, and 0.99 recall targets	Not executed	No fair latency comparison
Rebuild after repeated appends	Not executed	Cadence and pause cost unknown
Version cleanup and compaction	Not executed	Storage amplification unknown
Local versus S3/GCS latency	Not executed	Remote serving risk unknown
Cost per one million vectors	Not computable	No direct price comparison

Architecturally, the trade-offs look like this:

Option	Operations model	Storage model	Strongest fit	Main cost risk
LanceDB embedded	Application owns compute and maintenance	Local disk or object storage	Batch, single-tenant, multimodal pipelines	Engineering time, cache misses, rebuilds
PostgreSQL with pgvector	Existing database team owns it	Database block storage	Moderate vector scale plus transactional metadata	Database CPU, memory, index growth
Qdrant or Weaviate self-hosted	Dedicated vector service	Attached or distributed storage	Online retrieval with service isolation	Cluster operations and replication
Pinecone	Managed service	Vendor-managed	Fast production adoption and elastic service model	Recurring usage bill and platform dependency
Turbopuffer	Managed, object-storage-oriented service	Vendor-managed object-storage architecture	Cost-sensitive managed retrieval	Service dependency and workload-specific pricing

For break-even analysis, we use total monthly cost rather than storage price alone:

LanceDB monthly cost =
  compute
  + object storage
  + object requests
  + data transfer
  + backup/version retention
  + engineering hours
  + incident allowance

The managed alternative is:

Managed monthly cost =
  service usage
  + data transfer
  + integration overhead
  + vendor-risk allowance

An illustrative calculation shows why engineering ownership dominates early decisions. If self-hosting saves $2,000 per month in service charges but consumes 20 engineering hours per month at a loaded $150 per hour, it loses $1,000 before incident cost:

$2,000 saved - (20 × $150) = -$1,000 per month

At only five engineering hours, the same hypothetical saves $1,250:

$2,000 saved - (5 × $150) = $1,250 per month

Those are examples, not measured LanceDB, Pinecone, or Turbopuffer prices. The real break-even point must use the team’s cloud bill, concurrency target, retention policy, and labor rate. If you need that model built against an actual workload, our AI infrastructure services focus on this exact boundary between attractive component pricing and full production ownership.

Our Final Verdict: When to Deploy, When to Skip

LanceDB remains technically interesting. The embedded API removes service setup, the storage model aligns with multimodal data, and the index surface exposes the controls needed to reason about recall rather than hiding them behind a generic search endpoint.

We have not completed the production benchmark needed for approval. Our only isolated execution attempt was an optional local filtering-contract test, and dependency preparation blocked it before execution. That failure establishes neither a latency result nor a scaling limit, and the filtering test would not have validated ANN, compaction, or object-storage performance even if it had succeeded. Our production verdict therefore remains a no-decision.

Deploy this if:

• You can run a pinned native container rather than relying on a restricted function package.
• Your workload is batch-oriented, single-tenant, or tolerant of application-owned caching.
• Object storage is already the system of record.
• You can benchmark exact recall against every ANN configuration.
• You have a defined index-rebuild and compaction schedule.
• You value colocating vectors, metadata, and multimodal records more than receiving a managed-service SLA.

Run a deeper staging benchmark first if:

• Appends are continuous.
• Query latency must remain stable after process restarts.
• Tables will be queried directly from S3 or GCS.
• Metadata filters vary substantially in selectivity.
• You need recall above 0.95 and plan to use quantization.
• Multiple application replicas may write to the same table.

Hold off or avoid it if:

• You need a validated thousands-of-QPS serving layer immediately.
• Your team has no capacity to own index freshness, cleanup, and compaction.
• P99 latency is contractual and cold-cache object-store behavior is unmeasured.
• A managed service costs less than the engineering time required to operate the embedded path.
• Your deployment platform enforces package limits that the Arrow and native dependency stack cannot meet.

Our next valid run would move the harness into a digest-pinned container, publish raw JSON and environment metadata, and test local NVMe, S3, and GCS independently. We would not approve production until IVF_PQ and HNSW-family configurations were compared at identical recall targets and the append/rebuild cycle had survived a multi-hour soak.

For now, our verdict is promising architecture, insufficient measured evidence for production approval. Teams with controlled concurrency and strong storage engineering should keep LanceDB on the shortlist. Teams buying predictable online latency should keep the managed alternatives in the comparison until remote-storage tails, rebuild cost, and cleanup behavior are measured on their own data. For help designing that acceptance test, contact our engineering team.