The Controversial Pivot
Last quarter, we dumped our managed cloud vector DB subscription. We replaced it with an on-premise hybrid engine (local Qdrant + SQLite FTS5 on Docker loopback).
Here is the hard data from production:
1. The Multi-Hop Latency Tax
In an autonomous agent workflow with 8–10 sequential tool/memory steps, network latency compounds aggressively:
- Cloud Vector SaaS: 0 imes 55 ext{ms (TLS + network hop)} = \mathbf{550 ext{ms}}$ spent purely waiting on sockets.
- Local Loopback Mesh: 0 imes 3.8 ext{ms} = \mathbf{38 ext{ms}}$ total. That 14x difference is the line between a responsive agent and a timing-out worker.
2. Why Pure Vectors Failed Our Parts Catalog
Semantic similarity is terrible at exact part numbers and error codes. Querying or with dense embeddings returned semantically 'similar' bearings with wrong diameters.
We still had to bolt on BM25/FTS5 keyword search. If you need lexical precision anyway, running a dedicated cloud vector SaaS alongside local search creates a maintenance split-brain.
3. The Trade-Off
Self-hosting isn't free: you must enforce strict Linux memory cgroups so the OOM killer doesn't nuke your node during HNSW index builds.
The Question to You
Is managed vector search becoming the next big enterprise over-engineering trap?
Did your team stay on hosted SaaS (Pinecone, Weaviate Cloud), or did you move to embedded engines (pgvector, local Qdrant, SQLite)?
What is your real p95 latency in production?
Bohdan Hordii // Systems & Sovereign Infrastructure Architect
Details: bohdanhordii.dev
Top comments (0)