DEV Community

Vaibhav
Vaibhav

Posted on

Do You Actually Need a Vector Database? A Reality Check Before You Add One

#rag #database #ai #insurtech

A vector database is on every 2026 roadmap because it's what you buy when you're "doing AI." Sometimes it's the right call. Just as often it's infrastructure bought ahead of the problem — adopted because it's fashionable, then underused because nobody asked what question it was meant to answer. Here's an honest checklist before you sign the contract.

What it's actually for

A vector database stores embeddings and retrieves by semantic similarity rather than exact keywords. Ask "what's my excess for water damage?" and it can surface the relevant clause even when the document says "escape of water" and "deductible." That semantic retrieval is the engine behind RAG. For genuine natural-language search over large unstructured corpora, it's the right tool.

The key phrase is large unstructured corpora. Outside that, it's usually overkill.

When you don't need one

  • Your data is structured. "What's the sum insured on policy 12345?" is a SELECT, not a similarity search. A surprising number of "AI" projects reach for vectors to answer questions SQL answers better, faster, and exactly.
  • Your corpus is small. A few hundred docs don't need a dedicated store. pgvector on the Postgres you already run handles modest embedding workloads fine: 
CREATE EXTENSION IF NOT EXISTS vector;
ALTER TABLE clauses ADD COLUMN embedding vector(1536);
-- cosine distance, top 5
SELECT id, text
FROM clauses
ORDER BY embedding <=> :query_embedding
LIMIT 5;
Enter fullscreen mode Exit fullscreen mode
  • Keyword search would do. If users search known terms and codes, a good full-text index is cheaper and more precise. Semantic search shines on fuzzy language, not exact-term lookup.
  • You haven't solved retrieval quality. A vector DB doesn't make retrieval good. Bad chunking, missing metadata, and no re-ranking produce confidently-wrong answers no matter how fancy the store.

The trap: infrastructure before use case

The failure pattern is buying the vector DB first, then hunting for something to point it at. You end up running, securing, and paying for a specialized system that serves one half-finished chatbot. Reverse the order: find a real natural-language-over-documents problem, prototype retrieval on the simplest thing that works (usually pgvector), and graduate to a dedicated store only when scale or latency genuinely demands it. Most teams don't hit that ceiling for their first several use cases.

Ask these first

  1. Is the question semantic or exact? (A lot of it is exact.)
  2. How big is the corpus, really?
  3. Have you solved chunking, metadata, and re-ranking?
  4. Would pgvector do for now?

None of this is anti-vector-database — it's pro-solving-the-right-problem. In insurance specifically, RAG projects underdeliver because of the document engineering underneath (extraction, structure-aware chunking, metadata), not the choice of store. I wrote up the full version here:

Do You Actually Need a Vector Database? An Insurance Reality Check →

Part of an ongoing series on the data foundation under insurance AI from IntelliBooks.

What's the most over-engineered vector-DB deployment you've seen? Curious how often it's really a lookup in disguise.

Top comments (0)