Vector Databases: Foundations, Function, and the Future of AI Retrieval

Lorenzo Bradanini — Tue, 08 Jul 2025 09:05:03 +0000

Written By CortexFlow

A few years ago, searching for something online essentially meant typing a few keywords and hoping the algorithm guessed what you meant. The results weren’t always wrong (they were actually pretty decent) but they were rarely right in the way you wanted.

They didn’t understand context, or intent, or the quiet nuance behind a question. You basically searched for a book and just got a long list of titles. You asked a question and got a dump of related documents.

Today, that’s changing very fast.

Thanks to generative AI and powerful language models, our systems are becoming more than reactive engines. They’re starting to develop some forms of understanding. They can summarize, answer, reason, and even “remember”.

But beneath this newfound intelligence lies a silent architectural shift: we’ve begun to represent knowledge not as rows in a table, but as vectors, points in a vast, invisible space where meaning itself becomes geometry.

In this world, "apple" floats close to "fruit", far from "Microsoft", and somewhere near "orchard." Every idea, image, sentence, or product becomes a coordinate in a multi-dimensional landscape of relationships and meanings.

This is where vector databases come in. They’re not just tools — they’re the memory systems of modern AI. They allow our machines to store what they’ve seen, relate what they know, and retrieve what matters; not by keywords, but by closeness in meaning.

And if you want to understand how today’s AI systems think, remember, and search, it starts with understanding how we store thought itself: as vectors.

From Objects to Embeddings

In traditional applications, you deal with structured data. Tables. Fields. Types.

A user might have a name, an email, and a list of orders. You index them, maybe add a search bar, and call it a day.

But the vast majority of the world isn’t made of neatly structured fields. It’s made of conversations, images, gestures, and ambiguity; unstructured data, which is notoriously hard to query even with advanced traditional methods.

Vector Embeddings

In the past, solving these problems meant tedious feature engineering: hand-crafting numerical inputs for ML models.

But in the era of deep learning, this collapses under the weight of scale and complexity.

Instead, we now use vector embeddings, which are numerical representations of complex inputs (like text, images, or audio) learned automatically by neural networks.

The key idea: semantically similar inputs map to nearby vectors.

A cat picture and another cat picture? Close together in vector space.
A sentence and its paraphrase? Nearly overlapping.
A fraudulent transaction and a suspicious one? Nearby in embedding space.

This enables a completely new way of querying data: not by exact matches, but by semantic proximity.

What Is a Vector Embedding, Really?

At its core, a vector is just a list of numbers: a precise position in high-dimensional space. But when generated by an embedding model, it becomes a vessel of meaning.

For example:

vector = [0.14, -2.73, 3.99, ...]

Where wi,j is the importance of word i in document j, often computed using TF-IDF.

Similarity is measured using cosine similarity: the smaller the angle, the closer the meaning.

Limitation: Bag of Words

The original model treats terms as independent. It misses:

Context
Synonyms
Phrases

So “apple” and “fruit” could be far apart.

From Keywords to Concepts: The Rise of Embeddings

Modern models don’t just count: they actually understand.

They convert inputs into embeddings: dense vectors that encode semantic meaning.

Words like “banana”, “apple”, and “fruit” naturally cluster close.

This enables:

✅ Semantic search
✅ Cross-modal understanding (e.g., matching text with images)

Why Represent Information as Vectors?

Vectors give us a universal language:

🎵 Match a song to a hum
📄 Match a legal question to a contract clause
📚 Recommend books based on intent, not just purchase history

And you don’t need to define explicit rules, because
the geometry carries the meaning.

Why Is Searching in Vector Space Hard?

Searching for the closest vector in a large dataset means computing distance against every item.

This becomes prohibitive when:

📐 Dimensions = hundreds (e.g., 768, 1536)
📊 Dataset = millions or billions of vectors

This is known as the curse of dimensionality.

Traditional indexes like kd-trees break down.

What Are Approximate Nearest Neighbors (ANN)?

ANN algorithms trade a bit of precision for massive speed gains.

They don’t find the exact match, just one that’s close enough.

For semantic tasks (RAG, search, recommendations), that’s usually more than enough.

How Do ANN Algorithms Work?

1. 🔁 HNSW (Hierarchical Navigable Small World)

Builds a graph of vectors
Queries traverse the graph, hopping toward closer nodes
Highly efficient, logarithmic time

2. 🧩 LSH (Locality-Sensitive Hashing)

Projects vectors into buckets using similarity-preserving hashes
Similar vectors fall into the same bucket
Fast, but less precise than HNSW

3. 🧮 PQ (Product Quantization)

Compresses vectors using learned prototypes
Enables storing billions of embeddings in RAM
Full vectors used only for reranking

What Makes a "Good Enough" Neighbor?

In AI applications, exactness isn’t critical, relevance is.

ANN methods return results semantically close enough.

Even 90–95% recall is typically fine.

Why Not Just Use Brute Force?

You can — and should, for small datasets.

Brute-force is:

✅ Always accurate
🚀 Surprisingly fast (for <10K vectors)

But for large-scale, low-latency needs (e.g., <100ms)?

ANN is essential.

From Theory to Production: How Vector Databases Power AI

At their core, vector databases solve:

🧠 Given a query vector, find the most semantically similar vectors in a huge collection.

🔹 The Lifecycle:

Embedding Creation

Raw input → embedding model → dense vector
Indexing

Store vectors + metadata

Build an ANN index (e.g., HNSW, PQ)
Querying

Embed the query → run ANN search
Filtering & Ranking

Apply metadata filters (e.g., language, time)

Optionally re-rank results
Return Results

Return original content, not just vectors

Used for RAG, recommendations, personalization

Vector Databases vs Vector Indexes

Libraries like FAISS, Annoy provide fast indexes.

But full vector databases (e.g., Pinecone, Weaviate, Qdrant, Milvus) offer:

✅ CRUD operations
✅ Metadata filtering
✅ Replication & durability
✅ Access control
✅ Streaming ingestion
✅ LLM ecosystem integration (LangChain, LlamaIndex)

They’re not just indexes;

They’re intelligent memory layers with DB guarantees.

The Shift to Serverless Vector Infrastructure

Early vector DBs were fast, but resource-hungry.

Serverless vector DBs are the next evolution:

☁️ On-demand compute
🔎 Geometric partitioning
⚡ Freshness caches
🧠 Multi-tenant optimization

The result: scalable, cost-effective, elastic AI memory.

Final Thoughts

We’ve moved from:

TF-IDF vectors →
Deep semantic embeddings →
Real-time, vector-powered AI memory

Today, vector databases are the memory substrate of intelligent systems.

They enable:

📚 Grounded generation (RAG)
🧠 Personalized assistants
🔍 Semantic search
🧭 Conceptual understanding

Behind every capable AI product is a geometric engine doing the work.

And that engine is the vector database.

DEV Community: Lorenzo Bradanini

[Boost]