Find Similar Sentences Using Embeddings and a Vector Database

Why I Started Exploring Semantic Search

I'm building Second Memory, a personal memory application that allows users to save their memories and retrieve them later through natural-language questions.

One challenge is that users don't necessarily remember the exact words they used when saving a memory. They might remember its context or meaning instead.

For example, a user might have saved these memories:

• "I forgot my umbrella and got soaked in the rain."
• "I lifted weights at the gym after work."
• "I caught up with John over coffee yesterday."
• "I stayed up late debugging my application."

Later, the user asks:

"Who did I meet recently?"

The most relevant memory would be:

"I caught up with John over coffee yesterday."

A traditional keyword search might miss this because "meet" and "caught up" are different expressions.

This is where semantic search comes in. Instead of relying on exact keyword matches, we can search for text with similar meanings using embeddings.

In this post, I'll explore how to generate embeddings, calculate cosine distance, find relevant sentences, and perform vector searches using PostgreSQL with pgvector.

The examples use a locally runnable embedding model, so no API key or paid embedding service is required.

Generate Embeddings

An embedding is a numerical representation of text. An embedding model converts text into a vector, allowing us to compare sentences mathematically.

For this demonstration, I'll use sentence-transformers/all-MiniLM-L6-v2, a small, open-source embedding model that runs locally and produces 384-dimensional vectors.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

sentences = [
    "I forgot my umbrella and got soaked in the rain.",
    "I lifted weights at the gym after work.",
    "I caught up with John over coffee yesterday.",
    "I stayed up late debugging my application.",
]

embeddings = model.encode(sentences)

print(embeddings.shape)
# (4, 384)

print(embeddings)
# [[ 0.04035296  0.10288464  0.09306896 ... -0.00502765 -0.13130508  0.05253862]
#  [-0.07009698  0.04319851  0.00160278 ... -0.06155823 -0.09415536 -0.01941661]
#  [-0.04802262 -0.01395144  0.12706237 ... -0.0639021  -0.03755342 -0.06052431]
#  [ 0.00379441  0.01192298 -0.02876158 ... -0.02021407 -0.0892337   0.05982817]]

The model converts each sentence into a vector containing 384 floating-point values.

The shape (4, 384) represents four sentences, each with a 384-dimensional embedding.

Each number is a coordinate in a high-dimensional vector space. Individual dimensions don't necessarily correspond to specific concepts such as people, places, or activities. Instead, semantic information is distributed across the vector.

The output is abbreviated with ..., but each embedding contains all 384 values.

Looking at these numbers alone doesn't tell us which sentences are semantically related. To find that out, we need to compare their vectors.

Using an Embedding API

Embeddings can also be generated through an API rather than running a model locally.

For example, OpenAI provides an embeddings endpoint:

curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-small",
    "input": "I caught up with John over coffee yesterday."
  }'

The response contains an embedding vector, just like the local model.

The concept is the same, although different models produce different vector dimensions and values. Embeddings generated by different models generally shouldn't be mixed in the same similarity search.

For the rest of this post, I'll continue using the local model so the examples remain reproducible without an API key.

Calculate Cosine Distance

Now that we have embeddings, we need a way to measure how closely two sentences are related.

One commonly used metric is cosine distance, which compares the directions of two vectors rather than their magnitudes.

Let's calculate the distance between two sentences.

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

embeddings = model.encode([
    "I forgot my umbrella and got soaked in the rain.",
    "Who did I meet recently?"
], normalize_embeddings=True)

# Cosine distance using normalized vectors
distance = 1 - np.dot(embeddings[0], embeddings[1])

print(f"Cosine distance: {distance:.4f}")
# Cosine distance: 0.7981

The normalize_embeddings=True option scales each vector to unit length.

For normalized vectors, the dot product (np.dot()) is equivalent to cosine similarity. Subtracting it from 1 gives us cosine distance:

$$\text{cosine distance} = 1 - \text{cosine similarity}$$

Sentence Transformers also provides model.similarity(), which calculates cosine similarity by default. The equivalent calculation would be:

distance = 1 - model.similarity(
    embeddings[0:1], embeddings[1:2]
).item()

However, using NumPy keeps the distance calculation independent of Sentence Transformers. The same approach works with embeddings returned by OpenAI or other embedding APIs.

A smaller cosine distance indicates greater similarity. A distance of 0 means the vectors point in the same direction.

Why Cosine Distance?

There are several ways to compare vectors:

Metric	Description
Cosine distance	Compares vector directions, ignoring magnitude
Euclidean distance	Measures straight-line distance between vectors
Inner product	Measures vector alignment and is affected by magnitude

For text embeddings, we're generally interested in semantic relationships rather than differences in vector magnitude, making cosine distance a practical default.

When embeddings are L2-normalized, cosine distance, Euclidean distance, and inner product produce equivalent rankings. The choice of metric matters more when embeddings aren't normalized or when the model recommends a particular metric.

We'll use cosine distance throughout this post because pgvector supports it directly through the <=> operator.

Find the Most Relevant Memory

Now let's apply cosine distance to our original example and find the memory most relevant to the question:

"Who did I meet recently?"

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

texts = [
    "I forgot my umbrella and got soaked in the rain.",
    "I lifted weights at the gym after work.",
    "I caught up with John over coffee yesterday.",
    "I stayed up late debugging my application.",
]

query = "Who did I meet recently?"

memory_embeddings = model.encode(texts, normalize_embeddings=True)
query_embedding = model.encode(query, normalize_embeddings=True)

# Cosine distance using normalized vectors
distances = 1 - np.dot(memory_embeddings, query_embedding)

for i in np.argsort(distances):
    print(f"{distances[i]:.4f} - {texts[i]}")
# 0.6097 - I caught up with John over coffee yesterday.
# 0.7981 - I forgot my umbrella and got soaked in the rain.
# 0.8151 - I lifted weights at the gym after work.
# 0.8454 - I stayed up late debugging my application.

This time, we're comparing a single query against multiple candidate memories.

The results are sorted in ascending order because smaller cosine distance indicates greater relevance.

As expected, the memory about meeting John ranks first, even though the question uses "meet" while the memory uses "caught up".

This demonstrates semantic retrieval: finding related meanings rather than matching exact words.

However, the closest result isn't guaranteed to be relevant in every case. The ranking depends on the embedding model and the available memories.

In a production system, we'd need to evaluate retrieval quality using representative queries and data.

Introducing pgvector

So far, we've generated embeddings and calculated distances in Python. But generating embeddings for every stored memory on each query would be inefficient.

A better approach is to generate embeddings when memories are created or updated, persist them, and search against the stored vectors.

For this, I'll use pgvector, a PostgreSQL extension that adds vector storage and similarity search capabilities.

Before using it with text embeddings, let's look at a simple example with two-dimensional vectors.

Store Vectors

First, enable the extension and create a table:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE examples (
    id SERIAL PRIMARY KEY,
    embedding VECTOR(2)
);

Insert three vectors:

INSERT INTO examples (embedding)
VALUES
    ('[1,0]'),
    ('[0.8,0.6]'),
    ('[0,1]');

Using two-dimensional vectors makes it easier to see how pgvector calculates distances.

Search by Cosine Distance

Let's find the vectors closest to [1,0].

SELECT
    embedding,
    embedding <=> '[1,0]' AS distance
FROM examples
ORDER BY distance;

The <=> operator calculates cosine distance.

The results are:

 embedding | distance
-----------+----------
 [1,0]     | 0
 [0.8,0.6] | 0.2
 [0,1]     | 1

The first vector has a distance of 0 because it points in the same direction as the query.

The second vector has a distance of 0.2, while the third is orthogonal to the query and has a distance of 1.

The important point is that pgvector doesn't understand the meaning of text. It stores vectors and performs mathematical operations on them.

The embedding model is responsible for producing vectors that represent semantic relationships.

Vector Indexes

Without an index, PostgreSQL performs an exact nearest-neighbor search.

For larger datasets, pgvector supports approximate nearest-neighbor indexes such as HNSW and IVFFlat.

For example, we can create an HNSW index:

CREATE INDEX examples_embedding_hnsw_idx
ON examples
USING hnsw (embedding vector_cosine_ops);

The search query itself doesn't need to change.

PostgreSQL can use the index to accelerate searches, although the query planner may choose a sequential scan for small datasets.

Approximate indexes can improve search performance by trading off some retrieval accuracy.

Embedding Dimensions and Index Limitations

One practical consideration when using pgvector is the number of dimensions in an embedding.

pgvector supports storing vectors with up to 16,000 dimensions, but standard HNSW and IVFFlat indexes using the vector type support up to 2,000 dimensions.

This becomes relevant when using models such as OpenAI's text-embedding-3-large, which produces 3,072-dimensional embeddings by default.

One option is to use a shorter embedding supported by the model.

OpenAI's text-embedding-3 models use Matryoshka Representation Learning (MRL), which allows shorter embeddings to retain useful semantic information.

For example, we can request a 1,536-dimensional embedding from text-embedding-3-large:

curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-large",
    "input": "I caught up with John over coffee yesterday.",
    "dimensions": 1536
  }'

This reduces storage requirements and allows the embedding to fit within pgvector's standard vector index dimension limit.

Another option is pgvector's halfvec type, which supports indexing up to 4,000 dimensions using half-precision values.

The right approach depends on retrieval quality, storage, precision, and performance requirements.

For our example using 384-dimensional embeddings, the standard vector type is sufficient.

Store and Search Embeddings with pgvector

Now that we've covered embedding generation, cosine distance, and pgvector, let's bring everything together.

We'll store the four memories with their embeddings in PostgreSQL and retrieve them using cosine distance.

import psycopg
from pgvector.psycopg import register_vector
from sentence_transformers import SentenceTransformer

DB_URL = "postgresql://user:password@localhost:5432/mydatabase"

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

texts = [
    "I forgot my umbrella and got soaked in the rain.",
    "I lifted weights at the gym after work.",
    "I caught up with John over coffee yesterday.",
    "I stayed up late debugging my application.",
]

query = "Who did I meet recently?"

DIMENSIONS = 384

with psycopg.connect(DB_URL) as conn:
    conn.execute("CREATE EXTENSION IF NOT EXISTS vector")
    register_vector(conn)

    conn.execute(f"""
        CREATE TABLE IF NOT EXISTS journal (
            id BIGSERIAL PRIMARY KEY,
            content TEXT NOT NULL,
            embedding VECTOR({DIMENSIONS}) NOT NULL
        )
    """)

    embeddings = model.encode(texts)

    # Reset the demo data
    conn.execute("TRUNCATE TABLE journal RESTART IDENTITY")

    for content, embedding in zip(texts, embeddings):
        conn.execute(
            """
            INSERT INTO journal (content, embedding)
            VALUES (%s, %s)
            """,
            (content, embedding)
        )

    conn.commit()

    query_embedding = model.encode(query)

    results = conn.execute(
        """
        SELECT
            content,
            embedding <=> %s AS distance
        FROM journal
        ORDER BY embedding <=> %s ASC
        LIMIT 4
        """,
        (query_embedding, query_embedding)
    ).fetchall()

    print(f"Query: {query}")

    for content, distance in results:
        print(f"{distance:.4f} - {content}")
# Query: Who did I meet recently?
# 0.6097 - I caught up with John over coffee yesterday.
# 0.7981 - I forgot my umbrella and got soaked in the rain.
# 0.8151 - I lifted weights at the gym after work.
# 0.8454 - I stayed up late debugging my application.

There are two key differences from the in-memory implementation.

1. Embeddings are persisted

pgvector provides the VECTOR(384) data type, allowing embeddings to be stored alongside their original text.

register_vector(conn) integrates pgvector with psycopg so NumPy arrays can be passed directly as parameters.

In a real application, embeddings would be generated when memories are created or updated, rather than on every query.

2. Similarity search moves into PostgreSQL

Instead of loading every embedding into Python to calculate distances, we let PostgreSQL perform the search.

The <=> operator calculates cosine distance, and the query returns memories ordered by their distance from the query embedding.

As shown in the output, the ranking matches our earlier Python example, with the memory about meeting John appearing first.

For larger datasets, vector indexes can improve search performance without changing the core retrieval logic.

Conclusion

We've explored how embeddings represent sentence meanings numerically, how cosine distance helps identify related text, and how to retrieve relevant sentences even when their wording differs.

We started with a locally runnable embedding model, calculated distances in Python, and then introduced pgvector using simple two-dimensional vectors.

Finally, we combined everything into a PostgreSQL-backed semantic search implementation.

The same retrieval approach can be used in a Retrieval-Augmented Generation (RAG) system, where retrieved text is provided to an LLM as context for generating an answer.

There are further considerations for production systems, including vector indexing, hybrid search, relevance thresholds, and retrieval evaluation.

But the core mechanism is straightforward: generate embeddings, calculate distances, and retrieve the closest matches.