I Built a Local RAG Pipeline with TypeScript, PostgreSQL and pgvector

José Henrique Oliveira de Carvalho — Sat, 26 Sep 2026 16:09:29 +0000

Originally published on my personal website:

https://josehenriquedev.com/blog/como-implementei-o-backend-do-meu-site-pessoal-rag-local-com-bun-elysia-e-pgvector

I wanted my personal portfolio to be more than a collection of static pages.

The idea was to build an AI assistant capable of answering questions about my background, projects, experience, and technical decisions — while keeping the answers grounded in my actual data.

Instead of sending everything to a commercial embedding API and adding a dedicated vector database, I decided to build the retrieval pipeline myself using TypeScript.

The final architecture looks like this:

Markdown
   ↓
Parsing + Chunking
   ↓
Question enrichment
   ↓
Local embeddings
   ↓
PostgreSQL + pgvector
   ↓
Similarity search
   ↓
Relevance threshold
   ↓
LLM via Groq
   ↓
Response

Stack

The backend uses:

Bun
Elysia.js
TypeScript
PostgreSQL
pgvector
Drizzle ORM
@huggingface/transformers
Xenova/multilingual-e5-small
Groq
openai/gpt-oss-120b

The interesting part is that the embedding generation happens locally.

There is no external embedding API in the retrieval pipeline.

1. Markdown as the knowledge base

Instead of storing my professional information directly in the database, I keep the source knowledge in versioned Markdown files.

For example:

profile.md
experience.md
projects.md

Each document contains frontmatter with structured metadata.

Before generating embeddings, the Markdown is split into smaller chunks using LangChain's RecursiveCharacterTextSplitter:

const splitter = RecursiveCharacterTextSplitter.fromLanguage("markdown", {
    chunkSize: 800,
    chunkOverlap: 50,
});

The goal is to avoid embedding huge documents as a single vector.

2. Enriching the documents with probable questions

A technically correct document does not necessarily have the same semantic representation as the question a visitor will ask.

For example, a document might contain:

José has experience with React, Next.js, Node.js and PostgreSQL.

But the visitor may ask:

What technologies does José use?

To improve retrieval, I added probable questions to the text before generating the embedding.

const questionsText = Array.isArray(data.probable_questions)
    ? `Perguntas Frequentes Relacionadas:
- ${data.probable_questions.join("
- ")}

`
    : "";

const textToEmbbed = `Documento: ${data.title}
Tipo: ${data.type}
Locale: ${itemLocale}
${questionsText}Conteúdo:
${cleanContent}`;

This gives the embedding model additional semantic signals that are closer to the kinds of queries users are likely to make.

3. Generating embeddings locally

For embeddings, I use Xenova/multilingual-e5-small through @huggingface/transformers.

The model runs locally on CPU:

export class EmbbedingService implements IEmbedd {
    private extractor!: FeatureExtractionPipeline;

    async initialize() {
        this.extractor = await pipeline(
            "feature-extraction",
            "Xenova/multilingual-e5-small",
            { device: "cpu" }
        );
    }

    async embbed(text: string): Promise<number[]> {
        const output = await this.extractor(text, {
            pooling: "mean",
            normalize: true,
        });

        return Array.from(output.data);
    }
}

The resulting vectors have 384 dimensions.

One important detail with E5 models is the use of prefixes.

For stored documents:

passage: <content>

For user queries:

query: <question>

These prefixes are part of the model's expected input format.

I also wanted the system to work naturally with both Portuguese and English content, which is important for a developer portfolio.

4. PostgreSQL as the vector database

I didn't want to introduce another database just for vector search.

Since the project already uses PostgreSQL, I added pgvector.

The table stores both the original content and its embedding:

export const professionalProfileTable = pgTable("profile", {
    id: serial().primaryKey(),
    title: text(),
    content: text(),
    type: text({ enum: ["project", "experience", "profile"] }),
    locale: text(),
    metadata: json(),
    embedding: vector("embedding", { dimensions: 384 }).notNull()
});

For similarity search, pgvector provides operators such as <=>.

Using Drizzle, the query looks like this:

const similarity = sql<number>`
    (${professionalProfileTable.embedding}
    <=> ${JSON.stringify(embbeding)}::vector)
`;

let query = db
    .select({
        id: professionalProfileTable.id,
        title: professionalProfileTable.title,
        content: professionalProfileTable.content,
        similarity: similarity
    })
    .from(professionalProfileTable)
    .orderBy(similarity)
    .limit(5);

Here, the returned value represents cosine distance.

Lower distance means greater similarity.

For this project, PostgreSQL + pgvector was enough to keep the retrieval layer simple without introducing a separate vector database.

5. Filtering retrieval results

Retrieval alone isn't enough.

Even if the vector search returns the closest chunks, they may still be irrelevant to the question.

So I added a similarity threshold:

const gatherKnowledge = await this.retrieve.exec(input, locale);

const relevantChunks =
    gatherKnowledge?.filter(
        k => (k.similarity as number) < 0.35
    ) || [];

If no chunk passes the threshold, I don't inject arbitrary retrieved context into the prompt.

Instead, the LLM receives a basic prompt instructing it not to invent information.

This is important because a RAG system doesn't automatically prevent hallucinations.

The retrieval layer itself needs rules for deciding when retrieved information is relevant enough to be used.

6. The complete pipeline

Putting everything together:

             ┌──────────────────┐
             │ Markdown files   │
             │ profile.md       │
             │ experience.md    │
             │ projects.md      │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Parsing +        │
             │ Chunking         │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Question         │
             │ enrichment       │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Local embeddings │
             │ multilingual-e5  │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ PostgreSQL       │
             │ + pgvector       │
             └────────┬─────────┘
                      │
               User question
                      │
                      ▼
             ┌──────────────────┐
             │ Query embedding  │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Vector search    │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Similarity       │
             │ threshold        │
             └────────┬─────────┘
                      │
                      ▼
             ┌──────────────────┐
             │ Groq LLM         │
             └────────┬─────────┘
                      │
                      ▼
                   Response

The LLM is therefore only one part of the system.

The retrieval pipeline determines what information is allowed to reach it.

Why I built it this way

The main goal wasn't simply to make an AI chatbot.

I wanted to understand the retrieval pipeline end to end.

That meant dealing with questions such as:

How should documents be chunked?
How can retrieval be improved without changing the LLM?
How should multilingual content be embedded?
When is a retrieved chunk actually relevant?
Do I really need a dedicated vector database?
What should happen when retrieval finds nothing useful?
How can I keep the system lightweight?

The result is a small RAG backend with a relatively simple architecture:

Bun + Elysia
      +
PostgreSQL + pgvector
      +
Local embeddings
      +
Groq inference

The most important lesson for me was that the LLM is not the whole system.

A large part of the quality of a RAG application comes from the data representation, chunking strategy, embeddings, retrieval, and relevance filtering that happen before the model generates an answer.

Final thoughts

Building the retrieval layer myself also made the system easier to reason about.

The embedding model runs locally, PostgreSQL handles both relational and vector data, and the LLM only receives context that passes the retrieval rules.

It is not a universal architecture, and a dedicated vector database can make sense for larger or more complex workloads.

But for a personal portfolio, PostgreSQL + pgvector was enough to build the entire retrieval pipeline without adding another infrastructure component.

You can read the full article, including more implementation details, on my website: