Azariah Arthur

Posted on Sep 2

Building a RAG Pipeline with FastAPI — Part 1: From Documents to Vector Data

#ai #fastapi #python #rag

I've been experimenting with Retrieval-Augmented Generation (RAG), and instead of building the entire application in one massive project, I decided to break it into smaller pieces.

The goal is to eventually build a document intelligence application that can take a collection of documents and allow users to interact with the information inside them.

I'm splitting the build into four stages:

Document ingestion
Retrieval
Augmented generation
The complete application

This article covers Part 1: document ingestion.

The Problem

A document is easy for a human to read.

A PDF, Word document, or text file can contain exactly the information we're looking for, but an AI system can't simply perform a semantic search over the raw file.

We first need to transform the document into a representation that our system can search efficiently.

The basic pipeline looks like this:

Document
   ↓
Text Extraction
   ↓
Chunking
   ↓
Embeddings
   ↓
Vector Database

That is what I set out to build for the first stage of this project.

Step 1 — Extracting the Text

The first step is getting the actual text out of the document.

The system currently supports:

.txt
.pdf
.docx

Once the text has been extracted, we can begin preparing it for the embedding model.

But there is a problem.

A document can contain thousands or even millions of characters.

Trying to embed an entire document as a single vector would lose too much information and make retrieval much less useful.

That's where chunking comes in.

Step 2 — Chunking the Document

Instead of treating the document as one giant piece of text, we divide it into smaller chunks.

For this project, I'm using recursive chunking.

The basic idea is to progressively split the text using increasingly smaller separators until the resulting pieces fit within the desired chunk size.

Conceptually:

Large Document
      ↓
Paragraphs
      ↓
Sentences
      ↓
Smaller Text Segments

Chunk size and overlap matter here.

If chunks are too large, retrieval can return a lot of irrelevant information.

If they're too small, we can lose the surrounding context that makes a piece of text meaningful.

Overlap helps preserve some of that context between neighboring chunks.

The result is something more like:

Document
 ├── Chunk 1
 ├── Chunk 2
 ├── Chunk 3
 ├── Chunk 4
 └── ...

Now we have manageable pieces of text that can be converted into vectors.

Step 3 — Generating Embeddings

The next step is turning each chunk into an embedding.

An embedding is essentially a numerical representation of the meaning of a piece of text.

For this project, I'm using BAAI/bge-m3 through Sentence Transformers.

The process looks like:

"Some piece of document text..."
              ↓
        BGE-M3 Model
              ↓
     [0.012, -0.083, ...]

The exact numbers aren't particularly useful to look at individually.

What matters is that semantically similar pieces of text should have similar representations in vector space.

That gives us a way to perform semantic search later.

Step 4 — Storing the Vectors

Once the embeddings have been generated, we need somewhere to store them.

I'm using Qdrant as the vector database.

Each stored record contains the vector along with information that allows us to associate it with the original chunk of text.

Conceptually:

Qdrant

Vector
   +
Document Chunk
   +
Metadata

This gives us the foundation for the retrieval stage.

Later, when a user asks a question, we can embed that question and search Qdrant for vectors that are semantically similar.

But we're not there yet.

The Current Pipeline

At the end of Part 1, the system can take a document and process it through the entire ingestion pipeline:

             DOCUMENT
                 ↓
          TEXT EXTRACTION
                 ↓
          RECURSIVE CHUNKING
                 ↓
             BGE-M3
                 ↓
             EMBEDDINGS
                 ↓
              QDRANT

The important distinction is that we haven't built the question-answering system yet.

We've only built the foundation that makes retrieval possible.

And that's intentional.

What's Next?

Now that the documents have been converted into searchable vectors, we have a new problem:

How do we find the right information when someone asks a question?

That's the focus of Part 2.

We'll take a user query, convert it into an embedding, search the vector database, and retrieve the most relevant pieces of the original documents.

Eventually, those retrieved pieces will become the context used by an LLM.

The complete system will look more like:

Documents
    ↓
Ingestion
    ↓
Vector Database
    ↓
Retrieval
    ↓
Relevant Context
    ↓
LLM
    ↓
Answer

But for now, we have the first major piece working.

Documents → Chunks → Embeddings → Qdrant

I'm documenting the entire build as I go, including the problems and design decisions along the way.

DEV Community