Abdraouf Benabdsselem

Posted on Sep 11

# I Built a RAG Document Assistant with FastAPI, React, FAISS and Ollama

#rag #llm #fastapi #react

I Built a RAG Document Assistant with FastAPI, React, FAISS and Ollama

I've been learning more about Retrieval-Augmented Generation (RAG), and I wanted to understand what actually happens inside a RAG application.

So instead of building another simple chatbot, I decided to build a complete application around documents.

The result is CloudRAG:

https://github.com/abderaoufsec/CloudRAG

What does it do?

The idea is straightforward.

You upload documents, then ask questions about them.

Instead of sending the question directly to the LLM, the application first searches the uploaded documents for relevant information. That information is then given to the LLM as context.

The simplified flow looks like this:

Document
   ↓
Text extraction
   ↓
Chunking
   ↓
Embeddings
   ↓
FAISS
   ↓
Relevant chunks
   ↓
Ollama
   ↓
Answer + sources

The stack

For the backend I used:

Python
FastAPI
SQLite
SQLAlchemy

For the RAG pipeline:

SentenceTransformers
FAISS
Ollama

For the frontend:

React
Vite

I also used Docker and Pytest during development.

Why FAISS?

I wanted the project to work locally without depending on a paid external service.

FAISS makes it possible to store and search the embeddings locally, which made it a good fit for the current version of the project.

What I learned

The biggest thing I learned is that RAG isn't simply:

"Send documents to an LLM."

There are several steps that affect the final result.

How you split documents matters.

How you generate embeddings matters.

How you retrieve relevant chunks matters.

And you need a way to evaluate whether your retrieval is actually useful.

That's why I also added retrieval evaluation and testing to the project.

What about Qdrant?

I also experimented with Qdrant Cloud and added an integration for it.

However, I wasn't able to successfully deploy and validate the Qdrant part.

So the current working implementation uses FAISS locally.

I'd like to come back to the Qdrant deployment later and compare it with the local FAISS setup.

Final thoughts

This project was mainly a learning exercise, but it gave me a much better understanding of what goes into building a RAG application.

I now have a better understanding of:

document processing
chunking
embeddings
vector search
retrieval
local LLMs
API design
RAG evaluation

The source code is available here: