If you want an AI feature that answers questions about your data — your docs, your product, your knowledge base — you'll hear two options: fine-tune a model, or use RAG. For almost every founder, RAG is the right first answer, and understanding why saves you a lot of money and grief.
What RAG actually is
Retrieval-augmented generation is simpler than the acronym suggests. Instead of hoping the model already knows your information, you fetch the relevant pieces of your content at question time and hand them to the model as context. The model then answers using what you gave it.
The flow is three steps:
- Retrieve — find the chunks of your content most relevant to the user's question.
- Augment — insert those chunks into the prompt alongside the question.
- Generate — the model answers, grounded in the retrieved material.
The model isn't remembering your data; it's reading it fresh each time, like an open-book exam.
Why it beats fine-tuning for most cases
Fine-tuning bakes knowledge into the model's weights. That sounds appealing until you hit the practical problems: it's expensive, it's slow to update, and it teaches style far better than it teaches facts. When your content changes — and it always does — you'd have to retrain.
RAG sidesteps all of that:
- Fresh by default. Update a document and the next answer reflects it, no retraining.
- Cheaper. No training runs; you pay for retrieval and one inference call.
- Cite-able. Because answers come from specific chunks, you can show sources and build trust.
- Less hallucination. Grounding the model in real text sharply reduces invented answers.
Fine-tune when you need to change how the model behaves — tone, format, a narrow skill. Use RAG when you need it to know what your business knows.
How the retrieval works
To find relevant chunks, you convert text into embeddings — numeric vectors that capture meaning — and store them. At question time you embed the query and find the closest stored vectors by similarity. You don't need heavy infrastructure to start: pgvector, an extension for PostgreSQL (and available in Supabase), handles similarity search inside the database you already run. Reach for a dedicated vector database only when scale demands it.
Getting RAG right in practice
A naive RAG demo works; a reliable RAG feature takes care:
- Chunk thoughtfully. Too large and you bury the answer; too small and you lose context. Chunk by semantic boundaries, not arbitrary character counts.
- Retrieve enough, then re-rank. Pull a handful of candidates and reorder them so the best context leads.
- Handle "no answer." When nothing relevant is found, the model should say so, not improvise.
- Evaluate continuously. Keep a set of real questions with known answers and test retrieval quality as your content grows.
The most common failure isn't the model — it's retrieval handing over the wrong chunks. Invest your effort there.
If you're building a feature that answers questions over your own content and want it accurate, grounded, and cheap to run, talk to us.
Originally published on the Doktouri Agency blog. We build web, mobile, SaaS, and AI products — let's talk.
Top comments (0)