RAGs and Embedded Models Simplified

tl;dr

• RAG stands for: Retrieval-Augmented Generation.
• became massively popular in 2023.
• The core concept? Making LLMs appear more knowledgeable by shoving extra info into the prompt.

Context Injection

So the idea behind RAG is straightforward:

If you're a travel consultant trying to answer questions about travel costs, you could just shove all the relevant information into the prompt, things like:

• Cab prices.
• Train prices.
• Hotel/motel prices.
• ...

The LLM, when predicting the next tokens, will generate responses consistent with that context.

This works! You can try it yourself with deepseek, give it some extra information alongside your question, and it'll incorporate that context into its answer.

💡 Pro Tip

LLMs are notorious at being nondeterministic, so if we are going to give a financial advice it is always a good idea to mention this 😅. For example in this example the application must tell their clients that they need to treat the numbers as ballpark figures.

Overloaded LLM 😉

The obvious issue is that this approach doesn't scale. If you're a consultant agency for traveling, you will be answering all sort of questions for different destinations. You cannot cram all that information into a prompt. Issues with this approach:

1. It exceed context limits.
2. You are setting up LLM up to fail with an overwhelming amount of irrelevant information.

Solution -- Semantic Search & Embedded Models

Rather than sending all data, select a relevant subset that's most likely to answer the question. This requires what's called a fuzzy search or semantic search, finding information based on meaning rather than just keywords.

So now that we know about semantic search it is time to look at embedding models, also called an encoder. It is a special type of LLM. :

Take text as input -> Output a list of numbers (a vector) that represents the meaning of that text

So "How much does it cost to rent a hotel in Songapore?" and "What's the ticket price Singapore?" would produce similar vectors—even though the words are completely different!

BTW you can see one example of this here: Building AI-Powered Search and RAG with PostgreSQL and Vector Embeddings

It's important to understand that the LLM doesn't know anything about embeddings or vectors. It just receives a prompt with relevant context and predicts the most likely next tokens based on that context.

Chunking

When vectorizing data, you need to decide how to break up your documents/data:

• Whole document as one vector?
• Paragraphs?
• Sentences?

This way you won't be returning irrelevant data.

RAG is the go-to Technique for

• Expert knowledge workers with expertise about company products.
• HR systems that need to know all company policies.
• Customer support agents with access to product documentation
• Internal knowledge bases for employee questions.

Traditional RAG vs Agentic RAG

Traditional RAG	Agentic RAG
Linear workflow	Iterative, autonomous workflow
Code controls retrieval	LLM decides retrieval strategy
Single vector retrieval tool	Multiple tools (vector, SQL, etc.)
Fixed retrieval parameters	Dynamic, can retry with different params

Huge Context Windows

You might ask yourself with 1M+ token context windows, why not just put everything in the prompt? But that's not gonna scale. You can easily have gigabytes of documents. Plus, cramming irrelevant content wastes compute and can actually harm performance.

Building RAGs

1. Data Ingest (ETL + Chunking + Vectorizing).
   

   Source Data → Extract → Transform → Chunk → Vectorize → Load to Vector DB
   

   ETL stands for Extract, Transform, and Load.

2. Question Answering (The RAG Pipeline).
   

   User Question → Vectorize → Query Vector DB → Retrieve Relevant Chunks → Generate Response
   

If you want to build RAG yourself:

1. You need to setup a vector database (Supabase, Pinecone, Weaviate, or PostgreSQL with pgvector).
2. Choose an embedding model (OpenAI's text-embedding-3, Cohere, or Qwen3 Embedding)
3. Build data ingest pipelines to load and vectorize your documents.
4. Implement the retrieval workflow (can be code-controlled or agentic).
5. Measure, measure, measure and iterate!

Why We Need the Embedding Model?

Honestly the first time I was reading about RAGs and embedding models I was confused as to why we need it but a vector Database (like Pinecone, Milvus, or pgvector) is just a storage and indexing engine.

It stores lists of numbers (vectors) and organizes them using special math (like HNSW or IVF indexes) so it can find nearest neighbors really fast.

It does not contain a natural language processing model. It has no idea what words mean. If you feed it raw text ("Ticket to London costs $500"), it will throw an error because it only accepts numerical arrays.

To store data, you MUST pre-compute the vector.