Part 2 of 8 — RAG Article Series
Previous: Why AI Gets Things Wrong (Part 1)
TechNova is a fictional company used as a running example throughout this series.
Same Question, Different Answer
Same customer. Same question. The WH-1000 headphones were bought last month, and they want to know about returns.
This time, the AI assistant does not answer from what it learned during training. Before generating a response, it retrieves TechNova's current return policy — the document in the CMS, updated last quarter, version 4.1. The policy says fifteen days. The assistant reads it, and responds: fifteen days, and the window has closed.
The customer is disappointed, but they get the right answer. No escalation. No support agent cleaning up after the model. No confident wrong answer delivered with the authority of a system that cannot tell old facts from current ones.
The model did not get smarter. It did not retrain. It did not receive a fine-tuning update with the latest policy documents. The only thing that changed is where the answer came from.
What Just Changed
In Part 1, the model answered from its internal state — a compressed snapshot of everything it learned during training. That snapshot included a return policy that was accurate six months ago and wrong today. The model had no way to know the difference.
In the scenario above, the policy fact comes from retrieved context, not from what the model remembered. The system retrieved the current document from TechNova's knowledge base, placed it in the model's context, and asked it to generate. The model's answer reflected what the document actually says — right now, not six months ago.
RAG changes the model's source of truth at answer time. The model's reasoning capability is unchanged. Instead of relying on frozen parameters, it relies on retrieved context — context that can be updated, versioned, and kept current without touching the model itself.
The full name is Retrieval-Augmented Generation. Retrieve first, then generate. The retrieval step is what makes the difference between the wrong answer in Part 1 and the right answer above.
RAG Is a Pattern, Not a Product
RAG is not a tool you buy. It is a way of structuring the system.
This matters because it is easy to confuse the pattern with the tools used to build it. A vector database is one way to store knowledge the system can search. An embedding model is one way to help the system find documents by meaning, not just exact words. A prompt template is one way to format the retrieved text and question into a single prompt for the model. None of them are RAG. RAG is the system structure: retrieve relevant knowledge first, then generate from it.
Six Components in One Sentence Each
Every RAG system, regardless of implementation, has six components. They run in order, each feeding the next.
Query. The question or request that arrives from the user — in TechNova's case, "What is the return policy for the WH-1000?"
Retriever. The component that takes the query and finds relevant content from the knowledge base.
Knowledge base. The external store of documents, records, or data that the retriever searches — TechNova's policy documents, troubleshooting guides, and product specs.
Retrieved context. The specific content the retriever returns — the chunks of text that will be placed in front of the model.
Prompt assembly. The step that combines the retrieved context with the original query into a single input for the model.
Generation. The model reads the assembled prompt and produces an answer grounded in the retrieved context, not its training data.
Those six components run in sequence. The query enters, context is retrieved, the model generates. Everything in between is a design decision. Parts 3 and 4 examine those decisions and the ways they fail.
Knowledge vs Reasoning — The Line That Matters
People often get confused about what RAG actually improves. It does not make the model smarter. It does not improve its ability to reason, combine information, or draw conclusions. A model that struggles with multi-step logic will still struggle with multi-step logic after you add retrieval. RAG changes what the model knows at the moment it answers, not how well it thinks.
This distinction matters because it shows which problems RAG solves and which it does not. If TechNova's AI assistant gives the wrong return policy because the model never saw the updated document, that is a knowledge problem. RAG fixes it. If the assistant sees the correct document but misinterprets a conditional clause — "fifteen days from date of delivery, not date of purchase" — that is a reasoning problem. RAG does not fix it. The retriever did its job. The model did not.
When something goes wrong in a RAG system, the first question is always: did the retriever return the right content? If yes, the problem is generation. If no, the problem is retrieval. Learning to separate retrieval problems from generation problems is the most useful thing you can take from this series.
RAG matters because it changes the model's source of truth at answer time, not because it adds more boxes to the architecture.
Three Takeaways
1. RAG is a pattern: retrieve relevant context, then generate an answer grounded in that context.
No vendor, no framework, no specific stack defines RAG. The pattern is simple: retrieve first, then generate using external knowledge.
2. Retrieval quality sets the ceiling for the answer.
If the retriever returns the wrong content, the model will produce a well-reasoned wrong answer. The model still matters — but it cannot rescue bad retrieval.
3. RAG addresses knowledge currency. The model still handles reasoning.
RAG changes where knowledge comes from. It does not change how well the model reasons over that knowledge.
Part 3 breaks the pattern into two operational shifts — one that prepares knowledge before any question is asked, and one that answers the question at runtime — and shows where each shift fails.
Next: How RAG Works: The Complete Pipeline (Part 3 of 8)
Top comments (0)