In Part 1, we explored why modern AI systems need more than just good prompts.
In Part 2, we learned how tokens, context windows, and memory shape the quality of an AI application's responses.
By now, we know one important fact:
The quality of an AI system depends heavily on the quality of the context it receives.
But that naturally raises another question.
Where does all that context actually come from?
Imagine asking an AI assistant:
What is our company's leave policy?
Or perhaps:
Explain this private function in our GitHub repository.
Or even:
How many orders were placed today?
How can an AI answer questions about information that was never part of its training data?
It wasn't trained on your company's HR handbook.
It doesn't know your private GitHub repository.
It certainly doesn't know how many orders your application processed five minutes ago.
Yet modern AI applications answer these questions surprisingly well.
How?
Because today's AI applications do far more than simply send your prompt to a language model.
They search, retrieve, call tools, and assemble relevant information before asking the model to generate a response.
That's exactly what we'll explore in this article.
🧠 The Biggest Misconception About AI
Many people imagine AI applications working like this:
It feels simple.
Ask a question.
Receive an answer.
But modern AI systems are much more sophisticated.
A more accurate picture looks like this:
Notice something important.
The LLM isn't doing everything.
Instead, the application surrounding the LLM is responsible for gathering the right information before asking the model to generate a response.
This is Context Engineering in practice.
Why Language Models Don't Know Everything
Large Language Models are trained on enormous amounts of publicly available text.
That training gives them remarkable reasoning and language capabilities.
However, training has limits.
A model typically cannot know:
- Your company's internal documentation
- Private GitHub repositories
- Customer databases
- Current inventory
- Live weather
- Today's stock prices
- This morning's support tickets
- The latest production deployment
Even if the information exists somewhere, it may have been created after the model finished training.
This distinction is important.
The model isn't "searching the internet" every time you ask a question.
Instead, the surrounding application decides whether additional information needs to be retrieved.
📚 Retrieval-Augmented Generation (RAG)
One of the most common techniques for providing that information is called Retrieval-Augmented Generation, commonly known as RAG.
The name sounds complicated.
The idea isn't.
Think about an open-book exam.
During the exam, you're allowed to search the textbook.
You don't memorize every page beforehand.
Instead, you:
- Read the question.
- Find the relevant chapter.
- Read only those pages.
- Write the answer.
That's exactly how RAG works.
The language model doesn't memorize every document.
Instead, the application retrieves relevant information first, then includes it as part of the context sent to the model.
Conceptually:
The model hasn't learned anything new.
It simply received better context.
RAG Does Not Retrain the Model
This is another common misconception.
Sometimes people say:
"We added documents to our RAG system, so we trained the model."
That's not how RAG works.
Training changes the model's parameters through a computationally intensive learning process.
RAG doesn't modify the model at all.
Instead, it changes the information available during inference.
Think of it this way.
Training is like teaching someone a new language.
RAG is like handing them the correct reference book before asking a question.
The person's knowledge hasn't changed.
They simply have access to additional information.
A Practical Example
Suppose your company has thousands of pages of internal documentation.
A developer asks:
How do I deploy the billing service?
Without RAG:
The model can only rely on general knowledge.
It might produce a reasonable deployment process - but not your company's process.
With RAG:
The application first searches your documentation.
Perhaps it retrieves:
- Billing deployment guide
- Kubernetes manifest
- Environment variables
- Rollback procedure
Only these relevant documents are attached to the request.
The model now has enough context to generate an accurate answer.
Notice something interesting.
The user still asked the same question.
The improvement came entirely from better context.
🎯 Good RAG Isn't About Retrieving More
Many beginners assume:
More documents = Better answer
Usually, the opposite is true.
Imagine asking:
How do I reset my password?
Would you rather provide:
- The password reset documentation
or
- Every document in your company?
Clearly, the first option is better.
The goal of RAG isn't to retrieve everything.
The goal is to retrieve only what matters.
This connects directly to what we learned in Part 2.
Remember:
More context doesn't automatically produce better answers.
Retrieving unnecessary information wastes tokens, increases latency, and may distract the model from the information that actually matters.
Good RAG systems focus on precision, not volume.
A Simple Go Example
Imagine you have internal documentation stored in a database.
Without RAG, your application might simply forward the user's question to the LLM:
prompt := "How do I deploy the billing service?"
response := callLLM(prompt)
The model has no access to your documentation.
Now imagine a simplified RAG workflow.
question := "How do I deploy the billing service?"
docs := searchKnowledgeBase(question)
context := fmt.Sprintf(`
Relevant Documentation:
%s
User Question:
%s
`, strings.Join(docs, "\n\n"), question)
response := callLLM(context)
Notice something important.
The model wasn't changed.
The only difference is that your application retrieved relevant information before sending the request.
That's the essence of Retrieval-Augmented Generation.
How Does the Application Find the Right Documents?
At this point, you might be wondering:
"How does the application know which documents are relevant?"
That's an excellent question.
Modern RAG systems typically rely on embeddings and vector search to retrieve semantically similar documents rather than performing simple keyword matching.
Those topics deserve their own deep dive because they involve concepts like:
- Embedding models
- Vector databases
- Similarity search
- Chunking strategies
- Ranking
For now, it's enough to understand that the retrieval system attempts to find the most relevant pieces of information and passes only those into the context.
The important takeaway is this:
The LLM isn't searching your documents directly. The application retrieves them first.
🤖 Retrieval Gives Knowledge. But What If the AI Needs to Take an Action?
So far, we've looked at questions like:
What is our refund policy?
or
Explain this deployment guide.
These are knowledge retrieval problems.
But now consider a different request.
Create a new GitHub issue.
Or:
Send this report to Slack.
Or:
Book a meeting for tomorrow at 3 PM.
Reading documentation isn't enough anymore.
The AI now needs to perform an action.
How does that happen?
Does the language model directly call APIs?
Can it create GitHub issues on its own?
Can it send Slack messages by itself?
Not quite.
This is where Tool Calling enters the picture - and it's one of the most misunderstood concepts in modern AI systems.
🔧 Tool Calling: Giving AI the Ability to Act
So far, we've seen how RAG helps AI answer questions by retrieving relevant information.
But sometimes answering a question isn't enough.
Imagine asking an AI assistant:
Create a GitHub issue titled "Fix login bug".
Or:
Send today's deployment report to Slack.
Or:
What's the current CPU usage of our production server?
These aren't knowledge problems anymore.
They're action problems.
The AI now needs to interact with external systems.
This is where Tool Calling comes into the picture.
🤔 Does the LLM Execute APIs?
One of the biggest misconceptions is that language models directly execute code or call APIs.
They don't.
A Large Language Model generates text.
That's its primary capability.
When an application enables Tool Calling, the model can decide:
"To answer this request correctly, I need to use a specific tool."
The application then executes that tool and returns the result to the model.
The model never talks to GitHub, Slack, or your database directly.
The application does.
Think of it like this:
The LLM makes the decision.
The application performs the action.
This separation is important because it allows developers to validate permissions, handle authentication, retry failures, log requests, and control exactly what the model is allowed to do.
A Simple Go Example
Suppose your application exposes a tool for fetching the current weather.
type WeatherTool struct{}
func (WeatherTool) Execute(city string) (string, error) {
return fmt.Sprintf("%s: 29°C, Clear Sky", city), nil
}
When a user asks:
What's the weather in Ahmedabad?
The application might work like this:
tool := WeatherTool{}
weather, err := tool.Execute("Ahmedabad")
if err != nil {
log.Fatal(err)
}
prompt := fmt.Sprintf(`
Weather Information:
%s
Answer the user's question naturally.
`, weather)
response := callLLM(prompt)
fmt.Println(response)
Notice something important.
The weather API wasn't called by the language model.
It was called by your Go application.
The model simply received the weather information as additional context before generating the response.
Once again, Context Engineering is doing the heavy lifting.
📚 RAG Retrieves Knowledge. Tools Perform Actions.
This distinction is worth remembering.
Imagine asking these questions:
What is our leave policy?
You need documentation.
RAG is the right choice.
Approve leave request #1245.
Now the AI must interact with your HR system.
A tool needs to execute that action.
Another example:
Explain today's failed deployment.
Retrieve deployment logs.
RAG.
Restart the Kubernetes deployment.
Execute kubectl or a deployment API.
Tool Calling.
One retrieves information.
The other performs work.
Many production AI applications use both together.
🧩 What Happens When You Have Hundreds of Tools?
Now imagine your AI assistant can interact with:
- GitHub
- Jira
- Slack
- Google Drive
- PostgreSQL
- Filesystem
- Kubernetes
- AWS
- Internal APIs
Each system exposes its own:
- Authentication
- API format
- Documentation
- SDK
- Request structure
Without standardization, integrating each tool becomes increasingly complex.
That's exactly the problem Model Context Protocol (MCP) was designed to solve.
🌉 What Is MCP?
Think of USB.
Before USB existed:
Every device required a different connector.
Different cables.
Different ports.
Different drivers.
USB introduced a common standard.
Manufacturers still built different devices.
But computers interacted with them in a consistent way.
MCP plays a similar role for AI applications.
Instead of every tool exposing a completely different interface, MCP defines a standard way for AI applications to discover and interact with tools.
The protocol doesn't replace your APIs.
It standardizes how AI systems communicate with them.
🏗️ A Simplified MCP Architecture
Each MCP server exposes capabilities using the same protocol.
The client no longer needs to understand dozens of different integrations individually.
This makes adding new tools significantly easier.
MCP Is Not a Replacement for APIs
Another common misconception is:
"MCP replaces REST APIs."
It doesn't.
REST APIs still exist.
Databases still exist.
GitHub APIs still exist.
Slack APIs still exist.
MCP simply provides a standardized interface that AI applications can use to interact with those systems more consistently.
Think of MCP as a translator sitting between AI applications and external tools.
Putting It All Together
Let's revisit our original question:
Create a GitHub issue summarizing today's failed deployment.
Here's what might happen behind the scenes.
User
│
▼
AI Application
│
┌───────────────┼────────────────┐
▼ ▼
Retrieve Deployment Logs GitHub Tool
│ │
└───────────────┬────────────────┘
▼
Build Context
│
▼
LLM
│
▼
"Create issue titled...
Include deployment summary..."
Notice something interesting.
This single request used:
- Retrieval (deployment logs)
- Tool Calling (GitHub)
- Context Engineering (combining everything)
- The LLM (reasoning and language generation)
Modern AI assistants rarely rely on only one technique.
They're orchestration systems.
RAG vs Tool Calling vs MCP
These concepts are often confused because they frequently appear together.
Here's an easy way to distinguish them.
| Feature | RAG | Tool Calling | MCP |
|---|---|---|---|
| Primary Goal | Retrieve information | Execute actions | Standardize tool integrations |
| Changes external state | ❌ No | ✅ Yes (can) | Depends on the tool |
| Reads knowledge | ✅ Yes | Sometimes | Depends |
| Calls external systems | Indirectly | ✅ Yes | Provides the interface |
| Typical Example | Company documentation | GitHub, Slack, Database | GitHub MCP Server |
The important point is that these technologies complement one another rather than compete.
🧠 A Simple Mental Model
Whenever you're unsure which concept applies, ask yourself one question:
"What is the AI trying to do?"
If the answer is:
Find information
→ Think RAG
Perform an action
→ Think Tool Calling
Connect AI to many different tools consistently
→ Think MCP
This simple mental model eliminates much of the confusion surrounding these terms.
🚀 Production Best Practices
If you're building AI-powered applications, keep these principles in mind.
✅ Retrieve only relevant information
More documents don't automatically improve answers.
High-quality retrieval almost always beats high-volume retrieval.
✅ Keep retrieved context concise
Every additional token increases cost and latency.
Only include information that helps answer the current question.
✅ Design tools to do one thing well
Instead of creating one massive "do everything" tool, expose smaller, focused capabilities.
They're easier for both developers and language models to use correctly.
✅ Validate every tool invocation
Never assume the model made the correct decision.
Your application should validate permissions, inputs, authentication, and business rules before executing any action.
✅ Treat tool outputs as untrusted input
External systems can fail.
APIs can return unexpected data.
Always handle errors gracefully before sending results back to the model.
✅ Monitor retrieval quality
Poor RAG often isn't caused by the language model.
It's caused by retrieving the wrong documents.
Measure retrieval quality just as carefully as you evaluate model responses.
🏁 Final Thoughts
When people first begin exploring AI, they often focus on the language model itself.
But as you've seen throughout this series, the model is only one part of the system.
The real intelligence often comes from the application surrounding it.
That application decides:
- What information should be retrieved.
- Which tools should be available.
- How context is assembled.
- What actions are permitted.
- How responses are validated.
This is why two applications built on the same underlying model can produce dramatically different results.
The difference usually isn't the model.
It's the quality of the surrounding engineering.
And that's exactly what Context Engineering is all about.
📌 Key Takeaways
- Large Language Models don't automatically know your private or real-time data.
- RAG improves responses by retrieving relevant information before sending the request to the model.
- RAG doesn't retrain or modify the model - it enriches the context during inference.
- Tool Calling enables AI applications to perform actions by letting the application execute external operations.
- The language model decides when a tool is needed, but the application executes it.
- MCP provides a standardized way for AI applications to interact with many different tools.
- Effective AI systems combine retrieval, tool execution, and carefully constructed context to produce accurate and useful responses.
🚀 Coming Up in Part 4
So far, we've explored:
- Why prompts alone aren't enough.
- How tokens and context windows influence AI.
- How AI retrieves information and interacts with external tools.
But one fascinating question still remains:
How does an AI decide what to do first?
If a request requires searching documentation, querying a database, calling multiple APIs, verifying results, and finally generating a response…
Who plans those steps?
Who decides whether another tool should be used?
Who determines that the answer needs to be checked before responding?
That's where AI Agents come in.
In Part 4, we'll go beyond the buzzword and explore how modern AI agents think, plan, reason and orchestrate complex workflows - along with the patterns that power today's most capable AI systems.
Top comments (0)