DEV Community: Uma Baleboyina

Understanding Retrieval-Augmented Generation (RAG): My Learning Journey

Uma Baleboyina — Fri, 10 Jul 2026 10:12:17 +0000

Artificial Intelligence has made it possible for Large Language Models (LLMs) to answer a wide range of questions. However, one major limitation of LLMs is that they only know the information they were trained on. They cannot automatically access the latest documents or organization-specific knowledge. This is where Retrieval-Augmented Generation (RAG) becomes valuable.

What is RAG?

Retrieval-Augmented Generation (RAG) is a technique that allows an LLM to retrieve relevant information from external knowledge sources before generating a response. Instead of relying only on its pre-trained knowledge, the model first searches for relevant information and then uses it to produce a more accurate and context-aware answer.

In simple terms:

RAG = Retrieval + Generation

Retrieval: Find the most relevant information.
Generation: Generate an answer using the retrieved information.

The RAG Pipeline

1. Document Loading

The first step in the RAG pipeline is loading the documents. These documents can come from different sources such as:

PDF files
Word documents
Text files
Websites
Images (after OCR)
Audio transcripts
Databases

In frameworks like LangChain, a document loader converts these sources into Document objects.

Each document contains two important attributes:

page_content

page_content contains the actual textual content of the document.

Example:

Python is a high-level programming language.

metadata

metadata stores additional information about the document.

For example:

{
    "source": "python_notes.pdf",
    "page": 3,
    "author": "John"
}

Metadata helps identify where the retrieved information came from, making the responses more reliable and traceable.

2. Chunking

Large documents are difficult for embedding models and LLMs to process directly. Therefore, the documents are divided into smaller pieces called chunks.

This process is known as chunking or text splitting.

Different chunking strategies include:

Fixed-size chunking
Recursive chunking
Sentence-based chunking
Paragraph-based chunking
Semantic chunking

Two important parameters during chunking are:

Chunk Size: The amount of text contained in each chunk.
Chunk Overlap: The shared text between consecutive chunks, which helps preserve context.

Choosing the right chunk size and overlap significantly improves retrieval quality.

3. Embedding

After chunking, each chunk is converted into an embedding.

An embedding is a numerical representation of text in a high-dimensional vector space. Instead of storing text directly, the embedding model converts each chunk into a vector that captures its semantic meaning.

For example,

Chunk:
"Python is widely used for Artificial Intelligence."

may become

[0.12, -0.45, 0.78, 0.91, -0.23, ...]

It is important to understand that:

One chunk produces one embedding vector.
One embedding vector contains hundreds or thousands of numerical values (dimensions).

Depending on the embedding model, the vector may contain:

384 dimensions
768 dimensions
1536 dimensions
3072 dimensions

These vectors allow the system to compare the meanings of different pieces of text rather than simply matching keywords.

Different embedding models have different token limits and performance characteristics, so selecting the appropriate model depends on the application's requirements.

4. Vector Database

The generated embeddings are stored in a vector database.

Unlike traditional databases, a vector database stores vector embeddings and performs semantic similarity searches.

Popular vector databases include:

ChromaDB
FAISS
Pinecone
Milvus
Weaviate

The vector database stores the embeddings of the document chunks so they can be retrieved efficiently when a user asks a question.

5. User Query

When a user asks a question, the same embedding model converts the user's query into an embedding vector.

For example,

User Question

How is Python used in Artificial Intelligence?

The question is converted into a vector representation similar to:

[0.91, -0.20, 0.28, 0.62, 0.80, ...]

This query vector is not permanently stored in the vector database. It is generated temporarily to search for similar document embeddings.

6. Similarity Search

The vector database compares the query embedding with every stored document embedding.

It calculates how similar the vectors are using similarity measures such as:

Cosine Similarity (most commonly used)
Dot Product
Euclidean Distance

For example:

Stored Chunk	Similarity Score
Chunk 1	0.45
Chunk 2	0.30
Chunk 3	0.98 ✅
Chunk 4	0.40

A higher similarity score indicates that the chunk's meaning is closer to the user's question.

In this example, Chunk 3 has the highest similarity score because it is semantically most similar to the user's query.

The vector database retrieves this chunk and sends it to the application.

7. Context Augmentation

The retrieved chunks are combined with the user's original question to form an augmented prompt.

Instead of sending only the user's question to the LLM, the application sends both the retrieved context and the question.

For example,

Context:
Python is widely used for Artificial Intelligence.

Question:
How is Python used in AI?

This additional context enables the LLM to generate more accurate and grounded responses.

8. Response Generation

Finally, the augmented prompt is sent to the LLM.

The LLM reads both:

The user's question
The retrieved context

It then generates the final response based on both pieces of information.

RAG with Different Types of Data

One interesting aspect of RAG is that it can work with both structured and unstructured data.

Unstructured Data

Examples include:

PDFs
Word documents
Text files
Images
Audio transcripts

The typical pipeline is:

Load → Chunk → Embed → Store in Vector Database

Structured Data

Structured data is stored in relational databases such as MySQL, PostgreSQL, or SQLite, where the data is organized into rows and columns.

Unlike unstructured data, structured data does not require chunking, embeddings, or a vector database. Instead, AI frameworks such as LangChain use a SQL Toolkit to retrieve information from the database.

The process works as follows:

The user asks a question in natural language.
LangChain sends the user's question to the LLM.
The LLM analyzes the question and decides whether a tool is required.
If database access is needed, the LLM requests the SQL Toolkit.
LangChain executes the SQL Toolkit.
The SQL Toolkit generates and executes the appropriate SQL query on the relational database.
The database returns the query results to the SQL Toolkit.
LangChain passes these results back to the LLM.
Finally, the LLM generates a natural-language response, which is returned to the user.

The overall flow is:

User → LangChain → LLM → SQL Toolkit → Database → SQL Toolkit → LangChain → LLM → User

Why Do We Need RAG?

Without RAG, an LLM answers questions using only the knowledge it learned during training.

With RAG:

It can access up-to-date information.
It can answer questions about private documents.
It reduces hallucinations.
It improves the accuracy and reliability of AI applications.
It provides answers grounded in external knowledge.

My Key Takeaways

While learning RAG, I realized that building a RAG application involves much more than connecting an LLM to a database. Every stage of the pipeline contributes to the quality of the final answer.

Document loaders prepare documents with both content and metadata.
Chunking divides large documents into meaningful sections.
Embeddings convert each chunk into a high-dimensional vector representing its semantic meaning.
Vector databases store these vectors and perform efficient similarity searches.
The user's query is also converted into an embedding vector.
Similarity search retrieves the chunks whose meanings are closest to the user's question.
The retrieved context is combined with the user's query before being sent to the LLM.

Overall, RAG extends the capabilities of Large Language Models by allowing them to retrieve and use external knowledge, making AI applications more accurate, reliable, and useful for real-world scenarios.

Middleware in DeepAgents

Uma Baleboyina — Wed, 10 Jun 2026 03:55:00 +0000

Middleware is a component that sits between different parts of an agent system and can observe, manage, modify, or track the flow of information during execution.

In Deep Agents, middleware is part of the framework architecture. It operates between components such as the user request, the model, and tool execution. Middleware can perform different responsibilities depending on its purpose.

There are different types of middleware. For example:

TodoListMiddleware – Creates and manages a task list (todos) for complex requests and tracks task completion.
PatchToolCallsMiddleware – Helps manage and validate tool-calling behavior.
Logging Middleware – Records execution details and events.
Summarization Middleware – Can summarize conversation history or intermediate results.

It is important to note that not every middleware performs verification. Different middleware have different responsibilities. Some middleware may validate requests, some may track progress, some may summarize information, and some may simply log activity.

In Deep Agents, certain middleware are added automatically by the framework. For example, when I created an agent using create_deep_agent(), I observed middleware such as TodoListMiddleware and PatchToolCallsMiddleware appearing in the execution trace even though I did not explicitly add them.

The TodoListMiddleware creates and maintains todos internally. These todos are typically based on the user's goal and the tools available to the agent. The middleware tracks which tasks are pending and which have been completed during execution.

For example, if the user asks:

"Get me the weather of the most popular city in India."

The internal todo list might conceptually look like:

Find the most popular city.
Get the weather for that city.

As the agent completes each step, the middleware can update the status of the corresponding task.

The actual tool outputs are stored in the agent's state and used by the model for reasoning, while the TodoListMiddleware primarily focuses on planning and tracking task progress.

Some middleware are automatically included by the framework, while other middleware may need to be explicitly added by the developer depending on the application's requirements.

From Simple LLMs to Intelligent AI Agents

Uma Baleboyina — Tue, 26 May 2026 16:12:40 +0000

Understanding Deep Agents and Agentic AI

Artificial Intelligence has evolved from simple text generation models to intelligent systems called AI Agents. Before understanding agents, we first need to understand how Large Language Models (LLMs) work.

What are LLMs?

LLMs (Large Language Models) are AI models trained on massive amounts of data. Their main job is to predict the next token based on previous tokens.

For example:

Input: "India is a"
Output: "country"

The model continuously predicts the next token to generate complete responses.

Modern AI models can generate different types of outputs, also called modalities.

Some common modalities are:

Text generation
Code generation
Image generation
Audio generation
Video generation

Examples:

Chatbots generate text
Code assistants generate code
AI art tools generate images
Video models generate videos
Problem with Using Different LLM APIs

Different AI companies provide different APIs.

For example:

OpenAI API
Gemini API
Claude API

Each API has:

different syntax
different configurations
different SDKs

So developers had to write separate code for every model.

How LangChain Helped

LangChain introduced a common framework for working with multiple LLMs.

Instead of rewriting the entire codebase for each model, developers can use a common interface.

Example idea:

Without LangChain:

openai.chat()
gemini.generate()
claude.messages()

With LangChain:

llm.invoke()

Only the configuration changes slightly.
This made AI application development much easier.
From Simple LLMs to Intelligent Agents

Simple LLMs can generate:

text
code
images
videos

But they cannot directly perform real-world tasks like:

booking train tickets
reserving hotels
sending emails
searching live data

Initially, developers combined LLMs and traditional programming logic to perform such actions.

Example flow:

User Request
↓
LLM understands request
↓
Python code calls APIs
↓
Action gets completed
Tool Calling in AI Agents

Later, AI systems evolved into tool-using agents.

Now the LLM itself can decide:

which tool to use
when to use it
what parameters to pass

Examples of tools:

search engines
calculators
booking APIs
databases
browsers This made AI systems appear more intelligent and autonomous.

ReAct Agents

One important concept in Agentic AI is the ReAct Agent.

ReAct stands for: Reason + Act

The agent:

Reasons about the problem
Chooses an action/tool
Observes the result
Continues reasoning

Flow:

Thought → Action → Observation → Thought

This allows the AI agent to solve complex tasks step-by-step.

Challenges in AI Agents

Even though agents are powerful, they still face many challenges.

1. Garbage In, Garbage Out (Prompt Quality)

LLMs highly depend on prompts.

If the input prompt is poor or unclear, the output quality also becomes poor.

This is called:

Garbage In → Garbage Out

Better prompts usually produce better results.

2. Guardrails

Guardrails are safety mechanisms added to AI systems.

Their purpose is to:

prevent harmful outputs
protect sensitive information
restrict unsafe actions
ensure ethical behavior Example: An AI agent should not reveal private user data or perform dangerous actions.

3. Grounding
Grounding means the AI should provide information based on:

real facts
reliable sources
actual context If the model does not know something, it should honestly say: “I do not have enough information.” instead of generating false information. This helps reduce hallucinations.

Conclusion

AI systems are evolving from simple text generators into intelligent autonomous agents. Frameworks like LangChain and ReAct-based architectures are helping developers build more capable AI applications.

However, challenges such as:

prompt quality
safety
hallucinations
grounding

What is MCP? My Beginner's Guide to Model Context Protocol

Uma Baleboyina — Fri, 08 May 2026 07:51:19 +0000

Introduction

When I started learning Generative AI, one of the first things I came across was something called MCP. At first it sounded complex, but once I understood it, everything clicked. In this blog, I want to share my understanding in the simplest way possible.

What is MCP?
MCP stands for Model Context Protocol. It is a lightweight protocol developed by Anthropic — the company that also created Claude AI. In simple terms, MCP acts as a bridge between an AI assistant and the outside world — connecting it to functions, tools, and external environments so the AI can actually do things, not just talk.

Where did the idea come from?
MCP was inspired by LSP — the Language Server Protocol. If you have used VS Code, you already benefited from LSP without knowing it. LSP is the reason VS Code can support multiple programming languages like Python, JavaScript, Java and more — all through one common standard.
MCP follows the same idea. Instead of building a separate integration for every AI model, MCP provides one common standard that works across multiple models. You build the server once, and any MCP-compatible AI model can use it.

How do you build an MCP server?
Building your first MCP server is simpler than it sounds. You start by importing MCP from the FastMCP library. Then you define your tools using the @mcp.tool decorator and write the logic — the actual code that performs the action. Finally, you connect it to Claude and your server is ready.

The two main components of MCP
MCP has two core components:

MCP Client — This is the intelligent side, like Claude. It understands the user's request and decides what needs to be done.
MCP Server — This is the action side. It actually performs the tasks that the client requests.

Think of the client as the brain and the server as the hands.

How does the workflow work?
Here is the step-by-step flow of how MCP works in action:

Tools vs Resources vs Prompts — what is the difference?
This is one of the most important concepts in MCP:

Tools — These are used to perform actions. When a user wants something done — like fetching weather data or saving a file — a tool does it.
Resources — These are used to feed data to the agent. They are read-only information sources the AI can access when it needs context or data.
Prompts — These are reusable question templates. They represent the general questions or instructions a user commonly asks the AI.

Conclusion
MCP is a powerful and elegant idea — one standard protocol that connects any AI model to any tool or data source. It removes the need to build custom integrations for every model and makes AI agents truly useful in the real world. I am just getting started with MCP, and I am excited to build more servers and share what I learn along the way.