Beyond RAG: Building an AI Coding Agent with Planning, Tool Execution, and ReAct Reasoning

In my previous article, I explored how I wrapped a RAG agent inside an MCP server to make enterprise knowledge accessible through standardized tools.

However, while RAG improves retrieval, software engineering tasks require something more.

A developer assistant should not only retrieve information. It should investigate.

For example:

"Where is authentication implemented?"

A useful coding assistant should be able to:

• search the repository
• identify relevant files
• inspect source code
• understand classes and functions
• explain the implementation

This led me to build an AI coding agent that can reason, select tools, and analyze a codebase step-by-step. This one doesn't use RAG yet — it works directly against the repository — but that's a deliberate next step, more on that at the end.

From Chatbot to Agent

A traditional chatbot follows a simple pattern:

User Question
       |
       v
      LLM
       |
       v
    Response

This works well for general questions, but software repositories contain thousands of files and relationships.

A coding assistant needs additional capabilities:

• repository awareness
• search capability
• code understanding
• multi-step reasoning

An agent introduces a decision-making layer:

User Question
      |
      v
    Planner
      |
      v
 Choose Tool
      |
      v
 Execute Tool
      |
      v
 Observe Result
      |
      v
 Generate Answer

Architecture Overview

The agent consists of several components.

Planner

The planner decides the next action. It tries a rule-based plan first (keyword matching on the question), and falls back to an LLM (via Ollama's tinyllama) for JSON-structured tool selection when no rule matches.

Example:

User question:

Where is authentication implemented?

Planner response:

{
 "tool": "search_code",
 "input": "authentication"
}

The planner does not execute the action. It only decides what should happen next.

Tool Registry

The agent exposes capabilities through tools. Currently:

• search_code
• read_file
• analyze_file

Each tool has a specific responsibility.

Search Tool — finds files containing a keyword.

Input: authentication
Output: auth.py, app.py

Read File Tool — retrieves source code.

Input: auth.py
Output:

class AuthenticationService:
    def login(self, username, password):
        if self.authenticate(username, password):
            return "Login successful"

Analyze Tool — understands code structure using Python's ast module.

Output:

{
 "classes": ["AuthenticationService"],
 "functions": ["login", "authenticate"]
}

The Agent Execution Loop

The core of the system is the agent loop, following a ReAct-style pattern:

Reason
  ↓
Action
  ↓
Observation
  ↓
Reason Again

Step 1 — The agent determines it needs to locate authentication code.

Action: search_code("authentication")
Observation: auth.py, app.py

Step 2 — The agent identifies that auth.py is likely the implementation.

Action: read_file("auth.py")
Observation: AuthenticationService class found

Step 3 — The agent needs more understanding.

Action: analyze_file()
Observation:

Class: AuthenticationService
Functions: login(), authenticate()

The agent now has enough information to answer.

Challenges During Implementation

Building the agent introduced several interesting engineering challenges.

Challenge 1: Reliable tool selection. Initially, the LLM sometimes selected incorrect tools or returned invalid responses. To improve reliability, I restricted tool choices, added JSON validation, and enforced structured outputs.

Challenge 2: Avoiding repeated actions. An early version of the agent could repeat search_code indefinitely, because every decision was independent. The fix was maintaining previous observations as context, so the planner can see "auth.py contains authentication" and move to reading the file instead of searching again.

Challenge 3: Separating implementation from references. A search result may return both auth.py (which defines AuthenticationService) and app.py (which just imports it). The agent needs code analysis, not simple keyword matching, to tell the two apart.

Final Result — and a Current Limitation

For this specific demo question, the agent produces:

{
 "answer": "Authentication is implemented in auth.py",
 "classes": ["AuthenticationService"],
 "functions": ["login", "authenticate"]
}

I want to be upfront about where this stands today: the search, read, and analyze steps are genuinely general-purpose — they work against any Python codebase. The final answer-generation step, however, is currently scoped to authentication-style questions specifically; it doesn't yet generalize its explanation to arbitrary questions the way the reasoning steps before it do. Making that synthesis step question-agnostic is next on my list.

Future Improvements

Some areas I want to explore next:

• Generalize the answer-generation step beyond authentication-style questions
• Integrate Git repositories directly
• Generate dependency graphs
• Understand application architecture
• Connect Jira and documentation systems
• Automated pull request reviews
• Code migration assistants

Conclusion

Building an AI coding agent showed me that the biggest difference between a chatbot and an agent is not the language model itself.

The difference is the ability to:

• plan
• use tools
• gather evidence
• reason over observations
• decide when enough information is available

RAG helps an AI find information. Agents help an AI perform tasks. The next generation of developer assistants will combine both — and connecting this agent to real enterprise knowledge retrieval is exactly where I'm headed next.

Github: https://github.com/srirdeevi/ai-coding-agent