Agentic GraphRAG: From RAG to Graph-Based Agentic Reasoning

Large Language Models (LLMs) can answer many questions, but answering questions over a large and complex corpus becomes difficult when the required information is spread across multiple documents or depends on relationships between different entities.
To address this challenge, we developed Agentic GraphRAG, a system that combines:

• Retrieval-Augmented Generation (RAG)
• GraphRAG
• TigerGraph
• Agent-based reasoning

The project compares three retrieval and reasoning approaches under a controlled setup:
RAG → GraphRAG → Agentic GraphRAG

The goal is not simply to retrieve more information, but to understand when vector retrieval is enough, when graph relationships are useful, and when an agentic system can adapt its strategy to the question.
Core idea: Retrieve the evidence, understand the relationships, reason over the evidence, and generate the answer.

The Problem

When a user asks a question over a large corpus, the answer may not exist as a single sentence in one document.
For example, a question may require the system to:

• Determine the athlete's result.
• Combine information from multiple records.
• Apply a condition or calculation.

A conventional vector search system can retrieve documents containing semantically related information, but it does not explicitly represent how entities are connected.

Normal RAG systems may fail when a question needs information from multiple documents or relationships between different entities. They may retrieve the wrong information or miss important evidence
.

This creates an important challenge:

How can we combine unstructured document retrieval with structured relationships and adaptive reasoning?

Our Approach

We propose an*Agentic GraphRAG system* that combines document search , a knowledge graph,and AI agents to find the right evidence and answer complex questions more accurately.

How Our System Works

• The first approach is traditional Retrieval-Augmented Generation.
• The question is converted into an embedding and used to search a vector database for semantically relevant document chunks.

Why RAG?

• Vector retrieval is useful because it searches based on semantic similarity rather than requiring an exact keyword match.
• However, semantic similarity alone does not explicitly tell the system how different entities are related.

That becomes important for multi-hop questions.

GraphRAG with TigerGraph

• The second approach is GraphRAG.
• Instead of relying only on document similarity, GraphRAG uses structured entities and relationships.
• For our implementation, the knowledge graph is represented using TigerGraph.

The graph can represent entities such as:

• Athletes
• Events
• Sports
• Olympic Games
• Results or medals

These entities can be connected through explicit relationships.

_
The GraphRAG pipeline becomes:

Why use a graph?

• A graph database represents explicit relationships between entities.
• Instead of asking the LLM to infer every relationship from raw text, the system can retrieve relevant relationships and paths directly from the graph.
• This is particularly useful when a question requires multiple connected facts.

Agentic GraphRAG

• The third and most advanced approach is Agentic GraphRAG.
• Here, we introduce an Agent Router that determines what type of retrieval or reasoning strategy is appropriate for the question.
• Different questions require different strategies.
• A simple lookup may need direct retrieval.

A more complex question may require:

• Document retrieval
• Graph traversal
• Multi-hop reasoning
• Aggregation
• Temporal reasoning
• Superlative reasoning
• Re-querying or refinement

The system therefore does not force every question through exactly the same retrieval process.
Instead, the router decides which agents should participate.

The agents

Retrieval Agent:
Responsible for document lookup and vector-based retrieval.

Graph Agent
Responsible for structured graph queries and relationship-based retrieval.

Reasoning Agent

• Responsible for multi-step reasoning and deciding whether further planning or refinement is required.
• This is useful for questions that cannot be answered by a single retrieval operation.

Why TigerGraph?

- TigerGraph plays an important role in both our GraphRAG and Agentic GraphRAG pipelines.

• A vector database is effective for finding semantically similar document chunks.

• A graph database is designed to represent explicit relationships between entities.

• These relationships can be traversed to obtain structured evidence.

• The key advantage is that the system does not need the LLM to infer every relationship from text.

• TigerGraph → Structured relationships

• Vector DB → Relevant documents

• LLM → Natural-language understanding and answer generation

• Agents → Retrieval and reasoning strategy

• This separation of responsibilities is one of the core design principles of our architecture.

• Agentic Reasoning in Action

• The main difference between GraphRAG and Agentic GraphRAG is adaptive orchestration.

• GraphRAG follows a structured graph retrieval approach.

• Agentic GraphRAG first considers the nature of the question and then selects an appropriate strategy.

The system needs to:

• Identify the relevant Olympic Games.
• Identify the relevant biathlon events.
• Retrieve the number of competitors for each event.
• Apply the condition competitors > 73.
• Count the events satisfying the condition.
• Generate the final natural-language answer.
• This is an aggregation-style question. An agentic workflow can make these reasoning steps explicit. **

The Agentic Trace

One important feature of the Agentic GraphRAG pipeline is the agentic trace.
The trace records information about the system's process, such as:

• Router decisions
• Agents selected
• Retrieval steps
• Evidence collected
• Reasoning steps
• Re-query or refinement actions Conceptually:

The trace provides additional visibility into how the system reached its answer.
This is useful for inspecting and debugging complex retrieval workflows.

Controlled Comparison

A major part of the project is the controlled comparison between the three approaches.
The same user question is sent through:

The goal is to change the retrieval/reasoning strategy while keeping the LLM consistent.

This makes it easier to study the trade-offs between:

• Simple semantic retrieval
• Structured graph retrieval
• Adaptive agentic reasoning

Benchmark
We use a benchmark containing 100 questions.
The questions are divided into five categories:

• Category Description
• Lookup Direct information retrieval
• Temporal Questions involving time or dates
• Aggregation Questions requiring filtering, counting, or combining values
• Superlative Questions involving highest, lowest, largest, smallest, etc.
• Multi-hop Questions requiring multiple connected relationships

We evaluate the three approaches using metrics such as:

• Accuracy
• Average tokens per question
• Average latency

For Agentic GraphRAG, we also track additional information such as:

• Retrieval steps
• Reasoning steps
• Retrieval methods
• Processing time
• Agentic trace Benchmark Results Important: Replace the values below with the actual results from your benchmark dashboard. Do not publish placeholder values.

How to interpret the results

The benchmark helps us understand the trade-offs between the three approaches.

• RAG provides a relatively direct retrieval pipeline.
• GraphRAG adds structured relationships and graph traversal.
• Agentic GraphRAG adds adaptive orchestration and can perform additional reasoning steps when the question requires them.
• The final results should be discussed using the actual benchmark measurements rather than assuming that one approach is always better.

System Architecture

The architecture contains three parallel pipelines that receive the same user question.

RAG Pipeline

GraphRAG Pipeline

Agentic GraphRAG Pipeline

All three pipelines then use the same LLM for natural-language understanding and answer synthesis.

Architecture Diagram

The diagram should be placed after this section so readers can visually understand the complete pipeline before reading the
implementation details.

Division of Responsibilities

A key design principle in the system is the separation of responsibilities.

Demo Flow

Our demonstration follows the same progression as the architecture.
Demo 1: RAG

We start with a question that can be answered through semantic document retrieval.

Demo 2: GraphRAG

Next, we demonstrate a question that benefits from structured relationships.

Demo 3: Agentic GraphRAG

Finally, we demonstrate a more complex question requiring aggregation or multi-step reasoning.

Key Advantages

Better Relationship Understanding

• Graph-based retrieval can explicitly represent relationships between entities.

Flexible Reasoning

• The agentic layer can select different strategies depending on the question.

Traceability

• The agentic trace provides visibility into retrieval and reasoning steps.

Separation of Responsibilities

• The graph handles structured evidence while the LLM handles natural-language understanding and generation.

Comparative Evaluation

• Implementing RAG, GraphRAG, and Agentic GraphRAG allows us to study their strengths and trade-offs under a common evaluation setup.

Challenges We Faced

Building the system involved several challenges.

Designing the Knowledge Graph

• The graph structure needs to represent the entities and relationships in the corpus correctly.
• Poor graph modeling can lead to incomplete or incorrect graph retrieval.

Choosing the Right Strategy

• Different questions require different retrieval and reasoning strategies.
• The system therefore needs to distinguish between simple lookups, temporal questions, aggregation, superlatives, and multi-hop questions.

Latency and Token Usage

• Additional reasoning steps can increase computational cost.
• An agentic system may perform more retrieval and reasoning operations than a simple RAG pipeline, so latency and token usage need to be measured.

Fair Benchmarking

• The three approaches need to be evaluated under consistent conditions.
• Otherwise, differences in configuration could make the comparison misleading.

Future Improvements

There are several directions in which this project can be extended:

• More specialized reasoning agents
• Better query planning for complex questions
• Improved graph traversal strategies
• Larger and more diverse benchmark datasets
• More detailed hallucination evaluation
• Better evidence-grounding evaluation
• Improved visualization of agentic traces
• Adaptive selection between vector and graph retrieval
• More robust evidence verification

A future version could make the router more adaptive by learning which retrieval strategy works best for different question categories.

Technology Stack

The project uses the following technologies and concepts:

• TigerGraph — graph database and structured relationship retrieval
• Vector Database — semantic document retrieval
• Large Language Model (LLM) — natural-language understanding and answer synthesis
• Retrieval-Augmented Generation (RAG) — document-based retrieval pipeline
• GraphRAG — graph-based retrieval pipeline
• Agentic Reasoning — adaptive orchestration and multi-step reasoning

RAG vs GraphRAG vs Agentic GraphRAG

What We Learned

This project helped us understand that retrieval quality is not only about finding text that looks similar to a question.
For complex questions, the system may also need to understand:

• Which entities are involved
• How those entities are connected
• Which evidence is relevant
• Which reasoning strategy should be used
• Whether additional retrieval is required

RAG provides a strong foundation for semantic retrieval.

GraphRAG adds structured relationships.
Agentic GraphRAG adds adaptive orchestration on top of retrieval and graph-based evidence.

Conclusion

Our project demonstrates the progression:

• RAG focuses on retrieving relevant documents.
• GraphRAG adds structured relationships through a knowledge graph.
• Agentic GraphRAG adds adaptive orchestration and reasoning strategies on top of retrieval and graph-based evidence.

The central idea of the project is:

Retrieve the evidence, understand the relationships, reason over the evidence, and generate the answer.
By combining vector retrieval, graph-based knowledge, LLMs, and agentic reasoning, we explore a more structured and traceable approach