DEV Community

Gaurav Jha
Gaurav Jha

Posted on

I Built an Agentic AI System From Scratch: RAG, GraphRAG, 12 Tools & Kubernetes

TL;DR: Over five weeks, I built a full-stack Agentic AI system that can plan multi-step tasks, retrieve information using hybrid RAG and GraphRAG, call 12 tools, review its own intermediate results, learn from user feedback, and deploy to Kubernetes.

The project ended up at roughly 6,000 lines of code, 360 tests, and green CI.

More importantly, I built an evaluation harness so I could measure whether the system actually improved instead of just adding more AI components.

GitHub: https://github.com/zda25m005-netizen/agentic-ai-os


Why I Built This

A lot of agent demos follow this pattern:

User → LLM → Tool → Answer
Enter fullscreen mode Exit fullscreen mode

It looks impressive in a demo.

But real-world questions are usually more complicated.

For example:

"Summarize the Q3 business risks from these 40 PDFs, find the products that shipped to Berlin, and explain how those products are connected to the risks."

A single LLM call isn't enough.

The system needs to:

  1. Find the right information.
  2. Understand relationships between entities.
  3. Break the problem into multiple steps.
  4. Call different tools.
  5. Check whether each step actually worked.
  6. Retry when something goes wrong.
  7. Produce an answer with citations.
  8. Tell me how much the whole operation cost.

So I decided to build the system I would actually want to debug.

Not just an agent demo.

A production-shaped Agentic AI OS.


What I Built

At a high level, the system looks like this:

                    User Goal
                       │
                       ▼
                  ┌──────────┐
                  │  Planner │
                  └────┬─────┘
                       │
                       ▼
                ┌─────────────┐
                │   Executor  │
                └──────┬──────┘
                       │
            ┌──────────┼──────────┐
            ▼          ▼          ▼
          Tools       RAG      GraphRAG
            │          │          │
            └──────────┼──────────┘
                       │
                       ▼
                  ┌─────────┐
                  │  Critic │
                  └────┬────┘
                       │
                ┌──────┴──────┐
                │             │
             APPROVE        RETRY
                │             │
                ▼             └──→ Executor
             Finalize
                │
                ▼
       Answer + Trace + Metrics
Enter fullscreen mode Exit fullscreen mode

The backend is built with FastAPI + LangGraph, with a Next.js frontend.

For retrieval, I use both:

  • Hybrid RAG
  • GraphRAG with Neo4j

The system currently exposes 12 tools, including:

  • Python execution
  • SQL
  • Web search
  • RAG search
  • Graph search
  • HTTP requests
  • File operations

Each tool is isolated and guarded according to its use case.


The Part Most Agent Demos Skip: Evaluation

This is probably the part I'm most proud of.

It is very easy to build an AI system that looks better.

It is much harder to prove that it actually is better.

So I built the evaluation system alongside the product.

The harness measures things like:

  • Recall@k — did the correct source appear in the top-k results?
  • LLM-judge correctness
  • Citation accuracy
  • Agent task-success rate
  • GraphRAG fact coverage
  • Retrieval performance across different strategies
  • Reranker improvements
  • Fine-tuning before/after comparisons

I also included a retrieval ablation:

Vector Search
      ↓
BM25
      ↓
Hybrid RAG
      ↓
Hybrid + Reranker
Enter fullscreen mode Exit fullscreen mode

This matters because I don't want to simply say:

"Hybrid RAG is better."

I want to be able to show:

"Here are the test cases, here is the baseline, here is the hybrid result, and here is what changed."

And when something doesn't improve the baseline, the evaluation reports that too.

I think honesty in evaluation is a feature.


1. Hybrid RAG: Why Use Two Retrieval Strategies?

Dense vector search is excellent at understanding semantic meaning.

For example:

"How do I reset my password?"
Enter fullscreen mode Exit fullscreen mode

can retrieve:

"Steps for recovering access to your account"
Enter fullscreen mode Exit fullscreen mode

even though the wording is different.

But semantic search can struggle with exact identifiers.

Imagine searching for:

SKU-4471
Enter fullscreen mode Exit fullscreen mode

That's where BM25 can be much better.

So I combined both approaches.

The system performs:

        Query
          │
     ┌────┴────┐
     ▼         ▼
 Vector       BM25
 Search       Search
     │         │
     └────┬────┘
          ▼
 Reciprocal Rank
     Fusion
          │
          ▼
      Reranker
          │
          ▼
      Top Results
Enter fullscreen mode Exit fullscreen mode

For fusion I use Reciprocal Rank Fusion (RRF).

Instead of combining the raw scores from vector search and BM25, RRF combines their rankings.

The basic idea is:

RRF score(d) = Σ 1 / (k + rank(d))
Enter fullscreen mode Exit fullscreen mode

where:

  • rank(d) is the position of document d in a result list.
  • k controls how quickly the contribution decreases.
  • I use k = 60.

For example, if a document appears at rank 1 in one result list and rank 4 in another:

RRF score = 1 / (60 + 1) + 1 / (60 + 4)
Enter fullscreen mode Exit fullscreen mode

The important part is that RRF uses rank rather than raw retrieval scores.

This makes it useful when combining vector search and BM25 because their raw scores don't necessarily live on the same scale.

The actual implementation is tiny:

def reciprocal_rank_fusion(result_lists, k=60, limit=5):
    scores, payloads = {}, {}

    for hits in result_lists:
        for rank, hit in enumerate(hits, start=1):
            scores[hit.id] = (
                scores.get(hit.id, 0.0)
                + 1.0 / (k + rank)
            )
            payloads.setdefault(hit.id, hit.payload)

    ranked = sorted(
        scores.items(),
        key=lambda kv: kv[1],
        reverse=True
    )

    return [
        SearchHit(
            id=i,
            score=s,
            payload=payloads[i]
        )
        for i, s in ranked[:limit]
    ]
Enter fullscreen mode Exit fullscreen mode

I intentionally kept the implementation small instead of hiding the ranking logic behind another abstraction.

That makes the behavior easy to understand, test, and debug.


2. GraphRAG: When Documents Aren't Enough

Traditional RAG is great at retrieving relevant passages.

But some questions are fundamentally about relationships.

For example:

"How is Product A connected to Customer B?"

or:

"Which products are associated with the risks mentioned in this document?"

Finding individual passages isn't necessarily enough.

So I added a knowledge graph.

The ingestion pipeline looks roughly like this:

Documents
    │
    ▼
Chunks
    │
    ▼
LLM Entity + Relationship Extraction
    │
    ▼
Neo4j Knowledge Graph
Enter fullscreen mode Exit fullscreen mode

Entities and relationships are extracted from the documents and stored in Neo4j.

The graph writes use MERGE, making ingestion idempotent.

At query time:

User Query
    │
    ▼
Entity Detection
    │
    ▼
k-hop Neighborhood
    │
    ▼
Relevant Graph Facts
Enter fullscreen mode Exit fullscreen mode

The graph results are then combined with the normal RAG passages.

The API supports:

/ask?mode=vector
/ask?mode=graph
/ask?mode=fused
Enter fullscreen mode Exit fullscreen mode

I also expose GraphRAG through a dedicated:

graph_search
Enter fullscreen mode Exit fullscreen mode

tool.

This gives the agent another way to answer questions where relationships matter more than isolated text passages.


3. What Actually Makes It an Agent?

This was an important design question for me.

Calling a tool doesn't automatically make something an agent.

The interesting part is the control loop.

After the executor completes a step, a Critic evaluates the result.

Conceptually:

Step
 │
 ▼
Execute
 │
 ▼
Critic
 │
 ├── APPROVE ──→ Next Step
 │
 └── RETRY ────→ Execute Again
Enter fullscreen mode Exit fullscreen mode

The Critic can return:

APPROVE
Enter fullscreen mode Exit fullscreen mode

or:

RETRY: The retrieved evidence does not support the claim.
Enter fullscreen mode Exit fullscreen mode

The retry mechanism is intentionally bounded.

async def review(step, result, chat_fn):
    raw = (
        await chat_fn(
            _critic_messages(step, result)
        )
    ).strip()

    if raw.lower().startswith("approve"):
        return APPROVE, ""

    reason = (
        raw.split(":", 1)[1].strip()
        if ":" in raw
        else raw
    )

    return RETRY, reason
Enter fullscreen mode Exit fullscreen mode

And at the graph level:

if verdict == RETRY and retries < MAX_RETRIES:
    return {
        "cursor": idx,
        "retries": retries + 1,
        ...
    }
Enter fullscreen mode Exit fullscreen mode

That last condition is important.

The loop must terminate.

Agent systems can become expensive very quickly when they are allowed to keep reasoning forever.

So I explicitly enforce retry limits.

The goal is simple:

No unbounded loops. No runaway bills.


4. The 12-Tool Execution Loop

The executor runs a function-calling loop over 12 tools.

Some examples include:

Python
SQL
Web Search
RAG
Graph Search
HTTP
Files
...
Enter fullscreen mode Exit fullscreen mode

I also added safeguards around the tools.

For example:

  • Python execution is sandboxed.
  • SQL is read-only.
  • HTTP requests have SSRF protections.
  • File access is path-guarded.

Another design decision I made:

A tool failure should not automatically kill the entire agent run.

Instead of:

Tool Error → Crash
Enter fullscreen mode Exit fullscreen mode

the system returns the error to the model:

Tool Error
    │
    ▼
Agent observes error
    │
    ▼
Agent decides what to do next
Enter fullscreen mode Exit fullscreen mode

That gives the agent an opportunity to recover when possible.

It also makes debugging much easier because the error becomes part of the execution trace.


5. Learning From User Feedback

The system also has a feedback loop.

Users can give:

👍
👎
Enter fullscreen mode Exit fullscreen mode

and optionally provide a better answer.

That feedback is used in two ways.

Learned Reranking

The feedback can train a lightweight reranker.

When there isn't enough feedback yet, the system falls back to the LLM reranker.

That gives me:

Cold Start
    ↓
LLM Reranker
    ↓
User Feedback
    ↓
Learned Reranker
Enter fullscreen mode Exit fullscreen mode

rather than deploying a learned model before there is enough data.

DPO Preference Pairs

I also export preference data as JSONL.

Conceptually:

Chosen Answer
      vs
Rejected Answer
Enter fullscreen mode Exit fullscreen mode

I intentionally describe this as DPO, not RLHF.

There is:

  • No reward model
  • No online reinforcement learning

The goal is to keep the terminology technically accurate.


6. Observability: Because "It Feels Slow" Isn't a Metric

One of my biggest goals was to make the system observable.

The API exposes Prometheus metrics for:

  • Request count
  • Latency
  • Token usage
  • Estimated USD cost
  • Agent-node execution
  • Tool execution
  • Errors

The metrics are visualized in Grafana.

So instead of asking:

"Why is this agent expensive?"

I can look at the cost of individual parts of a run.

For example:

Request
  │
  ├── Planner       $0.002
  ├── RAG           $0.000
  ├── Tool #4       $0.001
  ├── Critic        $0.003
  └── Finalizer     $0.002
                  --------
                    $0.008
Enter fullscreen mode Exit fullscreen mode

I also added:

  • Optional Langfuse traces
  • /readyz dependency checks
  • Structured JSON logs
  • Request IDs

The result is that an agent run isn't just an answer.

It's an answer + execution trace + metrics.


7. Kubernetes Deployment

The project isn't just:

docker compose up
Enter fullscreen mode Exit fullscreen mode

I also created a Helm deployment.

The chart includes:

  • Ingress
  • API autoscaling
  • ConfigMaps
  • Secrets
  • Non-root containers
  • NetworkPolicies

The CI pipeline creates a kind Kubernetes cluster and verifies the deployment.

It checks the health endpoint after deployment.

So the deployment isn't merely:

"Here's a Helm chart. It should work."

The CI actually exercises it.

That distinction matters.

A deployment configuration that looks correct is not the same thing as a deployment that has actually been tested.


8. The Evaluation Harness

The evaluation system is what ties the whole project together.

I didn't want to evaluate the system only by asking:

"Does the answer look good?"

Instead, I separated evaluation into multiple dimensions.

Retrieval

I measure:

  • Recall@k
  • Vector-only retrieval
  • BM25 retrieval
  • Hybrid retrieval
  • Hybrid + reranker

This makes it possible to see which retrieval strategy actually helps.

Answer Quality

I measure:

  • LLM-judge correctness
  • Citation accuracy
  • Fact coverage

Agent Performance

For multi-step tasks, I measure:

  • Task-success rate
  • Successful tool execution
  • Retry behavior
  • Final answer quality

GraphRAG

For graph-based questions, I measure:

  • Relevant entity retrieval
  • Relationship coverage
  • Fact coverage

Most importantly, the evaluation reports the sample size.

I don't want to publish:

"Our approach improved accuracy by 17%."

without also saying:

"across N evaluation examples."

Numbers without context can be misleading.


What the Five Weeks Taught Me

1. Measure From Day One

The evaluation harness ended up being one of the most valuable parts of the project.

Without it, I would have been guessing whether changes actually improved the system.

Every new component creates another question:

"Did this actually help?"

An evaluation gives you an answer.


2. More AI Isn't Automatically Better

Adding:

RAG
+
GraphRAG
+
Reranker
+
Critic
+
12 Tools
Enter fullscreen mode Exit fullscreen mode

sounds impressive.

But every additional component introduces:

  • Latency
  • Cost
  • Failure modes
  • Complexity

The evaluation needs to justify the complexity.

Sometimes the simplest solution wins.


3. Bound Your Agent Loops

This is one of the easiest things to overlook.

An agent that can endlessly retry is not robust.

It's an expensive bug.

Set explicit limits.

Track the cost.

Make the failure mode predictable.


4. Don't Hide Bad Results

One of the most useful principles I learned from this project:

A benchmark that tells you your idea didn't work is still a successful benchmark.

If a reranker doesn't beat the baseline, report it.

If GraphRAG doesn't help a particular query type, report it.

If a model performs worse after fine-tuning, report it.

That information is much more valuable than a perfect-looking README.


5. Keep Infrastructure Boring

I also learned a less exciting but very practical lesson.

Small infrastructure mistakes can waste hours.

For example:

docker compose up -d --build
Enter fullscreen mode Exit fullscreen mode

became part of my development routine.

At one point, I spent an afternoon debugging behavior that turned out to be a stale Docker image running old code.

The lesson:

When the behavior makes no sense, check what code you're actually running.


The Result

After five weeks, the system looks roughly like this:

Component Implementation
Backend FastAPI
Agent orchestration LangGraph
Frontend Next.js
Vector retrieval Hybrid RAG
Keyword retrieval BM25
Fusion Reciprocal Rank Fusion
Knowledge graph Neo4j
Tools 12
Evaluation Automated eval harness
Metrics Prometheus
Dashboards Grafana
Tracing Langfuse
Deployment Docker + Helm
Kubernetes testing kind
Tests ~360
Code ~6,000 LOC

But the important part isn't the number of technologies.

It's that each component exists for a reason and has a test or evaluation behind it.


Architecture at a Glance

Putting everything together:

                         ┌──────────────┐
                         │     User     │
                         └──────┬───────┘
                                │
                                ▼
                         ┌──────────────┐
                         │    Planner   │
                         └──────┬───────┘
                                │
                                ▼
                    ┌──────────────────────┐
                    │       Executor       │
                    │   Function Calling   │
                    └──────────┬───────────┘
                               │
          ┌────────────────────┼────────────────────┐
          │                    │                    │
          ▼                    ▼                    ▼
     ┌─────────┐         ┌──────────┐        ┌───────────┐
     │  Tools  │         │ Hybrid   │        │ GraphRAG  │
     │   ×12   │         │   RAG    │        │  Neo4j    │
     └─────────┘         └────┬─────┘        └─────┬─────┘
                               │                    │
                               └─────────┬──────────┘
                                         │
                                         ▼
                                  ┌──────────────┐
                                  │    Critic    │
                                  └──────┬───────┘
                                         │
                                  ┌──────┴──────┐
                                  │             │
                               APPROVE        RETRY
                                  │             │
                                  ▼             │
                              Finalize ◄────────┘
                                  │
                                  ▼
                      ┌─────────────────────────┐
                      │ Answer + Citations      │
                      │ Execution Trace         │
                      │ Tokens + Cost           │
                      │ Evaluation Metrics      │
                      └─────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Around the system sits the infrastructure layer:

                 ┌─────────────────────────┐
                 │      Observability       │
                 │ Prometheus + Grafana     │
                 │ Optional Langfuse        │
                 └─────────────────────────┘

                 ┌─────────────────────────┐
                 │       Deployment        │
                 │ Docker + Helm + K8s     │
                 │ kind CI verification    │
                 └─────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

What I'd Build Next

There are still several things I'd like to improve.

Better Evaluation Datasets

The evaluation harness is only as good as the questions being evaluated.

A larger, more diverse benchmark would make the results more meaningful.

Better Agent Cost Optimization

The Critic and multiple retrieval stages add latency and token usage.

A future version could dynamically decide when a review step is actually necessary.

More Deterministic Tool Policies

Some tools can be made more deterministic by moving safety and validation logic outside the LLM.

The model should decide what to do.

The infrastructure should decide what is allowed.

Better Long-Term Memory

The current feedback system is intentionally lightweight.

A stronger memory layer could learn user preferences and task patterns while still keeping the system auditable.


Try It Yourself

The project is available here:

GitHub: https://github.com/zda25m005-netizen/agentic-ai-os

Clone the repository and run:

docker compose up --build
Enter fullscreen mode Exit fullscreen mode

Then open:

Frontend → http://localhost:3000
API      → http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

The repository contains:

  • Architecture documentation
  • Setup instructions
  • Evaluation harness
  • Retrieval experiments
  • GraphRAG implementation
  • Agent orchestration
  • Tool implementations
  • Observability configuration
  • Docker configuration
  • Helm deployment
  • Kubernetes CI verification

Final Thoughts

I started this project wanting to understand what it takes to move beyond a simple:

LLM + Prompt
Enter fullscreen mode Exit fullscreen mode

and build something closer to a real AI system.

The biggest lesson wasn't GraphRAG.

It wasn't LangGraph.

It wasn't even the multi-agent architecture.

It was measurement.

Building an agent is relatively easy.

Building an agent where you can answer:

"Did this change actually make the system better?"

is much harder.

That's the part I'm continuing to work on.

If you're building agentic systems too, I'd especially love feedback on the evaluation methodology, the retrieval experiments, and the trade-offs in the architecture.

Thanks for reading.

Top comments (0)