Classical RAG vs Agentic RAG: a practical decision guide

Ahmet Özel — Mon, 08 Jun 2026 14:07:52 +0000

"Should I use RAG or an agent?" comes up in almost every LLM project I work on. The honest answer is that they are not competing choices. Classical RAG and agentic RAG sit on a spectrum, and picking the wrong end of it either wastes money or gives you weak answers. This post is a practical way to decide, based on a guide and demo I put together.

Repo with runnable code: https://github.com/ahmet-ozel/rag-architecture-guide

Classical RAG in one paragraph

Classical RAG is a fixed pipeline: embed the query, retrieve the top-k chunks from a vector store, stuff them into the prompt, and generate an answer. One retrieval, one generation. It is cheap, fast, and predictable. For a knowledge base where the answer lives in one or two documents, this is usually all you need, and adding anything more just increases latency and cost.

Agentic RAG in one paragraph

Agentic RAG hands control to the model. Instead of a fixed pipeline, the LLM decides what to do: reformulate the query, retrieve, check whether the result is good enough, retrieve again from a different source, call a tool, and only then answer. It can loop. This is far more powerful for hard questions, but it is slower, costs more tokens, and is harder to make deterministic.

A decision tree that works in practice

Start simple and only add complexity when the data forces you to:

Is the answer usually contained in a single chunk or document? Use classical RAG.
Does answering require combining information from several documents or steps of reasoning? Lean agentic.
Do you need to query multiple sources (a vector DB, a SQL table, an external API) to answer? Agentic, because the model needs to choose tools.
Are latency and cost tight constraints (high traffic, user-facing)? Bias toward classical, and only escalate to an agent for the queries that actually need it.
Can you tolerate non-deterministic behavior? If not, classical with strong retrieval beats an agent that occasionally loops in unexpected ways.

A pattern I like: run classical RAG first, and if a confidence or self-check step says the retrieved context is weak, escalate that single query to the agentic path. Most queries stay cheap; only the hard ones pay the agent tax.

The part everyone skips: evaluation

Neither approach means anything without measurement. Before you argue about architecture, build an eval set of real questions with known good answers. Then track:

Retrieval quality: are the right chunks being retrieved at all? (recall@k, hit rate)
Answer quality: faithfulness (is the answer grounded in the retrieved context?) and relevance.
Cost and latency per query, so you can see what agentic behavior actually costs you.

Most "RAG is bad" complaints I see are actually retrieval problems: bad chunking, wrong embedding model, or no reranking. Fixing retrieval often beats switching to an agent.

What the demo covers

The repo walks through both architectures end to end with ChromaDB for vector search and works across OpenAI, Gemini, Claude, Ollama, and vLLM, so you can run it fully local or against a hosted model. It includes the chunking and retrieval steps, the agentic tool-selection loop, and the evaluation metrics so you can compare the two on your own data.

Takeaway

Default to classical RAG. Add agentic behavior when your questions genuinely need multi-step reasoning or multiple sources, and measure the cost when you do. Architecture is a dial, not a switch.

Repo: https://github.com/ahmet-ozel/rag-architecture-guide

How are you deciding between fixed pipelines and agentic retrieval in production? I am especially curious where people draw the line on cost.

Building an agentic Jira automation platform with MCP and Temporal

Ahmet Özel — Mon, 08 Jun 2026 12:28:21 +0000

Most "AI automation" demos fall apart the moment a workflow needs to run longer than a single request. An agent makes a few tool calls, the process crashes or times out, and you lose all state. I wanted something that could drive real, multi-step work inside Atlassian (Jira and Confluence) and survive restarts, retries, and failures. So I built an open-source platform around two ideas: MCP for tool access and Temporal for durable execution.

Repo: https://github.com/ahmet-ozel/atlassian-ai-workflow-platform

The problem with one-shot agents

A typical agent loop looks like: read a ticket, decide on an action, call a tool, repeat. This is fine for short tasks. It breaks down when a workflow spans minutes or hours, depends on external systems that fail intermittently, or needs to be resumed after a deploy. If your orchestration lives in a single Python process, any crash means you start over. For business workflows that touch real Jira issues, that is not acceptable.

Why MCP for tools

The Model Context Protocol (MCP) standardizes how an agent discovers and calls tools. Instead of hard-coding Jira API calls into the agent, I expose Jira and Confluence as MCP tools. The agent sees a clean, typed tool surface (create issue, transition status, search, comment, fetch a Confluence page) and the protocol handles the wiring.

The practical benefit is decoupling. I can add or change tools without touching the agent logic, and the same tools work with any MCP-compatible client. It also keeps the agent prompt focused on intent rather than API mechanics.

Why Temporal for orchestration

Temporal gives you durable workflows. The workflow code looks like ordinary Python, but every step is checkpointed. If a worker dies, the workflow resumes from the last completed step on another worker. Retries, timeouts, and backoff are declarative.

This maps perfectly onto agent workflows. Each LLM call and each tool call becomes a Temporal activity. If an LLM provider rate-limits you or a Jira call fails, Temporal retries that single activity instead of replaying the whole reasoning chain. Long-running approvals (wait for a human to review before transitioning a ticket) become a normal part of the workflow instead of a hack.

The tradeoff is added infrastructure. Temporal is one more service to run, and you have to think in terms of deterministic workflow code versus side-effecting activities. For short, stateless tasks it is overkill. For anything that has to be reliable, it pays for itself quickly.

Architecture

The stack ties together a few pieces:

An MCP integration layer that exposes Atlassian tools to the agent
Temporal workers that run the durable workflows and activities
A webhook gateway that turns Jira events into workflow triggers
An admin dashboard plus a Streamlit UI for running and inspecting workflows
Multi-provider LLM support (OpenAI, Anthropic, Gemini, and self-hosted vLLM)

Everything runs in a single Docker Compose stack, so you can bring the whole system up locally and see the moving parts together. Provider choice is config-driven, which makes it easy to swap a hosted model for a local one during development.

What I learned

Separating "what to do" from "how to survive doing it" was the key insight. The agent reasons about intent and picks tools. Temporal owns reliability. MCP owns the tool boundary. Keeping those three responsibilities apart made each one much simpler to reason about and test.

The other lesson: deterministic workflow code is a discipline. Anything non-deterministic (network calls, timestamps, random values) has to live in an activity, not the workflow body. Once that clicked, debugging got a lot easier because the workflow history is a precise, replayable log of what happened.

It currently targets Atlassian, but the tool layer is designed to extend to other platforms.

Feedback welcome

I would like to hear how others handle long-running agent workflows. Are you using Temporal, a queue plus your own state machine, or a custom orchestration loop? And for MCP users: how are you structuring tools when one agent needs access to several systems at once?

Repo and setup instructions: https://github.com/ahmet-ozel/atlassian-ai-workflow-platform

DEV Community: Ahmet Özel