What is agentic RAG?

What the interviewer is testing

Whether you can distinguish agentic retrieval from a fixed pipeline and explain its benefits, costs, and production controls.

Answer framework

• Agentic RAG replaces one fixed retrieve-then-generate pass with a model-controlled retrieval loop.
• The model decides when and what to search and can issue several queries during that loop.
• It reads the returned results and may search again when it still needs evidence.
• This supports multi-hop questions, but it can increase cost and make evaluation harder.
• Cap the number of steps and log each retrieval so the decision path can be inspected.

Sample answer

I would define agentic RAG as a retrieval-augmented generation system where the model controls part of the retrieval loop. In a basic RAG pipeline, I rewrite a query once, retrieve documents, and pass them to the model. In an agentic version, the model can decide whether it needs to search, choose what to look for, issue multiple queries, read the results, and decide whether to search again before it writes the answer.

That extra control is useful for questions that cannot be answered from one retrieval pass. For a multi-hop question, I might retrieve evidence about one entity, notice that another fact is missing, and issue a second query that connects the two. I can also ask for a narrower search if the first results look weak. The system therefore treats retrieval as an action the agent can iterate on, while the model uses the retrieved passages as evidence for the final response.

The tradeoff is that this can make the system more expensive, slower, and harder to evaluate than a fixed pipeline. Additional searches add retrieval overhead, and model calls to plan searches or process results can increase token cost. A model may also loop without converging. In production, I would cap the number of steps, log every query and retrieved document, and preserve a trace so I can diagnose both the answer and the path. I would test it on multi-hop tasks against a simpler baseline, measuring success as well as steps, cost, and latency rather than assuming the agentic design wins.

Likely follow-ups

What distinguishes agentic RAG from a multi-query RAG pipeline? A multi-query pipeline runs a preset query expansion, while agentic RAG lets the model choose later searches after reading the results of earlier ones.

When is agentic RAG worth the additional cost? I would favor it when a single retrieval pass cannot connect the evidence needed for a multi-hop task, and when testing shows better task success despite the extra steps.

How do I prevent the retrieval loop from running indefinitely? I would set a step cap, define stopping conditions, and monitor repeated queries and step counts in production logs.

Common mistakes

• Calling any pipeline with several retrievals agentic without a model-controlled retrieval loop.
• Assuming more searches will always produce a better answer.
• Running the agent without a step cap or stopping condition.
• Evaluating only the final answer without retaining retrieval logs and decision traces.

This article was prepared with AI assistance and reviewed before publishing. The original lives on the free question bank:

https://johnnnmai.github.io/ai-engineer-interview-questions/questions/what-is-agentic-rag/?utm_source=devto&utm_medium=community&utm_campaign=what-is-agentic-rag