lizer yang for SmartGate

Posted on Sep 30 Originally published at smartgate.network

Agentic Search vs RAG: A Tool Call or an Index You Own

#langchain #agentic #rag #mcp

Short answer: An agent's search tool and a RAG index are both retrieval, and they differ in
four places that decide everything downstream: what they read (the live web against a corpus you
ingested), who owns the ranking (an engine you do not operate against a pipeline you configured),
how fresh the answer can be, and what a call costs. A search tool trades control and predictability
for reach and freshness; an index trades reach for a ranking you can measure. Systems that work at
scale run both and route per question rather than choosing once.

Key takeaways

The generation step is identical. Both paths end in retrieved text placed in a prompt. Every difference is upstream of the model, which is why the choice is an engineering decision.
Ranking ownership is the sharpest split. A search tool borrows someone else's relevance model; an index is only as good as the chunking and embedding you chose.
The cost curves are opposites. An index is a fixed cost at ingest and almost free per query; a search call is free to build and billed per invocation.
Freshness is what the variable cost buys. "What changed this week" is a search question; "what our policy says" is an index question, because only the index can cite an exact revision.
Route by question, not by preference. Freshness, privacy and repeat rate are three signals you can evaluate before generation, which makes routing implementable rather than aspirational.
Do this next: take twenty real questions, mark each as freshness-dependent or corpus-bound, and count the split. That ratio is the only input the routing decision needs.

The generation step is the same; the retrieval path is not

Both arrangements end the same way: retrieved text is placed in a prompt and the model generates
from it, which is what the original RAG paper formalised.
Everything that differs happens before the prompt is assembled. A search tool hands the model
documents — a title, a URL, a snippet and often a date — ranked by a search engine whose ranking
function you cannot read and whose index you cannot inspect. An index hands the model passages:
fragments of your own documents with similarity scores produced by an embedding model you chose
over a chunking scheme you chose. The first is a window on a corpus nobody owns. The second is a
window on a corpus you built and are therefore responsible for.

That both are called retrieval is not an accident of vocabulary; it is why the two are usually
separate tools in a working system rather than two settings on one component. The research
literature makes the same split when it separates a fixed retriever from a model that decides when
to search, and when it measures how many searches an answer took (learned search
use). The failure modes, the cost curve and the evaluation
method all differ, and the difference starts with the request itself.

A search request is small enough to be serialised, logged, diffed and replayed. The function below
writes only the values that differ from a default, and returns early when one field makes the rest
irrelevant:

# lib/dashboard/logs-url.ts — source lines 31–45 (serializeLogsSearchParams)
function serializeLogsSearchParams(state: LogsViewState): URLSearchParams {
  const qs = new URLSearchParams();
  if (state.primaryMode === "task") {
    qs.set("mode", "task");
    return qs;
  }
  if (state.present === "table") {
    qs.set("present", "table");
    return qs;
  }
  if (state.density === "terminal") {
    qs.set("density", "terminal");
  }
  return qs;
}

Read it as a property of the request rather than as a UI helper. A query that can be expressed as a
handful of parameters is a query you can put in a URL, paste into a bug report, replay against a
different day, and cache by its own serialisation. An embedding is none of those things: it is an
opaque vector, you cannot hand-edit it, and two queries that mean the same thing in different words
produce unrelated vectors. That asymmetry is the practical reason retrieval tuning looks so
different on the two paths — on the search side you tune the query text and the parameters beside
it, and on the index side you tune the pipeline that produced the vectors, because the query is
generated rather than written.

Retrieval-augmented generation over a corpus you own

Owning the corpus is the index path's whole proposition, so it is worth being precise about what
ownership buys and what it costs. It buys recall you can measure on labelled questions, a ranking
you can change by editing chunk size, overlap, metadata filters or a reranker, and determinism: the
same query returns the same passages until somebody reindexes. It also buys a cost structure — the
expensive embedding happens at ingest, so queries are cheap — and privacy, because the corpus never
has to leave your boundary to be searched.

What it costs is freshness and reach. An index can only answer from what was ingested, so its
knowledge has a cadence rather than a date, and a question about last week is a question it cannot
answer no matter how good the retrieval is. Long documents have to be split to be retrievable,
which means the unit of evidence is a fragment whose surrounding context may live in a neighbour
that was not retrieved — the failure mode the retrieval-side surveys spend most of their length on
(RAG for large language models).

Every knob in that pipeline is a number you chose and can defend, and the shape of a defensible
choice is the same everywhere: a named constant with a default, a band it must sit inside, and a
refusal to start with a value outside the band. The snippet below is not a retrieval parameter
itself — it is a billing blend ratio — but it is the shape of every tuning knob on the side of the
stack you own:

# config/billing-savings-share.ts — source lines 7–14 (parseSearchKappa)
function parseSearchKappa(): number {
  const raw = process.env.BILLING_SEARCH_KAPPA;
  const n = raw == null || raw === "" ? 0.3 : Number(raw);
  if (!Number.isFinite(n) || n < 0 || n > 1) {
    throw new Error(`Invalid BILLING_SEARCH_KAPPA: ${raw}`);
  }
  return n;
}

The contrast is with the knobs you do not own. A search tool has a recency filter, a result count
and a provider's defaults, and behind them sit a ranking function, a crawl schedule and a spam
policy you can neither read nor change. When a search tool starts ranking a competitor's page above
the page you wanted, there is no constant to edit. When the blend ratio above is invalid, the
process refuses to boot, which is the honest behaviour for a parameter that silently changes every
number a customer sees. That is the difference between the two paths in one pair of functions: on
one side a value in your configuration with a validation boundary, on the other a black box whose
behaviour changes on somebody else's release schedule.
Which parameters exist at all is a design decision rather than an accident, and
the index's configuration surface is where a retrieval stack
lists them.

LangChain's retriever and search tool: two interfaces, one graph

Frameworks make the distinction mechanical, and that is their useful contribution here. A retriever
is an object your code calls: typed input, documents out, invoked exactly once by the chain that
needs evidence, testable in isolation, and able to fail loudly without confusing anything. A search
tool is an action the model chooses: the model writes the arguments as text, the runtime validates
them against a schema the model was shown, and the result has to be rendered back into the context
as text the model can read. In LangChain, as in the other agent frameworks, both can live in the
same graph, and wrapping a retriever as a tool is the standard bridge between them.

Three consequences follow, and none of them is cosmetic.

The retriever can be evaluated offline; the tool cannot. Retrieval quality is a property of the index and the query string, so it is measurable on a labelled set. Tool quality includes the model's ability to write a good query, which only shows up in end-to-end traces.
The retriever's call count is fixed; the tool's is not. One request, one retrieval, by construction. A tool may be called zero times on a question that needs no evidence and six times on one that does, which is the entire cost story of an agentic system in one sentence.
The retriever's parameters are invisible to the model. Top-k, filters and thresholds live in your code. Every parameter you add to a tool's schema is prompt surface: tokens in every request, and one more argument the model can get wrong.

The learned variants sit between the two: a model trained to decide when to retrieve and to
critique what came back (retrieve, generate, critique) behaves
like a retriever with judgement attached, while a reasoning-acting loop keeps the decision in the
prompt (reasoning and acting) — that loop is
how an agentic loop works in the cluster's own words. Which of the three you build
depends on how often the right answer is "do not retrieve", and that frequency is measurable in
your own traffic before you commit to an architecture.

What an agentic search call costs, and how to count it

The two paths have opposite cost curves, and the difference is large enough that it should be
visible in the accounting before it is visible in a bill. An index is expensive once — embedding the
corpus, and re-embedding it when the model or the chunking changes — and then nearly free per
query, because a similarity lookup is the cheapest operation in the stack. A search call is free to
build and billed every time it runs, sometimes twice, because a good search tool also fetches the
pages it decided were worth reading. Freshness is what that variable cost buys, and it is the only
thing a fixed index cannot buy at any price.

Counting it honestly means one record per call, with a correlation identifier so that calls can be
folded back into the answer they belong to. The builder below writes exactly that kind of entry:

# dashboard-calibration/dashboard_calibration/entry_builder.py — source lines 43–56 (build_search_entry)
def build_search_entry(
    *,
    tokens: int,
    route: str,
    ts_day: str,
) -> tuple[str, dict[str, Any], int]:
    request_id = str(uuid.uuid4())
    params = {
        "route": route,
        "correlation_id": f"cal-{ts_day}-{request_id[:8]}",
        "agent_platform": AGENT_PLATFORM,
        "query": "calibration mock search",
    }
    return request_id, params, tokens

Three fields in that structure are the whole accounting argument. The request identifier makes the
row unique, so a retried call is two rows rather than one and a lost result is visible. The
correlation identifier ties the row to a day and an id prefix, which is what turns "calls per
answer" from an estimate into a query. And the token count is carried on the row rather than
inferred from the query length, because a search result is only free until you put it in a prompt —
at which point the retrieved pages become the dominant term in the bill, which is where fitting evidence into context stops being a comfort
argument and becomes arithmetic.

Latency deserves the same treatment. An index query's p95 is a property of your own infrastructure,
so it is tunable, cacheable and measurable on a graph you control. A live search's p95 is somebody
else's rate limiter and somebody else's crawl, and no amount of caching fixes a tail that is decided
outside your boundary. Cost per completed answer
is the single number that makes the comparison fair, because the cheap path that needs three
attempts is not the cheap path.

Implementation: choosing the retrieval path per question

The choice is not architectural in the sense of being permanent; it is a routing rule that can be
written down and tested. Five signals decide it, and all five are knowable before generation.

Freshness. If the true answer changed this week and the change matters to the reader, the live search is the only path that can be correct. If the answer is a policy that changes twice a year, freshness is not a reason to pay per call.
Ownership of the truth. If the answer lives in documents you control — product documentation, tickets, contracts, runbooks — index them and retrieve from the index, because only there can you guarantee that the clause you cite is the clause that is currently in force.
Privacy and licence. Private data must not be sent to a third-party engine, and licensed material must not be copied into an index. Those two constraints point in opposite directions, and a system that ignores either one fails an audit rather than a test.
Repeat rate. A question asked a thousand times should be answered from an index or a cache once the corpus covers it. Paying a third party per call for the same question is a choice you can only justify while the corpus is thin.
Precision of citation. A reference answer wants a passage with a stable identifier. A snippet from a page that may be rewritten tomorrow is evidence that expires with the page.

The hybrid is the pattern most teams converge on, and it is worth naming because it grows the index
rather than replacing it: search once for pages that look authoritative, fetch them, store them,
and retrieve from the store afterwards. The first question pays the search cost and every
subsequent one pays the index cost. What the pattern does not do is remove the corpus's obligations
— once you are storing fetched pages you own a freshness cadence and a recall problem again, and the
rules of the ingestion and retrieval pipeline
apply to pages you scraped exactly as they apply to documents your team wrote.

Tool definition: the contract a search tool exposes

A search tool is defined by its schema, and the schema is prompt surface, so it is worth keeping
small and explicit. A defensible one declares the query as natural language, the number of results,
an optional recency window, an optional domain restriction, and an output shape that carries the
title, URL, snippet and publication date of each result. The date matters more than it looks: it is
the only field that lets the model reason about whether the page is current, and it is the field
most tools quietly omit. A timeout, a cost annotation and a structured failure response belong in
the contract too, because a loop that cannot tell a rate limit from an empty result will retry the
wrong one. How a tool is declared is the reference for the
mechanics of that schema.

The contrast with a retriever's interface is instructive in both directions. A retriever has no
schema the model sees: its signature is fixed in your code, changing it means a code review rather
than a prompt change, and adding a filter costs nothing at inference time. A tool's signature is
read by the model on every request, so every parameter is tokens and every optional field is a
chance to be misused. That is the reason good search tools look austere: one query, one count, and
perhaps one filter for freshness — with the rest of the tuning moved into the application, where it
costs nothing to run and can be tested.

Architecture: freshness, ranking ownership and cost per call

Laid out side by side, the two paths differ on almost every operational dimension, which is why
"search or RAG" is usually the wrong question and "which of these two do I need for this question"
is the right one.

Dimension	A search tool (agentic search)	A retrieval index (RAG)
What is read	The live web, through a provider's index	Fragments of a corpus you ingested
Who owns the ranking	A search engine you do not operate	Your chunking, embedding model and filters
Freshness	Minutes to days, decided outside your boundary	Your ingestion cadence, decided by you
Cost shape	Per call, plus per page fetched	Fixed at ingest, near zero per query
Cacheability	Weak — the same query can return different pages	Strong — the same query returns the same passages until a reindex
Failure mode	Rate limits, empty result sets, pages that changed under you	Missing evidence, stale fragments, recall that nobody measured
Right for	Current events, public facts, open-ended discovery	Policies, product knowledge, private documents, exact citations

The table also names the two failure modes that surprise teams most. A search-backed answer fails
by being untraceable: the page moved, or the snippet lost the clause that mattered, and the
transcript no longer explains the answer. An index-backed answer fails by being confidently narrow:
the fragment that came back is real, current and insufficient, and nothing in the pipeline notices.
Both failures are cheaper to detect with instrumentation than with prose, which is the argument for
measurement on the same page as the choice,
the pipeline-versus-loop decision being the workload-level version
of it and the survey of retrieval loops naming the pieces each path
shares.

That framing is the workload-level half of a larger split: rag vs agentic ai
is where the vocabulary itself is separated, so a choice made here is not mistaken for a choice
about the architecture above it.

How SmartGate compares

The question this page answers is where the retrieval path, its limits and its record live, and
whether a single endpoint can carry both paths without the accounting splitting in two.

	What it decides	What that means for a mixed workload
A vector store and a client library	What the index returns	Cheap per query, blind to anything that happened after the last ingest
A provider SDK called directly	What the search provider returns	Fresh by construction, billed per call, with the ranking and the rate limit outside your control
A bespoke proxy in front of both	Whatever it counts	Two accounting paths that drift apart exactly when a cost problem appears
SmartGate	When each tool is called, with per-key limits, a spend guard and one audit row per call	A search call and a retrieval call produce the same shape of record, so cost per completed answer is a query rather than a reconciliation

The free tier covers 2,000,000 tokens a month and all seven tools; Pro starts at 18 dollars a month
and Teams at 55 dollars a month. Those limits sit on the call path itself rather than in a
dashboard, which is what makes them comparable across the two retrieval paths this page is about —
where tool calls are metered is the longer version of that argument, and
what a run remembers is what keeps a long search-heavy run
from re-reading the same pages.

How to get started

Classify twenty real questions. Freshness-dependent or corpus-bound, and write the ratio down. Everything else on this page is downstream of that number.
Measure the index path first, because it is cheap to measure. Recall at k on a labelled set, plus the ingestion date of the fragments that were retrieved. An index without a freshness field is an index nobody can route around.
Instrument the search calls before you add a second one. One row per call with a correlation identifier and a token count. Without the record, the comparison between the paths is a preference.
Write the routing rule as code, not as a guideline. A cheap classifier for freshness and ownership in front of the retrieval decision, with both paths behind the same tool surface.
Re-measure when the corpus changes. A workload that needed live search last quarter is often answerable from the index once the pages it kept fetching have been ingested, and the repeat rate is what tells you when.

Start on the free tier — 2,000,000 tokens a month and all seven tools — with
start free; the plan limits are on the
pricing page, the tool parameters are in the
product docs, and contract traffic starts at the
contact form.

FAQ

Is agentic search just RAG with a web search tool?
No. Both place retrieved text in a prompt, so the generation step is the same, but the retrieval
step differs in three properties that decide the design: the corpus is not yours, the ranking is
not yours, and the cost is per call rather than per ingest. That is a different engineering problem
with a different failure mode, not a configuration change to a retriever.

When should I build an index instead of calling a search API?
When the truth lives in documents you control, when the same questions repeat, or when the answer
has to cite an exact revision. An index also wins whenever private data may not be sent to a third
party. Build the index when those three conditions hold, and keep the search tool for everything
that happened after your last ingest.

Do I have to choose one?
No, and most systems at scale run both behind one interface, routing per question. The signals are
cheap to evaluate before generation: how recent the answer needs to be, whether the fact is
owned by a document you have, and how often the question repeats. Routing beats a global preference
because the ratio of freshness questions to corpus questions changes as your corpus grows.

Why does freshness cost more?
Because a live search is somebody else's infrastructure rented per call, while an index is your own
infrastructure paid for once. The index converts a recurring cost into a fixed one and accepts a
cadence in exchange. The recurring cost is not waste — it is the only way to answer a question
whose answer changed this morning, and it comes with a latency tail you cannot cache away.

What breaks first in a search-backed system?
Rate limits and untraceable answers. The first is a capacity problem you can see coming; the second
is worse, because by the time a reader asks why the answer changed, the pages have been rewritten
and the transcript does not show what the model actually read. Recording the result set, its dates
and the query text is what keeps the answer auditable after the web has moved on.

Limitations and what this does not do

Three of the seven planned sections carry code. The LangChain comparison, the routing rules, the tool contract and the architecture comparison matched no unique symbol in this codebase and are written from the published research with no quoted implementation, as the abstain rule requires.
The quoted code is the plumbing around a search call, not a search engine. The snippets show how a request is serialised, how a configuration constant is validated and how one call is recorded for calibration. None of them ranks, fetches or summarises results, and the page does not claim otherwise.
The comparison table is a shape, not a benchmark. It names the dimensions on which the two paths differ so that you can measure your own; it reports no numbers, because the latency and cost that matter are the ones on your deployment against your providers.
The routing rules are a skeleton. Freshness, ownership, privacy, repeat rate and citation precision are the signals worth evaluating; the thresholds that turn them into a classifier are properties of your traffic and your corpus, and this page does not invent them.
This page is not a build guide for either path. The loop that calls a tool and the stages inside an index each have their own page in this cluster, and neither is reproduced here.

Sources

Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401
Gao et al. — Retrieval-Augmented Generation for Large Language Models: A Survey: https://arxiv.org/abs/2312.10997
Asai et al. — Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection: https://arxiv.org/abs/2310.11511
Jin et al. — Search-R1: Training LLMs to Reason and Leverage Search Engines: https://arxiv.org/abs/2503.09516
Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models: https://arxiv.org/abs/2210.03629
Singh et al. — Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG: https://arxiv.org/abs/2501.09136
Es et al. — RAGAS: Automated Evaluation of Retrieval Augmented Generation: https://arxiv.org/abs/2309.15217
Liu et al. — Lost in the Middle: How Language Models Use Long Contexts: https://arxiv.org/abs/2307.03172
SmartGate — documentation, pricing and sales contact: https://smartgate.network/docs · https://smartgate.network/pricing · https://smartgate.network/contact

Method note

The code in this article is not transcribed. Each block was cut directly out of the slice body
returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body
before publication; the first line inside every fence records the file and the exact source lines.
Symbols were pinned with whole-name containment (rule A level 2) and confirmed by the service's
slot-proof endpoint — 3 of the 7 planned sections matched; the other four are written from the
published research with no code. The three pinned slices are the parts of a search call that a
retrieval decision actually depends on: the request as a set of serialisable parameters, a tuning
constant with a validated range, and the per-call record that makes cost per answer a query. They
are evidence about what each path leaves you in control of, not an implementation of a search tool.

Slice provenance

#	SERP keyword	Symbol	File	Source lines	How it was pinned	sha256(12)
1	agentic retrieval augmented generation a survey on agentic rag	`serializeLogsSearchParams`	`lib/dashboard/logs-url.ts`	31–45	rule A L2 → slot-proof	`09b90cf6849f`
2	agentic retrieval-augmented generation: a survey on agentic rag	`parseSearchKappa`	`config/billing-savings-share.ts`	7–14	rule A L2 → slot-proof	`741541c43d1a`
3	rag vs agentic	`build_search_entry`	`dashboard-calibration/dashboard_calibration/entry_builder.py`	43–56	rule A L2 → slot-proof	`34911a2425fc`

Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before
publication. 3 of 7 sections pinned, 0 abstentions, and
4 section(s) written from external sources because no unique symbol matched.

DEV Community