DEV Community

Solon Framework
Solon Framework

Posted on

Agent RAG in Solon: Make Retrieval a Tool, Not a One-Shot Pre-Step

#agents #ai #llm #rag

Most RAG demos stop at one search. You embed a question, pull top-k chunks, paste them into a prompt, and hope the first hit was enough. That works for FAQ pages. It fails when the first query is vague, the right facts are split across documents, or the model needs a second, more precise lookup.

Solon AI calls the upgrade Agent RAG: retrieval is no longer a fixed pre-step. It becomes a tool the model can call zero or more times inside a ReAct loop.

This article sticks to official Solon v4.0.3 APIs from the RAG and Agent docs.

Passive RAG vs Agent RAG

Mode Flow Who decides to search? Retry if context is weak?
Passive RAG search → augment → generate Your code, always once No
Agent RAG Thought → tool search → Observation → … The model Yes, multi-round

Passive RAG is still useful. Agent RAG is what you want when answers need iterative retrieval.

1. Build a small knowledge base

Same building blocks as classic Solon RAG: embedding model, storable repository, load + split + save.

import org.noear.solon.ai.embedding.EmbeddingModel;
import org.noear.solon.ai.rag.Document;
import org.noear.solon.ai.rag.loader.TextLoader;
import org.noear.solon.ai.rag.repository.InMemoryRepository;
import org.noear.solon.ai.rag.splitter.RegexTextSplitter;
import org.noear.solon.ai.rag.splitter.SplitterPipeline;
import org.noear.solon.ai.rag.splitter.TokenSizeTextSplitter;

import java.net.URI;
import java.util.List;

EmbeddingModel embeddingModel = EmbeddingModel.of(embeddingApiUrl)
        .apiKey(embeddingApiKey)
        .model(embeddingModelName)
        .build();

InMemoryRepository repository = new InMemoryRepository(embeddingModel);

// Load → split → save
List<Document> documents = new SplitterPipeline()
        .next(new RegexTextSplitter("\n\n"))
        .next(new TokenSizeTextSplitter(500))
        .split(new TextLoader(URI.create("https://example.com/policy.txt")).load());

repository.save(documents);
Enter fullscreen mode Exit fullscreen mode

InMemoryRepository is fine for demos. Production can swap in Milvus, Redis, Elasticsearch, Qdrant, and other repository plugins without changing the Agent wiring below.

2. Passive RAG still has a clean path

When one search is enough, keep the pipeline explicit:

import org.noear.solon.ai.chat.ChatModel;
import org.noear.solon.ai.chat.message.ChatMessage;

List<Document> context = repository.search(question);
ChatMessage prompt = ChatMessage.ofUserAugment(question, context);

String answer = chatModel.prompt(prompt).call().getMessage().getContent();
Enter fullscreen mode Exit fullscreen mode

Or let the repository do the one-shot augment:

ChatMessage prompt = repository.promptAugment(question);
Enter fullscreen mode Exit fullscreen mode

This is the classic “retrieve then generate” path. No agent required.

3. Turn the repository into a tool

Agent RAG starts when search becomes a callable tool. Official docs introduce RepositoryTool for that (available since Solon AI 3.10.1):

import org.noear.solon.ai.rag.RepositoryTool;

RepositoryTool repositoryTool = new RepositoryTool(repository);
Enter fullscreen mode Exit fullscreen mode

Stage A — ChatModel with tools

The model decides whether to search at all:

ChatModel chatModel = ChatModel.of(chatApiUrl)
        .apiKey(chatApiKey)
        .model(chatModelName)
        .defaultToolAdd(repositoryTool)
        .build();

chatModel.prompt("What is the refund window for lost packages?").call();
Enter fullscreen mode Exit fullscreen mode

Stage B — ReActAgent for multi-hop retrieval

For harder questions, upgrade to ReAct so the model can search, rethink, and search again:

import org.noear.solon.ai.agent.react.ReActAgent;
import org.noear.solon.ai.agent.session.InMemoryAgentSession;

ReActAgent agent = ReActAgent.of(chatModel)
        .defaultToolAdd(repositoryTool)
        .sessionWindowSize(8)
        .maxTurns(10)
        .build();

String answer = agent.prompt(
                "Customer paid 158 yuan. Package marked lost at Shanghai hub. "
                        + "What policy applies and what should support do next?"
        )
        .session(InMemoryAgentSession.of("support_user_42"))
        .call()
        .getContent();
Enter fullscreen mode Exit fullscreen mode

That is the heart of Agent RAG in Solon:

  1. Knowledge lives in a Repository
  2. RepositoryTool exposes it as a tool
  3. ReActAgent runs Thought → Action → Observation until it has enough evidence

4. Mix knowledge search with business tools

Real support agents need more than docs. They also need order APIs, logistics APIs, and compensation actions. Solon tools are just ToolProviders — knowledge tools and domain tools hang on the same agent.

import org.noear.solon.ai.annotation.ToolMapping;
import org.noear.solon.ai.chat.tool.AbsToolProvider;
import org.noear.solon.annotation.Param;

public class OrderTools extends AbsToolProvider {
    @ToolMapping(description = "Look up order amount and tracking number by order id")
    public String get_order(@Param(description = "order id") String orderId) {
        // call your order service
        return "{\"orderId\":\"" + orderId + "\",\"amount\":158.0,\"trackNo\":\"track_123\"}";
    }
}

public class LogisticTools extends AbsToolProvider {
    @ToolMapping(description = "Get logistics status by tracking number")
    public String get_logistic_status(@Param(description = "tracking number") String trackNo) {
        return "{\"status\":\"lost\",\"info\":\"lost at Shanghai hub\"}";
    }
}

ReActAgent agent = ReActAgent.of(chatModel)
        .defaultToolAdd(repositoryTool)   // policy / FAQ knowledge
        .defaultToolAdd(new OrderTools())
        .defaultToolAdd(new LogisticTools())
        .modelOptions(o -> o.temperature(0.0))
        .maxTurns(10)
        .build();
Enter fullscreen mode Exit fullscreen mode

Typical loop the model may invent:

  1. Search policy for “lost package refund”
  2. Call get_order
  3. Call get_logistic_status
  4. Answer with both policy text and operational next step

No custom orchestrator graph required for this pattern.

5. Keep multi-turn memory under control

Agent RAG often spans several user turns. Solon separates session storage from window size:

ReActAgent agent = ReActAgent.of(chatModel)
        .defaultToolAdd(repositoryTool)
        .sessionWindowSize(5) // only recent messages go to the LLM
        .build();

// Isolate users by session id
agent.prompt("Continue from the last ticket.")
        .session(InMemoryAgentSession.of("user_123"))
        .call();
Enter fullscreen mode Exit fullscreen mode

Built-in session backends (official comparison):

Class Storage When to use
InMemoryAgentSession Local map Dev / tests
FileAgentSession Local files CLI / single-node tools
RedisAgentSession Redis Multi-node production

sessionWindowSize is short-term history for the model. Long knowledge still lives in the repository, not in the chat transcript.

6. Optional hybrid senses

When internal docs are not enough, official Agent RAG samples also attach web/code tools from the skill ecosystem:

ReActAgent agent = ReActAgent.of(chatModel)
        .defaultToolAdd(repositoryTool)
        .defaultToolAdd(new CodeSearchTool())
        .defaultToolAdd(new WebsearchTool())
        .defaultToolAdd(new WebfetchTool())
        .build();
Enter fullscreen mode Exit fullscreen mode

Use this carefully in production: each extra tool increases tool-selection noise. Start with repository + domain APIs; add web tools only when the product needs live external facts.

A practical decision guide

Situation Prefer
Fixed FAQ, one clear query Passive RAG (search + ofUserAugment)
Vague question, multi-hop facts Agent RAG (RepositoryTool + ReActAgent)
Docs + order/logistics actions Agent RAG + AbsToolProvider tools
Long chat with same user AgentSession + sessionWindowSize
Clustered app servers RedisAgentSession

What not to invent

  • Do not fake a custom implements Tool protocol for knowledge search when RepositoryTool already wraps the repository.
  • Do not put entire policy PDFs into the system prompt “for simplicity” — load, split, and store them.
  • Do not confuse Harness memory (coding-agent long-term facts) with business RAG. For product Q&A, start from Repository + Agent tools.

Official references (Solon v4.0.3)

Takeaway

In Solon, Agent RAG is not a second product. It is a composition rule:

Repository (facts) + RepositoryTool (search action) + ReActAgent (decision loop).

Start passive when one search is enough. Flip the repository into a tool when the model must decide if, when, and how to retrieve — and keep domain tools on the same agent when answers must become actions.

Top comments (5)

Collapse
 
jacksonxly profile image
Jackson Ly •

strong framing, and the passive-vs-agent table nails the real difference. the piece i'd add: making retrieval a tool the model calls freely trades a fixed cost for a variable one, and the new failure mode is the stop condition. the model can under-search (answer confidently off a weak first hit) or over-search (re-query in a loop and burn tokens and latency). passive rag was at least predictable at "always once." so the thing that actually makes agent rag work isn't the tool loop, it's giving the model a decent "do i have enough yet" signal, otherwise you swap one-shot brittleness for premature answers or retrieval storms. curious how solon handles the stopping decision, model discretion or a max-rounds cap?

Collapse
 
solonjava profile image
Solon Framework •

Great point — the stop condition is the real product problem once retrieval becomes a tool.

In Solon, it is usually both layers:

  1. Model discretion first: the agent decides whether to call the retrieval tool again or answer.
  2. Hard guards around that loop:
    • max step / max tool-round limits on the agent run
    • StopLoopInterceptor for repeated same-action loops
    • ToolRetryInterceptor / ToolSanitizerInterceptor so bad observations do not keep the loop alive forever
    • sessionWindowSize so history does not silently expand the context

So Agent RAG is not “retrieve forever.” It is “retrieve when needed,” with explicit caps and loop guards. Passive RAG is predictable because it always runs once; Agent RAG is better when the model has a clear enough/not-enough signal and the runtime can still force a stop.

If useful, I can write a short follow-up on stop policies for retrieval tools.

Collapse
 
jacksonxly profile image
Jackson Ly •

i'd read that follow-up. the piece i'd want it to tackle is how far you can trust the model's own enough/not-enough signal, since sufficiency judgment is exactly where calibration is weakest. the cheapest external proxy i've seen work is marginal gain: if the last retrieval round came back mostly overlapping what's already in context, force the stop no matter what the model wants. it turns retrieve-when-needed into something the runtime can check without understanding the query.

Thread Thread
 
solonjava profile image
Solon Framework •

That marginal-gain proxy is a solid idea -- turning "do I have enough?" into a runtime check that doesn't need to understand the query. The overlap ratio between the current retrieve round and already-in-context chunks is exactly the kind of cheap external signal that can force a stop without trusting model calibration.

In Solon, you could implement this as a custom interceptor on the agent run: compare the Observation from RepositoryTool against the existing chat history (e.g., Jaccard similarity on chunk IDs or content hashes), and if the overlap exceeds a threshold, inject a stop signal into the turn loop. The HardGuard interceptor chain makes this straightforward to slot in without modifying the agent internals.

The real trick is picking the right overlap metric. Chunk ID overlap is cheap but fragile after repository updates. Content hash overlap (MinHash on chunk text) is more robust. Either way, the agent stays "retrieve-when-needed" and the runtime gets a deterministic stop trigger that doesn't depend on the model's sufficiency calibration.

Appreciate the thoughtful framing -- this would make a good standalone post on retrieval stop policies. Added to the writing queue.

Thread Thread
 
solonjava profile image
Solon Framework •

That marginal-gain proxy is a solid idea -- turning "do I have enough?" into a runtime check that doesn't need to understand the query. The overlap ratio between the current retrieve round and already-in-context chunks is exactly the kind of cheap external signal that can force a stop without trusting model calibration.

In Solon, you could implement this as a custom interceptor on the agent run: compare the Observation from the RepositoryTool against the existing chat history (e.g., Jaccard similarity on chunk IDs or content hashes), and if the overlap exceeds a threshold, inject a stop signal into the turn loop. The HardGuard interceptor chain makes this straightforward to slot in without modifying the agent internals.

The real trick is picking the right overlap metric. Chunk ID overlap is cheap but fragile after repository updates. Content hash overlap (MinHash on chunk text) is more robust. Either way, the agent stays retrieve-when-needed and the runtime gets a deterministic stop trigger.

Appreciate the thoughtful framing -- this would make a good standalone post on retrieval stop policies.