RAG gives the model access to your documents. It retrieves, it answers — but it never does anything. If answering a question requires checking a live API, querying a database, or chaining several steps together, RAG isn't enough.
That's where agents come in.
What an agent actually is
A regular chat call is one-shot: you send a message, the model responds, done. The model produces text — and only text. It has no way to interact with the outside world.
An agent breaks that limitation.
The idea is simple: you give the model a list of tools — capabilities your application exposes — and the model can invoke them during its response. It can call a tool, read the result, decide what to do next, call another tool, and eventually produce a final answer.
The model doesn't execute the tools itself. Your code does. The model just decides when to call them and what arguments to pass.
Think of it like a senior engineer delegating work. The manager (the model) decides what needs to be done. The engineers (your tool functions) actually do the work and report back. The manager reviews the results and figures out the next step.
What is a "tool"?
In the context of AI agents, a tool is just a function in your application that the model is allowed to call.
That function can do anything a normal Java method can do: call an external API, run a database query, read a file, do a calculation, send a notification. From the model's perspective, a tool is a named capability with a description and a set of inputs. From your code's perspective, it's a plain method.
Before the model responds to a user message, it sees:
- The user's question
- A list of available tools (name, description, what parameters each expects)
The model decides whether it needs a tool to answer. If yes, it responds not with text but with a tool call request: "call this tool with these arguments." Your application executes the function, sends the result back, and the model continues.
The ReAct pattern — how agents reason
The ReAct pattern (Reason + Act) describes how a model behaves inside this loop:
Reason — the model thinks about what it knows, what it still needs, and which tool will help next.
Act — it calls a tool.
Observe — it receives the tool's result.
Reason again — given the new information, what should happen next? Call another tool? Combine results and answer?
Act or answer — this repeats until the model has enough to respond.
Here's a concrete example. A user asks: "What's the weather like in Mumbai, and should I carry an umbrella?"
Without tools, the model can only guess based on training data. With a weather tool wired up:
-
Reason: I need live weather data for Mumbai. I have a
getWeathertool. -
Act: Call
getWeather("Mumbai") - Observe: Result: "32°C, humidity 88%, chance of rain 70%"
- Reason: I now have the data. I can answer.
- Answer: "Mumbai is hot and humid today with a 70% chance of rain. Yes, carry an umbrella."
The model drove that entire sequence. Your code just executed the getWeather function when asked.
The agent loop — step by step
Before looking at code, it helps to understand exactly what happens behind the scenes on every request:
1. User sends a message
2. Your app sends: message + list of available tools → model
3. Model responds with a tool call request (not final text yet)
4. Your app executes the tool, gets the result
5. Your app sends: original conversation + tool result → model
6. Model either calls another tool (back to step 3)
or produces the final answer (done)
Step 6 is the key: the loop continues until the model decides it has enough information. One question might trigger three tool calls. Another might need none. The model decides.
In Spring AI, you don't write this loop manually. The framework handles it. You write the tools; Spring AI runs the loop and returns the final answer.
Defining tools in Spring AI
Spring AI uses a @Tool annotation to mark methods as callable by the model. Any public method in a Spring bean can become a tool.
Here's the simplest possible tool:
@Service
public class WeatherTools {
@Tool(description = "Get the current weather for a city. Returns temperature in Celsius and a short description.")
public String getWeather(
@ToolParam(description = "The city name, e.g. Mumbai or Chennai") String city) {
// In a real app, this calls a weather API
// For now, returning dummy data
return "32°C, partly cloudy with high humidity";
}
}
A few things to understand here:
@Tool marks the method as a tool. The description is what the model reads to decide whether this tool is relevant to the current task. Write it as if you're explaining the tool to a new developer: what does it do, what does it return?
@ToolParam describes each parameter. The model reads this description when deciding what argument to pass. If the user says "weather in Bombay", the model needs to know whether to pass "Bombay" or "Mumbai" or a city code — your description guides that decision.
The return value is what the model sees. Whatever your method returns gets sent back to the model as the result of that tool call. Return a plain, readable string — not raw JSON that the model has to parse and interpret. Write it like you'd write a sentence explaining the result.
Wiring tools into the ChatClient
Once the tools are defined, you pass them to the ChatClient when making a request:
@RestController
@RequestMapping("/api/agent")
public class AgentController {
private final ChatClient chatClient;
private final WeatherTools weatherTools;
public AgentController(ChatClient.Builder builder, WeatherTools weatherTools) {
this.chatClient = builder.build();
this.weatherTools = weatherTools;
}
@PostMapping("/chat")
public String chat(@RequestBody ChatRequest request) {
return chatClient
.prompt()
.user(request.message())
.tools(weatherTools) // register tools for this request
.call()
.content();
}
}
.tools(weatherTools) tells Spring AI: "for this request, the model is allowed to call any @Tool method in this bean."
Spring AI automatically converts your annotated methods into the tool definition format the model expects — name, description, parameter names, types. You don't write any JSON schema by hand.
.call().content() waits for the entire agent loop to finish and returns the model's final text response. The tool calls, the loop, the result passing — all of that happens inside this one call. From your controller's perspective, it looks like a regular chat request.
Multiple tools and multi-step reasoning
An agent can have many tools. The model decides which ones to call, in what order, and how many times.
@Service
public class TravelTools {
@Tool(description = "Get current weather for a city. Returns temperature in Celsius and sky conditions.")
public String getWeather(@ToolParam(description = "City name") String city) {
return "24°C, clear skies";
}
@Tool(description = "Get top tourist attractions for a city. Returns a list of places to visit.")
public String getAttractions(@ToolParam(description = "City name") String city) {
return "Mysore Palace, Chamundi Hills, Brindavan Gardens, Devaraja Market";
}
@Tool(description = "Get current bus or train ticket price between two cities in India. Returns price in INR.")
public String getTicketPrice(
@ToolParam(description = "Departure city") String from,
@ToolParam(description = "Destination city") String to) {
return "Bus: ₹180, Train: ₹240 (2nd class)";
}
}
Now ask: "Plan a weekend trip from Bangalore to Mysore — weather, what to see, and how to get there."
The model has three tools and an open-ended request. It will:
- Call
getWeather("Mysore")→ "24°C, clear skies" - Call
getAttractions("Mysore")→ list of places - Call
getTicketPrice("Bangalore", "Mysore")→ fare info - Combine all three into a readable trip plan
Three tool calls, one user message, one final answer. The model orchestrated the sequence — your code just ran the functions.
Parallel tool calling
When multiple tool calls are independent of each other — when neither result depends on the other — the model can request them all at once.
For the trip planning example: weather, attractions, and ticket price can all be fetched simultaneously. None of them depend on each other's results. A capable model will issue all three calls in a single step, and Spring AI will execute them in parallel.
You write no parallel execution code. Spring AI detects that the model returned multiple tool calls in one response, runs them concurrently, and sends all results back together. The model then continues with the full set of results.
This matters for performance. Three sequential tool calls in a slow API might take three seconds. In parallel, they take the time of the slowest one.
System prompt for agents
Just like RAG needs a grounding instruction in the system prompt, agents benefit from a clear directive about when to use tools.
ChatClient.builder(chatModel)
.defaultSystemPrompt("""
You are a travel assistant for India.
Use the available tools to answer questions — always call tools for current information.
Do not answer from memory when a tool can provide the answer.
If the user asks about something outside travel, politely say you can only help with travel questions.
""")
.build();
"Always call tools for current information — do not answer from memory" prevents the model from skipping tool calls when it happens to know something from training data. Without this, the model might answer a weather question confidently from stale knowledge instead of calling your live weather tool.
When to use an agent — and when not to
Agents are powerful but have a real cost: every tool call is a round trip to your application. A three-tool agent call makes four calls to the model (initial request + one per tool result). That's slower and more expensive than a plain chat call.
The decision is straightforward:
Plain chat call — the model reads input and produces text. Summarisation, classification, translation, writing. No external data needed.
RAG — the model needs knowledge from your documents. No actions, no live data — just retrieval and response.
Agent — answering requires doing something: checking a live API, running a database query, performing a multi-step process where each step's result shapes the next.
Only reach for agents when the task genuinely requires action. An agent that just wraps a static lookup is overkill — that's a plain function call.
What's next
Tools are only as good as their descriptions. The next article covers tool design in depth — how to write descriptions the model understands, how to return results the model can reason about, and the failure modes that come from poorly designed tools.
Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.
Top comments (0)