Building a Production AI Agent in Spring Boot: Tools and Function Calling

tags: [java, springboot, ai, springai]

Six months ago, I added an AI assistant to an e-commerce platform I was building. The idea was simple: let users type "show me orders with delayed shipping" or "what is the inventory level for wireless headphones?" and get real answers from live data.

The first version worked. Barely. The model hallucinated order IDs, made up inventory numbers, and confidently claimed it could process refunds (it could not). The problem was not the LLM — it was that I had given it no real tools. It was guessing everything from its training data.

Every AI agent I have seen in production follows the same pattern: the LLM decides what action to take, calls a tool, gets a result, and continues. The hard part is wiring that loop correctly — defining tools that the model understands, handling edge cases, and keeping the code testable.

I am a Senior Software Engineer II at BS23 in Dhaka, with six years of Spring Boot experience and an OCP certification. I have been building with Spring AI since its early releases. This article kicks off a series on building production AI agents with Spring Boot 4 and Spring AI 2.0. Here is how I think about tools and function calling.

What Makes an Agent an Agent?

A lot of products call themselves "AI agents" these days. The term has been stretched to cover everything from a chatbot with a system prompt to fully autonomous coding assistants. For this series, I use a specific definition:

An AI agent is a system where an LLM decides which tool to call and in what sequence, based on the conversation context, and executes those calls with real side effects.

The key difference from a simple chatbot is the loop:

1. User sends a request
2. LLM decides: respond directly, or call a tool
3. If tool call: execute the tool, feed the result back to the LLM
4. LLM decides again — respond or call another tool
5. Repeat until done

This is called the agent loop or the thought-action-observation loop. Spring AI 2.0 gives you the building blocks for it without reinventing the prompt engineering wheel.

Setting Up the Project

I start every Spring AI project from start.spring.io with Spring Boot 4.1.0 and JDK 26. You need at least two dependencies:

• Spring Web for the REST layer
• Your LLM provider (OpenAI, Anthropic, Google Gemini, or Ollama for local models)

Spring AI's abstraction layer means the code stays the same regardless of which model you plug in. Here is the basic structure:

@SpringBootApplication
public class AgenticApplication {

    public static void main(String[] args) {
        SpringApplication.run(AgenticApplication.class, args);
    }

    @Bean
    ChatClient chatClient(ChatClient.Builder builder) {
        return builder
            .defaultSystem("""
                You are an e-commerce assistant. You help users find
                products, check inventory, and track orders.
                Use the tools available to you. Never make up data.
                """)
            .build();
    }
}

The ChatClient.Builder is your entry point. You configure default behavior here — system prompt, default advisors, tool registration. In production, you will configure these per-session, but the builder gives you sensible defaults.

Defining Tools with @tool

Spring AI's @Tool annotation is the cleanest way I have found to expose Java methods as LLM-callable tools. The annotation approach gives you:

• Automatic schema generation — Spring AI converts your method signature into JSON schema that the LLM understands
• Parameter descriptions — the model reads your Javadoc to decide when and how to call the tool
• Type-safe return values — the model gets structured data, not raw strings

Here is a simple product lookup tool from my e-commerce agent:

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;

@Service
public class ProductTools {

    private final ProductRepository productRepository;

    public ProductTools(ProductRepository productRepository) {
        this.productRepository = productRepository;
    }

    @Tool(description = "Search products by name, category, or keyword")
    public List<Product> searchProducts(
            @ToolParam(description = "Search query (product name, brand, or keyword)")
            String query,

            @ToolParam(description = "Category filter (optional)", required = false)
            String category,

            @ToolParam(description = "Max results to return", required = false)
            int limit) {

        if (limit <= 0) limit = 10;
        return productRepository.search(query, category, limit);
    }
}

Three things matter here:

The @tool description. This is what the LLM sees when it decides whether to call this tool. Be specific. "Search products by name, category, or keyword" is better than "Product search" because it tells the model what kinds of queries map to this tool.

The @ToolParam descriptions. The model uses parameter descriptions to decide how to fill arguments. If you omit descriptions, the model guesses — and it guesses wrong more often than you would expect.

The return type. Spring AI serializes the return value to JSON and feeds it back to the model. A List<Product> with proper fields (name, price, stockLevel, imageUrl) gives the model structured data it can use in its response.

Registering Tools with the ChatClient

Once you have defined tools, you register them with the ChatClient. Spring AI scans the bean for @Tool methods and generates the function-calling schemas automatically:

@RestController
public class AgentController {

    private final ChatClient chatClient;

    public AgentController(ChatClient.Builder builder, ProductTools productTools) {
        this.chatClient = builder
            .defaultTools(productTools)
            .build();
    }

    @PostMapping("/chat")
    public String chat(@RequestBody String message) {
        return chatClient.prompt()
            .user(message)
            .call()
            .content();
    }
}

That is the simplest possible agent. When a user asks "do you have wireless headphones under $150?", the flow is:

1. The request goes to the LLM with the product search tool available
2. The LLM decides to call searchProducts(query="wireless headphones", limit=10)
3. Spring AI executes the method and returns the results to the LLM
4. The LLM formats the response: "Yes, we have the Sony WH-1000XM6 at $129 and the JBL Tune Beam at $89"
5. The user sees the formatted answer

This works for single-call scenarios. But real agents need multiple tools and multiple rounds.

Building the Agent Loop

A production agent rarely stops after one tool call. The user asks a follow-up, the agent needs data from two different systems, or the first tool call does not return enough information. The agent loop handles all of these.

Spring AI's ToolCallback gives you lower-level control than annotations. You can build tools programmatically, compose them, and manage the conversation history yourself:

@Service
public class AgentOrchestrator {

    private final ChatClient chatClient;

    public AgentOrchestrator(ChatClient.Builder builder,
                             ProductTools productTools,
                             OrderTools orderTools,
                             CustomerTools customerTools) {
        this.chatClient = builder
            .defaultTools(productTools, orderTools, customerTools)
            .build();
    }

    public String execute(String userMessage) {
        return chatClient.prompt()
            .user(userMessage)
            .call()
            .content();
    }
}

Spring AI's ChatClient handles the loop internally when the model requests multiple tool calls. But for complex agents, I prefer explicit control. Here is a pattern I use for multi-turn conversations:

public AgentResponse executeWithHistory(String userMessage, List<Message> history) {
    var prompt = chatClient.prompt()
        .messages(history)
        .user(userMessage);

    var response = prompt.call();

    // Check if the LLM requested tool calls
    while (response.hasToolCalls()) {
        response = prompt
            .tools(response.getToolCalls())
            .call();
    }

    return new AgentResponse(
        response.getContent(),
        response.getMetadata()
    );
}

The key insight: each tool call result goes back to the model for evaluation. The model may decide to call another tool, refine its previous answer, or respond directly to the user. You iterate until no more tool calls are requested.

A Real Example: Order Tracking Agent

Let me show you a complete tool from my Agentic Shop project. This tool looks up real order data and returns structured information:

@Service
public class OrderTools {

    private final OrderRepository orderRepository;

    public OrderTools(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    @Tool(description = "Look up an order by its ID and return status, items, and delivery info")
    public OrderLookupResult getOrder(
            @ToolParam(description = "The 10-character order ID (format: ORD-XXXXXXXX)")
            String orderId) {

        var order = orderRepository.findById(orderId);

        if (order.isEmpty()) {
            return new OrderLookupResult(
                "not_found",
                null,
                "Order " + orderId + " was not found. Ask the user to check the order ID."
            );
        }

        var o = order.get();
        return new OrderLookupResult(
            o.getStatus(),
            o.getEstimatedDelivery().toString(),
            o.getSummary()
        );
    }

    @Tool(description = "List all orders for a customer email address")
    public List<OrderSummary> listCustomerOrders(
            @ToolParam(description = "Customer's email address")
            String email) {

        return orderRepository.findByCustomerEmail(email)
            .stream()
            .map(o -> new OrderSummary(
                o.getId(),
                o.getStatus(),
                o.getTotal(),
                o.getCreatedAt().toString()
            ))
            .collect(Collectors.toList());
    }
}

record OrderLookupResult(String status, String estimatedDelivery, String summary) {}
record OrderSummary(String id, String status, BigDecimal total, String createdAt) {}

The record-based return types are intentional. Spring AI serializes records cleanly to JSON, and the model can reference specific fields in its response. "Your order ORD-ABCD1234 is currently shipped and estimated to arrive on August 5, 2026" comes directly from the tool output, not from the model's training data.

Error Handling That Saves Your Reputation

The most common failure mode in AI agents is silent hallucination. The model calls a tool, gets an error, and invents a plausible-sounding answer instead of admitting failure.

I handle this with a simple convention: every tool returns a structured response that includes a success flag. The model is instructed (in the system prompt) to never answer tool questions unless the success flag is true:

record ToolResult<T>(boolean success, T data, String errorMessage) {}

Your tools catch exceptions internally and return ToolResult.success(false, null, "details"). The system prompt explicitly tells the model: "If a tool returns success=false, tell the user exactly what went wrong. Do not guess."

This alone eliminated about 80% of the hallucination issues I saw in early versions.

Testing the Agent

Testing an AI agent is different from testing a regular Spring Boot service. You are testing the orchestration, not the LLM's specific responses. I use three testing strategies:

Unit tests for tools. Each @Tool method is a plain Java method. Test it like any service:

@Test
void searchProducts_returnsResultsForValidQuery() {
    var tools = new ProductTools(productRepository);
    var results = tools.searchProducts("headphones", null, 5);
    assertThat(results).hasSize(3);
    assertThat(results.get(0).name()).containsIgnoringCase("headphones");
}

Integration tests for the agent loop. Mock the LLM to return specific tool call requests, then verify the orchestration:

@Test
void agent_callsOrderTool_whenUserAsksAboutOrder() {
    var response = agent.execute("Where is my order ORD-001?");
    assertThat(response).contains("shipped");
}

Manual scenarios for behavioral testing. I keep a list of 10-15 realistic user queries and run them against the agent after every change. This catches regressions that unit tests miss — like when a tool parameter rename makes the model confused.

What We Did Not Cover

This article focused on tools and function calling — the mechanical foundation of an AI agent. I did not cover:

• Conversation memory — storing and retrieving conversation history across sessions
• Tool result caching — avoiding redundant database calls when the model repeats a query
• Observability — tracing tool calls, measuring latency, debugging failures
• Multi-agent orchestration — when one agent delegates to another

All of those are coming in future parts of this series. If there is one thing I have learned building production AI agents, it is that the tool layer is the foundation. Get it right, and everything else becomes manageable.

The Takeaway

Spring AI 2.0's @Tool annotation and ChatClient give you a solid foundation for building AI agents in Java. The pattern is consistent: define tools with clear descriptions, register them with the client, handle errors explicitly, and test each layer independently.

I have been building with Spring AI since version 0.8, and the 2.0 release is the first time the tool abstraction feels production-ready. The annotation-driven schema generation alone saves hours of manual JSON schema writing.

Have you built an AI agent with Spring AI? What tools did you expose, and what edge cases surprised you? I read every response.

I write about Java, Spring Boot, and AI agents every week. Subscribe — it is free, and Part 2 (conversation memory and context management) goes out soon.

If you found this useful, bookmark it. The code patterns here are the base I use across every agent I build. Save it for the next time you need to add an AI feature to a Spring Boot application.