DEV Community

Cover image for Spring AI Tutorial: How Java Developers Can Build Generative AI Applications with Spring Boot
Ayush Shrivastava
Ayush Shrivastava

Posted on

Spring AI Tutorial: How Java Developers Can Build Generative AI Applications with Spring Boot

#openai #ai #springboot #java

If you are a Java developer working with Spring Boot, you already know how to build REST APIs, microservices, database-driven applications, authentication systems, and production backends.

But AI development introduces a new set of questions:

  • How do I connect an LLM to my Spring Boot application?
  • How do I send prompts from Java?
  • How do I work with OpenAI or local models?
  • How do I give an AI application access to my application's data?
  • How can an LLM call my Java methods?
  • How do I build RAG applications with Spring Boot?
  • How do I build AI agents instead of simple chatbots?

This is where Spring AI becomes interesting.

Spring AI brings AI application development into the Spring ecosystem and provides abstractions for working with AI models, embeddings, vector stores, memory, tools, and other AI application patterns.

According to the Spring AI roadmap in this series, we will move from a simple AI application to concepts such as RAG, tool calling, MCP, evaluation, observability, and multimodal applications.

In this article, we will start from the beginning.

What Is Spring AI?

Spring AI is a framework for integrating AI capabilities into Spring applications.

The idea is simple:

Instead of learning a completely different programming model for AI, Spring developers can use familiar Spring concepts and abstractions to communicate with AI models.

Spring AI provides integrations with multiple AI providers and supports capabilities such as chat models, embeddings, vector stores, memory, tool calling, MCP, and AI observability.

You can think about the architecture like this:

Spring Boot Application
        |
        v
     Spring AI
        |
        +------------------+
        |                  |
        v                  v
   Chat Model          Embedding Model
        |                  |
        v                  v
     LLM API          Vector Store
Enter fullscreen mode Exit fullscreen mode

The important part is that your business application remains a Spring Boot application.

You are simply adding an AI layer to it.

Why Should Java Developers Learn Spring AI?

Imagine that you are building an employee management application.

Your application already has:

Java
Spring Boot
Spring Security
PostgreSQL
REST APIs
Docker
AWS
Enter fullscreen mode Exit fullscreen mode

Now your product team asks:

"Can we add an AI assistant that answers employee questions?"

For example:

User:
How many paid leaves can I take every year?

AI Assistant:
According to the company policy, employees are eligible
for 18 days of paid leave annually.
Enter fullscreen mode Exit fullscreen mode

A traditional backend developer might think:

Frontend
   |
   v
Spring Boot
   |
   v
Database
Enter fullscreen mode Exit fullscreen mode

But an AI-powered application may look more like:

Frontend
   |
   v
Spring Boot
   |
   v
Spring AI
   |
   +------> LLM
   |
   +------> Vector Database
   |
   +------> Internal APIs
   |
   +------> Business Logic
Enter fullscreen mode Exit fullscreen mode

This is the shift from building traditional backend systems to building AI-powered backend systems.

Spring AI provides the abstractions needed to connect these pieces.

Spring AI Is Not an AI Model

This distinction is important.

Spring AI is not itself an LLM.

It is an application framework that helps your Spring application communicate with AI models.

For example:

Your Spring Boot Application
            |
            v
        Spring AI
            |
       +----+----+
       |         |
       v         v
    OpenAI     Ollama
Enter fullscreen mode Exit fullscreen mode

The model is responsible for generating the response.

Spring AI provides the developer-friendly interface for communicating with that model.

This separation is useful because your business logic does not have to be tightly coupled to a single AI provider.

What Can You Build With Spring AI?

Spring AI can be used for much more than a basic chatbot.

Some examples include:

1. AI Customer Support

A customer asks:

Where is my order?
Enter fullscreen mode Exit fullscreen mode

Your application can combine AI with backend APIs to retrieve order information and generate a natural response.

2. Document Question Answering

Suppose your company has:

employee-policy.pdf
leave-policy.pdf
insurance-policy.pdf
Enter fullscreen mode Exit fullscreen mode

Instead of manually searching these documents, users can ask:

How many annual leaves do I get?
Enter fullscreen mode Exit fullscreen mode

A RAG pipeline can retrieve the relevant document content and provide it to the LLM. The PDF's roadmap specifically covers chunking, embeddings, vector stores, and document retrieval for this purpose.

3. AI-Powered Recommendations

You can use embeddings to understand user intent and build semantic recommendations.

4. AI Data Processing

Spring AI can also be used for:

  • Text generation
  • Content moderation
  • Transcription
  • Summarization
  • Classification

These are among the use cases highlighted in the course material.

5. AI Agents

Eventually, your application can move beyond simply generating text.

An AI system can:

Understand request
      |
      v
Retrieve information
      |
      v
Call Java method
      |
      v
Call external API
      |
      v
Make another decision
      |
      v
Return final response
Enter fullscreen mode Exit fullscreen mode

This is where concepts such as tool calling and agentic AI become important.

The First Spring AI Application

Let's build something simple.

Our first application will accept a message through a REST API and send that message to an AI model.

The flow is:

HTTP Request
     |
     v
Spring Boot Controller
     |
     v
ChatClient
     |
     v
ChatModel
     |
     v
AI Provider
     |
     v
Response
Enter fullscreen mode Exit fullscreen mode

This is intentionally simple.

The goal is to understand the basic Spring AI programming model before introducing RAG, memory, tools, or agents.

Step 1: Create a Spring Boot Project

Create a Spring Boot project with:

Java
Spring Boot
Spring Web
Spring AI
Enter fullscreen mode Exit fullscreen mode

The PDF uses the Spring Web MVC starter and the Spring AI OpenAI model starter for the initial example.

Your Maven dependencies can look like:

<dependencies>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webmvc</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-openai</artifactId>
    </dependency>

</dependencies>
Enter fullscreen mode Exit fullscreen mode

The exact Spring AI dependency/version should be aligned with the Spring AI version you choose for your project.

Step 2: Configure the API Key

If you are using a hosted model provider, you need credentials to communicate with it.

Do not hard-code API keys inside your Java source code.

Instead, use an environment variable.

For example:

spring.ai.openai.api-key=${OPENAI_API_KEY}
Enter fullscreen mode Exit fullscreen mode

Then configure:

OPENAI_API_KEY=your-api-key
Enter fullscreen mode Exit fullscreen mode

The course material also follows this approach by resolving the API key from an environment variable.

This is much better than:

String apiKey = "sk-xxxxxxxx";
Enter fullscreen mode Exit fullscreen mode

Never commit secrets into Git.

Step 3: Create the Chat Controller

Now let's create our first Spring AI endpoint.

@RestController
@RequestMapping("/api")
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    @GetMapping("/chat")
    public String chat(@RequestParam("message") String message) {

        return chatClient
                .prompt(message)
                .call()
                .content();
    }
}
Enter fullscreen mode Exit fullscreen mode

This is the core of our first application.

The PDF uses the same basic approach: inject ChatClient.Builder, build a ChatClient, provide a user message, call the model, and extract the response content.

Step 4: Run the Application

Start your Spring Boot application.

Now make a request:

GET /api/chat?message=What is Spring Boot?
Enter fullscreen mode Exit fullscreen mode

The request reaches:

ChatController
      |
      v
ChatClient
      |
      v
AI Model
      |
      v
Generated Response
Enter fullscreen mode Exit fullscreen mode

You might receive something like:

Spring Boot is a framework built on top of Spring
that simplifies the development of Java applications...
Enter fullscreen mode Exit fullscreen mode

You have now created your first Spring AI application.

Understanding ChatClient

The most important class in this example is:

ChatClient
Enter fullscreen mode Exit fullscreen mode

ChatClient provides a higher-level API for interacting with an AI model.

It gives us a fluent programming style:

chatClient
        .prompt(message)
        .call()
        .content();
Enter fullscreen mode Exit fullscreen mode

This is much easier to work with than manually constructing provider-specific HTTP requests.

The course material describes ChatClient as the higher-level developer-friendly abstraction that handles prompt construction, message handling, model invocation, and extracting content from the response.

ChatModel vs ChatClient

This is one of the first concepts you should understand when learning Spring AI.

There are two important abstractions:

ChatClient
    |
    v
ChatModel
    |
    v
AI Provider
Enter fullscreen mode Exit fullscreen mode

What Is ChatModel?

ChatModel represents the lower-level interface for communicating with an AI model.

It handles things such as:

  • Communication with AI providers
  • Model-specific configuration
  • API interaction
  • AI request and response handling

The PDF describes implementations such as:

OpenAiChatModel
GeminiChatModel
MistralChatModel
Enter fullscreen mode Exit fullscreen mode

depending on the provider being used.

What Is ChatClient?

ChatClient sits at a higher level.

Instead of worrying about the underlying provider API, you can write:

chatClient
        .prompt("Explain dependency injection")
        .call()
        .content();
Enter fullscreen mode Exit fullscreen mode

The relationship can be visualized as:

Your Java Code
      |
      v
   ChatClient
      |
      v
   ChatModel
      |
      v
AI Provider API
Enter fullscreen mode Exit fullscreen mode

A useful mental model is:

ChatModel  = Engine

ChatClient = Steering wheel + Dashboard
Enter fullscreen mode Exit fullscreen mode

The engine does the actual work.

The dashboard gives you a convenient interface for controlling it.

The course material describes this same relationship between ChatClient and ChatModel.

Why Is This Abstraction Useful?

Imagine your application starts with one AI provider.

Spring Boot
    |
    v
Spring AI
    |
    v
Provider A
Enter fullscreen mode Exit fullscreen mode

Later, you decide that some workloads should use another model.

You don't want your entire application to become:

OpenAIHttpClient
OpenAIRequest
OpenAIResponse
OpenAIAuthentication
OpenAIModelConfiguration
Enter fullscreen mode Exit fullscreen mode

everywhere.

Instead, Spring AI provides abstractions that allow you to work at the application level.

This becomes particularly valuable when you have multiple models.

For example:

Simple Question
      |
      v
Lightweight Model

Complex Reasoning
      |
      v
Powerful Model

Local Development
      |
      v
Ollama
Enter fullscreen mode Exit fullscreen mode

The course material identifies task-based model selection, fallback strategies, A/B testing, user preferences, and specialized models as reasons to work with multiple chat models.

Spring AI Is Not Limited to OpenAI

One of the important benefits of Spring AI is that it is designed around multiple providers.

The course material covers:

OpenAI
Ollama
Docker Model Runner
AWS Bedrock
Enter fullscreen mode Exit fullscreen mode

as part of the introductory journey.

For example, Ollama allows developers to run models locally.

The course demonstrates:

ollama run llama3.2:1b
Enter fullscreen mode Exit fullscreen mode

and then configures Spring AI to use the local model.

This can be useful during development when you want to experiment with local models instead of depending entirely on a hosted API.

What About AWS Bedrock?

If you are already building applications on AWS, another option covered in the course is Amazon Bedrock.

Bedrock is a managed AWS service for building generative AI applications and provides access to foundation models from multiple vendors.

Your architecture can therefore look like:

Spring Boot
     |
     v
Spring AI
     |
     v
AWS Bedrock
     |
     v
Foundation Model
Enter fullscreen mode Exit fullscreen mode

This is particularly interesting for enterprise applications that already operate inside AWS environments.

The Bigger Spring AI Journey

Our first example is intentionally tiny.

In a production application, simply sending a prompt to an LLM is rarely enough.

Consider a customer support application.

A basic version might look like:

User
 |
 v
ChatClient
 |
 v
LLM
 |
 v
Answer
Enter fullscreen mode Exit fullscreen mode

But a real production system may need:

                    +----------------+
                    |  Chat Memory   |
                    +-------+--------+
                            |
                            v
User --> ChatClient --> Advisor --> LLM
                            |
             +--------------+--------------+
             |                             |
             v                             v
        Vector Store                  Java Tools
             |                             |
             v                             v
        Company Docs                 Backend APIs
Enter fullscreen mode Exit fullscreen mode

And then we can introduce:

RAG
Embeddings
Vector Databases
Tool Calling
MCP
Evaluators
Observability
Memory
Streaming
Structured Output
Enter fullscreen mode Exit fullscreen mode

These concepts form the larger Spring AI journey outlined in the PDF.

From Chatbots to AI Applications

There is an important progression to understand.

A simple LLM application looks like:

LLM
Enter fullscreen mode Exit fullscreen mode

It can answer questions based primarily on what it learned during training.

Then we add external knowledge:

LLM + RAG
Enter fullscreen mode Exit fullscreen mode

Now it can retrieve information from our documents or knowledge base.

Then we add tools:

LLM + RAG + Tools
Enter fullscreen mode Exit fullscreen mode

Now it can retrieve information and interact with external systems.

Finally, we can move toward agentic systems:

Agentic AI
Enter fullscreen mode Exit fullscreen mode

where the application can reason through tasks, use tools, retrieve information, and execute multiple steps.

The PDF presents this progression from LLM to RAG, tools, and agentic AI as the broader generative AI journey.

This is where Spring AI becomes particularly interesting for backend developers.

A Real-World Example

Let's take an e-commerce application.

A user asks:

Where is my order and when will it arrive?
Enter fullscreen mode Exit fullscreen mode

A normal LLM cannot automatically know the user's latest order.

Your backend might have:

PostgreSQL
    |
    +-- orders
    +-- customers
    +-- payments
    +-- shipments
Enter fullscreen mode Exit fullscreen mode

The AI application needs access to that real-time information.

A future Spring AI application could look like:

User
 |
 v
Spring Boot API
 |
 v
Spring AI
 |
 +----> LLM
 |
 +----> Order Tool
 |          |
 |          v
 |       Database
 |
 +----> Shipping Tool
            |
            v
        Shipping API
Enter fullscreen mode Exit fullscreen mode

Now the AI is not simply generating text.

It is interacting with your backend.

This is the foundation for tool calling, which the course introduces as a way to allow an AI application to access current data and perform actions through Java code.

What We Will Build Throughout This Series

This article is only the starting point.

The complete series will gradually move from basic Spring AI concepts to production-oriented AI application development.

The learning path will look approximately like this:

Spring AI Fundamentals
        |
        v
ChatClient and ChatModel
        |
        v
Prompts and Message Roles
        |
        v
Prompt Templates
        |
        v
Chat Options
        |
        v
Structured Output
        |
        v
Generative AI Fundamentals
        |
        v
Tokens and Embeddings
        |
        v
Chat Memory
        |
        v
RAG
        |
        v
Vector Databases
        |
        v
Tool Calling
        |
        v
MCP
        |
        v
AI Evaluation
        |
        v
Observability
        |
        v
AI Agents
        |
        v
Production AI Applications
Enter fullscreen mode Exit fullscreen mode

The source material specifically includes prompt templates, streaming, ChatOptions, advisors, prompt stuffing, structured output, memory, RAG, tool calling, MCP, evaluators, observability, voice, and image generation.

What You Should Know Before Starting

This series is designed primarily for developers who already understand Java and Spring Boot.

You should be comfortable with:

Java
Spring Boot
REST APIs
Basic Docker
Basic Postman
Enter fullscreen mode Exit fullscreen mode

You do not need to be an AI researcher.

You do not need to understand neural networks before writing your first Spring AI application.

We will learn the AI concepts gradually and connect them back to backend development.

The source material also lists Java, Spring Boot, Docker, and Postman familiarity as prerequisites.

Final Thoughts

If you are already a Java and Spring Boot developer, learning AI application development does not mean throwing away everything you already know.

In fact, your backend experience is extremely useful.

You already understand:

APIs
Databases
Authentication
Security
Microservices
Caching
Cloud
Distributed Systems
Testing
Observability
Enter fullscreen mode Exit fullscreen mode

Now you are adding another layer:

AI Models
Prompts
Embeddings
RAG
Tools
Memory
Agents
Enter fullscreen mode Exit fullscreen mode

Spring AI helps bring these AI capabilities into the Spring ecosystem using familiar application-development patterns.

And that is the goal of this series:

Go from Java Developer to AI Engineer by building real AI-powered backend applications with Spring AI.

In the next article, we will go one level deeper into ChatModel vs ChatClient in Spring AI, understand how the two abstractions work together, and build practical examples using different AI providers.

Frequently Asked Questions

What is Spring AI?

Spring AI is a framework for integrating AI models and AI application capabilities into Spring applications.

Is Spring AI only for OpenAI?

No. The course material covers multiple providers and approaches, including OpenAI, Ollama, Docker Model Runner, and AWS Bedrock.

Do I need Python to learn Spring AI?

No. This series focuses on Java and Spring Boot.

Do I need to train my own AI model?

No. Spring AI is primarily about integrating existing AI models into applications.

Can Spring AI work with my existing backend?

Yes. That is one of the main reasons it is useful for Spring developers. AI capabilities can be added alongside existing APIs, databases, services, and business logic.

What will I learn after the basics?

The series will progressively cover prompts, memory, embeddings, RAG, vector stores, tool calling, MCP, evaluation, observability, and more.

Series Navigation

Part 1: Spring AI Tutorial: How Java Developers Can Build Generative AI Applications with Spring Boot

Part 2: ChatModel vs ChatClient in Spring AI

Part 3: Build Your First Spring AI Application with OpenAI

Part 4: Run Local LLMs with Ollama and Spring AI

Part 5: Run AI Models Locally with Docker Model Runner

Part 6: Using AWS Bedrock with Spring AI

Part 7: Understanding Multiple Chat Models in Spring AI

Part 8: Understanding AI Message Roles

Part 9: System Messages and User Messages in Spring AI

Part 10: Default Configuration in Spring AI ChatClient

The series will then move into prompt engineering, structured output, LLM fundamentals, tokens, embeddings, memory, RAG, vector databases, tool calling, MCP, evaluation, observability, and AI agents.

Top comments (0)