In the previous parts of this series, we explored how to build AI-powered applications with Spring AI and run models locally. Local inference is great for experimentation, development, privacy-focused use cases, and understanding how LLM applications work.
But production AI systems often need more.
They need access to powerful foundation models, managed infrastructure, scalability, reliability, and the ability to switch between models without managing GPU servers yourself.
This is where AWS Bedrock comes in.
In this article, we'll explore how to integrate AWS Bedrock with Spring AI and move from local AI inference to a managed cloud-based architecture.
What Is AWS Bedrock?
AWS Bedrock is a fully managed AWS service that provides access to foundation models from multiple AI providers through AWS infrastructure.
Instead of downloading and running a model locally, your application sends requests to AWS Bedrock.
The architecture changes from this:
Spring Boot Application
↓
Spring AI
↓
Local Model Runtime
(Ollama / Local LLM)
To this:
Spring Boot Application
↓
Spring AI
↓
AWS Bedrock API
↓
Foundation Model
Your application no longer needs to manage the underlying AI infrastructure.
AWS handles the model hosting, scaling, availability, and infrastructure required to run inference.
Why Move from Local Models to AWS Bedrock?
Local models are incredibly useful, but they come with limitations.
For example:
- You may need powerful hardware.
- Large models require significant memory and GPU resources.
- Scaling inference for multiple users becomes difficult.
- You are responsible for hosting and managing the infrastructure.
- Production reliability becomes your responsibility.
AWS Bedrock solves many of these infrastructure challenges.
With Bedrock, you can focus more on your application instead of managing model servers.
Your architecture becomes more like this:
Client
↓
Spring Boot API
↓
Spring AI
↓
AWS Bedrock
↓
Foundation Model
This makes it easier to build cloud-native AI applications.
How Spring AI Fits Into the Architecture
Spring AI provides abstractions for working with different AI providers.
Your application interacts with high-level APIs such as ChatClient instead of manually writing HTTP requests for every model provider.
For example:
String response = chatClient.prompt()
.user("Explain AWS Bedrock in simple terms")
.call()
.content();
The application code can remain relatively similar even when the underlying AI provider changes.
For example, you might start with:
Spring AI → Ollama → Local Model
And later move to:
Spring AI → AWS Bedrock → Cloud Model
This abstraction is one of the major advantages of using Spring AI.
Your business logic should ideally depend on AI capabilities rather than being tightly coupled to a specific provider.
Setting Up AWS Bedrock
Before your Spring Boot application can communicate with AWS Bedrock, you need access to AWS and the required permissions.
At a high level, the process looks like this:
AWS Account
↓
Enable Model Access
↓
Configure IAM Permissions
↓
Configure AWS Region
↓
Spring Boot Application
↓
Spring AI + AWS Bedrock
The exact models available can vary depending on your AWS region and account configuration.
You also need appropriate AWS credentials and permissions for invoking models.
For local development, credentials can be provided through the AWS credential provider chain, environment variables, profiles, or other supported AWS authentication mechanisms.
A typical environment configuration might look like this:
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=your-region
However, in production, avoid hardcoding credentials in your application configuration.
Prefer IAM roles and managed identity mechanisms whenever possible.
Adding the Spring AI Bedrock Dependency
Spring AI provides integrations for AWS Bedrock models.
The dependency you use depends on the specific Spring AI version and Bedrock model integration you want to work with.
For example, your Maven configuration may include a Spring AI AWS Bedrock starter:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-bedrock-converse</artifactId>
</dependency>
Spring AI's dependency management should also be configured using the appropriate BOM for your version.
For example:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>YOUR_SPRING_AI_VERSION</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
The important idea is that Spring Boot auto-configuration can create the required AI components once the correct dependencies and AWS configuration are available.
Configuring AWS Bedrock
Your application configuration tells Spring AI how to connect to AWS Bedrock.
A simplified configuration might look something like this:
spring:
ai:
bedrock:
aws:
region: us-east-1
Depending on the Spring AI version and model integration, additional configuration may be required for the selected model, credentials, generation settings, or Bedrock Converse API.
For example, you may configure values such as:
- AWS region
- Model ID
- Temperature
- Maximum tokens
- Top P
- Credential provider settings
Conceptually:
spring:
ai:
model:
chat: bedrock-converse
bedrock:
aws:
region: us-east-1
The exact property names can vary between Spring AI releases, so always check the documentation for the version you are using.
Building a Chat API with Spring AI and Bedrock
Once the integration is configured, we can expose a simple API.
First, inject ChatClient.Builder:
@RestController
@RequestMapping("/api/chat")
public class ChatController {
private final ChatClient chatClient;
public ChatController(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
@GetMapping
public String chat(@RequestParam String message) {
return chatClient.prompt()
.user(message)
.call()
.content();
}
}
Now a request such as:
GET /api/chat?message=Explain Spring AI
follows this flow:
HTTP Request
↓
ChatController
↓
ChatClient
↓
Spring AI
↓
AWS Bedrock
↓
Foundation Model
↓
Generated Response
↓
Spring Boot API
The controller does not need to know the low-level details of the Bedrock API.
Spring AI handles the integration layer.
The Big Architecture Change
The most important change is not just replacing one dependency with another.
The architecture itself changes.
Local Inference
User
↓
Spring Boot
↓
Spring AI
↓
Ollama
↓
Local LLM
Here, you are responsible for the machine running the model.
You need to think about:
- CPU or GPU resources
- RAM
- Model downloads
- Model startup time
- Concurrent requests
- Scaling
- Monitoring the inference server
Cloud Inference
User
↓
Load Balancer
↓
Spring Boot Application
↓
Spring AI
↓
AWS Bedrock
↓
Foundation Model
Now the model infrastructure is managed separately from your application.
Your Spring Boot application becomes a consumer of AI infrastructure rather than the host of the model itself.
This separation can significantly simplify production architecture.
Choosing Between Different Foundation Models
One major benefit of Bedrock is access to multiple foundation models.
Different models can be better suited for different workloads.
For example:
Chat Application
↓
General Model
Document Analysis
↓
Long Context Model
Structured Extraction
↓
Model with Strong
Structured Output
Complex Reasoning
↓
Higher Capability Model
This means model selection becomes an architectural decision.
You should consider factors such as:
- Response quality
- Latency
- Cost
- Context window
- Tool calling capabilities
- Structured output support
- Regional availability
The best model is not always the largest or most expensive one.
A production system may even route different requests to different models.
Managing AI Configuration
One common mistake is placing model-specific configuration throughout the application.
Instead of this:
if (provider.equals("bedrock")) {
// Bedrock logic
}
if (provider.equals("ollama")) {
// Ollama logic
}
Try to keep provider-specific infrastructure separate from your application logic.
For example:
Application Layer
↓
Spring AI Abstraction
↓
Provider Configuration
↓
AWS Bedrock / Ollama / Other Models
This makes it easier to change providers later.
Your application should ideally focus on:
What should the AI do?
Instead of:
How does this specific provider's HTTP API work?
Authentication and Security
When working with AWS Bedrock, authentication becomes part of your architecture.
Your application needs permission to invoke models.
For production environments, the preferred approach is generally to use AWS IAM roles instead of embedding access keys inside application properties.
For example:
Spring Boot on AWS
↓
IAM Role
↓
Temporary Credentials
↓
AWS Bedrock
This is safer than:
application.yml
↓
Hardcoded AWS Keys
You should also think about:
- Least-privilege IAM permissions
- Credential rotation
- Environment separation
- Request logging
- Sensitive prompt data
- PII protection
Moving AI to the cloud also means thinking carefully about what data leaves your application.
Cost Becomes Part of the Architecture
With local models, the main cost is often infrastructure.
You pay for:
GPU
RAM
Compute
Storage
Servers
With managed inference, cost is often related to model usage.
Conceptually:
Application Request
↓
Input Tokens
+
Output Tokens
↓
Inference Cost
This means AI applications should treat token usage as an engineering concern.
You may need:
- Token monitoring
- Request limits
- Model selection policies
- Budget controls
- Prompt optimization
- Caching
For example, sending unnecessary context to a model can increase both latency and cost.
A good AI architecture should therefore optimize the entire request pipeline.
Adding Observability
Production AI systems need more than application logs.
You may want to monitor:
Request
↓
Prompt Size
↓
Model
↓
Latency
↓
Token Usage
↓
Cost
↓
Response
This helps answer important questions:
- Which requests are slow?
- Which model is being used?
- How many tokens are being consumed?
- Which prompts are expensive?
- Are failures coming from the application or the AI provider?
As your AI application grows, observability becomes essential.
Local Development vs Production
You don't necessarily need to abandon local models completely.
A practical architecture could use different providers for different environments.
Development
Spring Boot
↓
Ollama
↓
Local Model
Production
Spring Boot
↓
Spring AI
↓
AWS Bedrock
↓
Cloud Model
This gives developers a fast and inexpensive local development workflow while allowing production to use managed cloud infrastructure.
The important part is keeping your application architecture flexible enough to support both.
What's Next?
AWS Bedrock gives us access to powerful foundation models without managing the underlying AI infrastructure ourselves.
But moving AI into production introduces new challenges.
How do we manage conversation history?
How do we provide the model with our own documents?
How do we build applications that can retrieve relevant information before generating a response?
In Part 7, we'll take the next step and explore RAG with Spring AI.
We'll look at how documents move through the RAG pipeline:
Documents
↓
Chunking
↓
Embeddings
↓
Vector Database
↓
Similarity Search
↓
Relevant Context
↓
LLM
↓
Answer
This is where AI applications start becoming truly connected to your own data.
Next up: Building RAG Applications with Spring AI.
Top comments (0)