DEV Community

Cover image for Giving Your Spring AI Agent a Memory — Conversations That Actually Make Sense
Sham Prakash K
Sham Prakash K

Posted on AI-assisted

Giving Your Spring AI Agent a Memory — Conversations That Actually Make Sense

#java #springboot #ai #agents

Send two messages to a basic agent:

"My budget for the trip is ₹5,000."
"What can I do within that budget?"

The second message makes no sense without the first. If the agent has no memory, it treats every message as a fresh conversation with no history. It doesn't know what trip. It doesn't know what budget. It has to ask again — or worse, it guesses.

Memory is what turns a stateless request-response system into an agent that can hold a real conversation.

What memory means in this context

Memory in an AI agent means one thing: the conversation history that gets sent to the model on each request.

The model itself has no persistent state. Between requests, it remembers nothing. What creates the illusion of memory is appending previous messages to each new request, so the model always sees the full conversation up to that point.

Request 1:  [user: "My budget is ₹5,000"]
            ↓ model responds
            [assistant: "Got it, ₹5,000 budget."]

Request 2:  [user: "My budget is ₹5,000"]        ← history replayed
            [assistant: "Got it, ₹5,000 budget."] ← history replayed
            [user: "What can I do within that budget?"]  ← new message
            ↓ model responds with full context
Enter fullscreen mode Exit fullscreen mode

Your application stores the conversation history. On every new request, it prepends the stored history before sending to the model. That's the entire mechanism.

MessageChatMemoryAdvisor in Spring AI

Spring AI handles conversation history automatically through an advisor called MessageChatMemoryAdvisor. You saw it briefly in the RAG article — here's what it actually does.

@Bean
public ChatClient chatClient(ChatModel chatModel) {
    return ChatClient.builder(chatModel)
        .defaultAdvisors(
            new MessageChatMemoryAdvisor(new InMemoryChatMemory())
        )
        .build();
}
Enter fullscreen mode Exit fullscreen mode

InMemoryChatMemory is a simple in-process store — a Map from conversation ID to message list. Before every request, the advisor loads the history for the current conversation ID and prepends it to the messages. After every request, it appends the new user message and the model's response to the history.

You don't write any of this yourself. The advisor handles it.

Conversation IDs — one per user session

Memory is per-conversation, not global. Each user gets their own conversation history. You identify conversations with a string ID — typically a UUID generated when the session starts.

@PostMapping("/chat")
public String chat(@RequestBody ChatRequest request) {
    return chatClient
        .prompt()
        .user(request.message())
        .advisors(a -> a.param(
            AbstractChatMemoryAdvisor.CHAT_MEMORY_CONVERSATION_ID_KEY,
            request.conversationId()
        ))
        .call()
        .content();
}
Enter fullscreen mode Exit fullscreen mode 
public record ChatRequest(String message, String conversationId) {}
Enter fullscreen mode Exit fullscreen mode

The client generates a conversationId once (e.g. UUID.randomUUID().toString()) and sends it with every message in a session. The advisor uses it to look up and update the right history.

Two users with different conversation IDs get completely separate histories. Same agent, isolated memory.

What gets stored in history

The history isn't just user messages and assistant responses. In an agent with tools, the conversation history also includes tool calls and their results.

A typical history after one tool-calling turn looks like this:

[user]      "What's the weather in Chennai?"
[assistant] → tool_call: getWeather("Chennai")
[tool]      "Chennai: 34°C, humid, partly cloudy."
[assistant] "It's 34°C and humid in Chennai right now with some cloud cover."
Enter fullscreen mode Exit fullscreen mode

All four messages are stored and replayed on the next turn. This is what allows the agent to remember not just what it said, but what it did — which tools it called, what they returned, and what conclusion it drew.

If a user follows up: "Is that hotter than usual for this time of year?" — the model has the tool result in context and can reason about it without calling getWeather again.

The context window limit

Conversation history grows with every turn. The model has a context window — a maximum number of tokens it can process in a single request. Pack enough history into a conversation and you hit that limit.

When history overflows the context window, one of a few things happens depending on how your application handles it:

  • The request fails with a token limit error
  • The oldest messages get silently truncated (you lose early context)
  • The model gets confused because the history appears to start mid-conversation

Spring AI's MessageChatMemoryAdvisor accepts a chatHistoryWindowSize parameter that limits how many previous messages to include:

new MessageChatMemoryAdvisor(
    new InMemoryChatMemory(),
    AbstractChatMemoryAdvisor.DEFAULT_CHAT_MEMORY_CONVERSATION_ID,
    20  // keep last 20 messages
)
Enter fullscreen mode Exit fullscreen mode

With a window of 20, the advisor includes only the 20 most recent messages in each request. Older messages are still stored — the window just controls what gets sent to the model.

What window size to use? Start with 20–30 messages. A message is typically 50–200 tokens; 20 messages is roughly 1,000–4,000 tokens of history, leaving plenty of room for the system prompt, tool definitions, and the current response.

In-memory vs persistent memory

InMemoryChatMemory stores history in a Map inside your running application. It's fast and requires no infrastructure. The trade-off: it disappears when the application restarts.

For a development server or a demo, in-memory is fine. For production, you need persistent storage.

Spring AI provides a CassandraChatMemory and a Neo4jChatMemory, and the ChatMemory interface is straightforward to implement against any database:

public class PostgresChatMemory implements ChatMemory {

    private final JdbcTemplate jdbc;

    @Override
    public void add(String conversationId, List<Message> messages) {
        messages.forEach(msg ->
            jdbc.update(
                "INSERT INTO chat_history (conversation_id, role, content, created_at) VALUES (?,?,?,?)",
                conversationId, msg.getMessageType().name(),
                msg.getContent(), Instant.now()
            )
        );
    }

    @Override
    public List<Message> get(String conversationId, int lastN) {
        return jdbc.query(
            "SELECT role, content FROM chat_history " +
            "WHERE conversation_id = ? ORDER BY created_at DESC LIMIT ?",
            (rs, i) -> toMessage(rs.getString("role"), rs.getString("content")),
            conversationId, lastN
        );
    }

    @Override
    public void clear(String conversationId) {
        jdbc.update("DELETE FROM chat_history WHERE conversation_id = ?", conversationId);
    }
}
Enter fullscreen mode Exit fullscreen mode

Swap new InMemoryChatMemory() for new PostgresChatMemory(jdbc) and history survives restarts, scales across multiple instances, and can be queried or audited.

Clearing history

When a user starts a new topic or wants a clean slate, clear the history:

chatMemory.clear(conversationId);
Enter fullscreen mode Exit fullscreen mode

Without explicit clearing, history accumulates indefinitely (or until the window drops the oldest messages). For session-based use cases, clear on logout or on session expiry.

Memory and RAG together

When both MessageChatMemoryAdvisor and QuestionAnswerAdvisor are registered, the order matters:

ChatClient.builder(chatModel)
    .defaultAdvisors(
        new MessageChatMemoryAdvisor(new InMemoryChatMemory()),  // runs first
        new QuestionAnswerAdvisor(vectorStore,
            SearchRequest.defaults().withTopK(5))               // runs second
    )
    .build();
Enter fullscreen mode Exit fullscreen mode

Memory runs first so that follow-up questions like "tell me more about that" contain the previous turn's context before RAG tries to retrieve relevant chunks. With the order reversed, RAG searches based on an incomplete message that has no conversational context.

What memory doesn't solve

Memory gives the model access to recent conversation history. It doesn't give the model access to:

Facts from much earlier in a long conversation. Once messages scroll past the history window, they're gone from the model's view. If a user mentioned their budget 40 messages ago and the window is 20, the model won't see it.

Knowledge from other conversations. Each conversation ID has its own isolated history. An insight the model developed in one session isn't available in another.

Structured data. History is a list of text messages. If you need to remember structured facts — the user's name, preferences, profile — store them in your database and inject them via the system prompt rather than relying on them being in conversation history.

What's next

Memory keeps context across turns. But there's another dimension to agent design: what happens when a task is too complex to complete in one shot — when the agent needs to break a goal into sub-tasks, work through them over multiple steps, and adapt its plan when something doesn't work?

The next article covers structured agent patterns — how to design agents that plan, delegate, and recover from failures.

Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.

Top comments (0)