DEV Community

jamilxt
jamilxt

Posted on

Spring AI Is About to Deprecate ChatMemory - Here Is How to Move to the New Session API

#java #springboot #ai #springai

Three weeks ago I wrote about managing agent conversation memory in Spring AI with ChatMemory and MessageWindowChatMemory. It is the setup I run, and for a sliding window of the last N messages, it works. But there is a line in the Spring team's own Agentic Patterns series that tells you where this is going: the new Session API "targets Spring AI 2.1 (November 2026), when ChatMemory will be deprecated in its favour."

So the abstraction most Spring AI agents are built on today has an expiry date. Not because it is broken, but because it has a structural flaw nobody noticed until agents started making tool calls: when your window slides, it slides over individual messages, not conversations. If a tool call gets evicted while its tool result survives, the model receives an orphaned result with no idea what asked for it. The new Session API fixes exactly that, and this post is the migration guide I will be following myself: what changed, how to wire it, and the decisions that matter before you flip the switch.

One disclosure first: the Session API is incubating in the spring-ai-community organization, not yet in Spring AI itself. I have prototyped with it but my production agents still run ChatMemory. Treat this as a hands-on evaluation, not a battle report.

The Problem With Evicting Individual Messages

Start with how ChatMemory works. It stores a flat list of messages per conversation ID. When the list exceeds your window size, the oldest messages get evicted. Simple, cheap, and fine in a world where a conversation is just user says something, assistant replies.

Agents broke that assumption. A single user question can now produce a chain like this: user message, assistant tool call, tool result, another tool call, another tool result, final assistant answer. That is one logical exchange, six messages in the log. A window capped at 20 messages evicts the middle of that chain without knowing it is cutting an exchange in half. The Spring team describes the consequence bluntly: naive truncation "silently discards tool-call sequences mid-exchange, leaving the model with orphaned results and broken turn structure."

I hit a version of this with MessageWindowChatMemory tuning: crank the window down to save tokens and the agent starts hallucinating what its own tools returned, because the tool call is gone but the result is still in view. The only fix under ChatMemory is to keep the window big enough that you never cut a turn, which means paying for tokens you did not need, or engineering your own turn-aware eviction, which is exactly what the Spring team just did for everyone.

What the Session API Changes

The core shift is from a message list to an event log. A Session is an immutable, metadata-only value object: session ID, user ID, TTL, arbitrary metadata. The conversation itself lives in a separate repository as an ordered stream of SessionEvent objects. Each event wraps a Spring AI Message and adds the things a Message deliberately omits: a UUID, the session ID, a timestamp, an optional branch label, and framework flags.

The second shift is the turn as the atomic unit. A turn is one user message plus everything that follows it, assistant replies, tool calls, tool results, until the next user message. Every compaction strategy in the new API operates at turn granularity. The kept window always starts on a user message, and the model never sees a split exchange. This is the whole reason the API exists.

The third shift is compaction as a first-class concept. Instead of one eviction behavior, you get two composable abstractions: a trigger that decides when to shrink history, and a strategy that decides how. ChatMemory gave you none of this configurability.

Here is the API in its smallest form:

SessionService service = new DefaultSessionService(
        InMemorySessionRepository.builder().build());

Session session = service.create(
        CreateSessionRequest.builder().userId("alice").build());

service.appendMessage(session.id(), new UserMessage("What is Spring AI?"));
service.appendMessage(session.id(), new AssistantMessage("Spring AI is..."));

List<Message> history = service.getMessages(session.id()); // ready for an LLM
Enter fullscreen mode Exit fullscreen mode

Wiring It Into ChatClient

You rarely touch SessionService directly. The SessionMemoryAdvisor does for sessions what MessageChatMemoryAdvisor did for ChatMemory: on every request it loads the session history, prepends it to the prompt, appends the new user and assistant messages after the call, and runs compaction if the trigger fires. No manual history code in your application.

@Bean
SessionMemoryAdvisor sessionMemoryAdvisor(SessionService sessionService) {
    return SessionMemoryAdvisor.builder(sessionService)
            .defaultUserId("alice")
            .compactionTrigger(new TurnCountTrigger(20))
            .compactionStrategy(
                SlidingWindowCompactionStrategy.builder()
                    .maxEvents(20)
                    .build())
            .build();
}

@Bean
ChatClient chatClient(ChatClient.Builder builder, SessionMemoryAdvisor advisor) {
    return builder.defaultAdvisors(advisor).build();
}
Enter fullscreen mode Exit fullscreen mode

And at call time, you pass the session ID the same way you passed a conversation ID before:

String response = chatClient.prompt()
        .user("Hello!")
        .advisors(a -> a.param(
            SessionMemoryAdvisor.SESSION_ID_CONTEXT_KEY, "session-abc"))
        .call()
        .content();
Enter fullscreen mode Exit fullscreen mode

If no session exists for that ID, the advisor creates one automatically. The migration surface for basic usage is genuinely small: swap the advisor, swap the ID parameter, done.

One gotcha worth flagging: trigger and strategy must be configured together. Setting one without the other throws IllegalArgumentException at build time. Either set both, or omit both to disable compaction entirely. The library refuses to guess your intent here, which I appreciate after debugging one-too-many silently-defaulting configs.

Choosing a Trigger and a Strategy

Triggers answer "when does history shrink?" You get three building blocks:

  • TurnCountTrigger(20) fires when the session exceeds 20 turns
  • TokenCountTrigger with a threshold like 4000 fires on estimated tokens
  • CompositeCompactionTrigger.anyOf(...) fires when either condition is met, which is the realistic production setup: cap turns AND cap tokens

Strategies answer "what gets kept?" There are four:

  • SlidingWindowCompactionStrategy keeps a verbatim suffix of the last N events. No LLM call, essentially free. Best for cost-sensitive, short-term context.
  • TurnWindowCompactionStrategy keeps the last N complete turns. Also free. Best when turn structure matters more than raw message count.
  • TokenCountCompactionStrategy keeps events within a token budget. Also free. Best when you are pressed against a hard context-window limit.
  • RecursiveSummarizationCompactionStrategy is the interesting one: it uses an LLM to summarize the events being archived, stores the result as a synthetic user-plus-assistant turn, and each subsequent compaction builds on prior summaries, so you get a rolling compressed history that never starts from scratch. It costs an extra model call per compaction, so reach for it only when the agent genuinely needs recall of older context. 
RecursiveSummarizationCompactionStrategy.builder(chatClient)
        .maxEventsToKeep(10)
        .overlapSize(2) // feed 2 events from the active window into the summary
        .build();
Enter fullscreen mode Exit fullscreen mode

All four strategies snap the cut point to the nearest turn boundary. That constraint is not configurable, and that is the point.

The Part That Sold Me: Recall Storage

Here is the feature that made me stop treating this as a nicer ChatMemory. Compaction removes old events from the prompt, but not from the log. The full verbatim event history is always retained, and SessionEventTools exposes it to the model as a searchable tool, implementing the recall storage pattern from MemGPT:

ChatClient client = ChatClient.builder(chatModel)
        .defaultTools(SessionEventTools.builder(sessionService).build())
        .defaultAdvisors(advisor)
        .build();
Enter fullscreen mode Exit fullscreen mode

The model gets a conversation_search tool automatically. When it needs to recall a prior exchange that compaction pruned from its context, it calls the tool with a keyword and gets chronologically ordered results back. Synthetic summary events are searchable too.

That reframes the whole design. Under ChatMemory, your window size was a memory limit: evicted messages were gone. Under the Session API, the window is a prompt-efficiency decision, and memory is a tool call away. That is a much better mental model, and it maps to how the frontier coding agents already work.

Multi-Agent Branch Isolation

If you run an orchestrator that fans out to sub-agents, this one matters. All agents can share one Session, but each sees only its own events plus its ancestors'. Every SessionEvent carries a dot-separated branch path recording which agent produced it:

orchestrator  branch = "orch"
├── researcher  branch = "orch.researcher"
└── writer      branch = "orch.writer"
Enter fullscreen mode Exit fullscreen mode

Events with a null branch are root-level and visible to every agent. Isolation is applied through an event filter:

SessionMemoryAdvisor researcherAdvisor = SessionMemoryAdvisor.builder(sessionService)
        .defaultSessionId(sharedSessionId)
        .eventFilter(EventFilter.forBranch("orch.researcher"))
        .build();
Enter fullscreen mode Exit fullscreen mode

The researcher sees root events, orchestrator events, and its own. The writer's events stay hidden. A nice detail: compaction summaries always carry a null branch, so every agent in the session sees the shared history summary.

ChatMemory has no answer for any of this. One conversation ID per agent, N parallel stores, and you stitch it together yourself.

Production Persistence

The in-memory repository is for development. For production there is a JDBC module that stores sessions in two tables, AI_SESSION and an append-only AI_SESSION_EVENT log, with support for PostgreSQL, MySQL, MariaDB, and H2. The Spring Boot starter auto-configures everything:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>spring-ai-starter-session-jdbc</artifactId>
</dependency>
Enter fullscreen mode Exit fullscreen mode

For PostgreSQL or MySQL, enable schema initialization:

spring:
  ai:
    session:
      repository:
        jdbc:
          initialize-schema: always
Enter fullscreen mode Exit fullscreen mode

An append-only event log also means you get auditing for free, the same reason we run append-only audit trails elsewhere. Writes use optimistic compare-and-swap across all implementations, so concurrent turns on one session fail loudly instead of silently overwriting.

The Migration Cheat Sheet

If you are on MessageWindowChatMemory today, here is the direct equivalence from the Spring team's post:

// Before
MessageWindowChatMemory.builder().maxMessages(20).build();

// After
SessionMemoryAdvisor.builder(sessionService)
        .compactionTrigger(new TurnCountTrigger(20))
        .compactionStrategy(
            SlidingWindowCompactionStrategy.builder().maxEvents(20).build())
        .build();
Enter fullscreen mode Exit fullscreen mode

What you gain, item by item:

  • Storage unit: flat Message list becomes an immutable, timestamped, identified event log
  • Compaction: evict-oldest becomes four pluggable strategies, including LLM summarization
  • Turn safety: not enforced becomes guaranteed, every strategy snaps to turn boundaries
  • Multi-agent: unsupported becomes branch isolation with dot-separated labels
  • Recall: none becomes a keyword-searchable conversation_search tool over full history
  • Concurrency: implementation-dependent becomes optimistic CAS everywhere

Requirements to know before you start: Java 17+, Spring AI 2.0.0-M4 or later, Spring Boot 4.0.2 or later, and the community BOM (spring-ai-session-bom, version 0.2.0 at the time of the announcement, with 0.5.x releases already shipping per the Baeldung guide).

What I Would Do Differently, and When

My plan, and the checklist I am handing you:

  • Do not rip out ChatMemory this week. It is not deprecated yet, and the Session API is a community incubator. Shipping production memory on an 0.x library is a risk call, not a default.
  • Do prototype now if you have multi-agent fan-out or long sessions. Those are the two shapes that hit ChatMemory's flaws hardest, and the branch isolation plus recall storage alone justify the evaluation.
  • Pick the boring strategy first. Start with TurnCountTrigger plus SlidingWindowCompactionStrategy, measure token spend and failure modes, and only add RecursiveSummarizationCompactionStrategy when you can point at a concrete recall problem it solves. Summarization is an extra LLM call per compaction; it should earn its keep.
  • Set trigger and strategy together. The build-time IllegalArgumentException will remind you anyway, but save yourself the cycle.
  • Watch Spring AI 2.1 in November 2026. That is when ChatMemory deprecation lands and migration stops being optional. Starting on a community build now means your migration is a version bump instead of a project.

The uncomfortable summary: most of us taught our agents to remember by keeping a window of raw text, and that was always a approximation. The Session API replaces it with something that looks like what it actually is, an event-sourced log with an explicit compaction policy. If you build Spring agents, this is the migration to get ahead of.

I write about Java, Spring Boot, and AI every week. Subscribe - it's free.

Have you hit the orphaned tool-result problem with ChatMemory windows, or did you engineer your own turn-aware eviction? I would genuinely like to hear what you built, because a lot of us are about to throw that code away.

Top comments (0)