Chat memory that compacts by turns, so your agent never loses half a tool call

Cross-posted from the ByteChef blog. Part 7 of the AI Agentic Patterns in ByteChef series.

Every chat agent has the same slow leak. Each message grows the history, and the whole history goes into the next prompt. Long chats get slower, cost more and eventually hit the model's context limit.

The usual fix is "keep the last N messages". It works fine until the agent uses tools.

The problem with "last N messages"

A tool call and its result are two separate messages. Trim the history between them and the model gets one without the other: a result with no call, or a call with no result.

user:       What's the status of order 4512?
assistant:  [tool call] getOrder(4512)      <- window cuts here
tool:       { "status": "shipped" }         <- this survives
assistant:  It shipped yesterday.

Nothing throws. The agent just starts answering strangely, and you spend an afternoon figuring out why.

Chat Memory v2 in ByteChef fixes this by compacting the conversation in whole turns instead of loose messages.

Turns, not messages

Chat Memory v2 is built on the Spring AI community Session library. It stores a conversation as an ordered log of events. Each user message, assistant reply, tool call and tool result is one event.

A turn is one user message plus everything that follows it (replies, tool calls, tool results) up to the next user message. Compaction only ever removes whole turns. A tool call and its result stay together or leave together.

With the Turn window strategy:

• When to compact: as soon as the history has more turns than Max turns.
• How to compact: archive the oldest turns until only Max turns remain. The system prompt is always kept.

Archived, not deleted

Compacted events leave the prompt, but they stay in storage.

Turn on Enable conversation search tool and the agent gets a conversation_search tool. It passes a keyword, and the tool searches the whole conversation, including archived events that are still kept. Results come back in pages (Search page size, 10 by default).

So the agent's view is short, but its reach is long.

Two limits to know:

• Summaries are not searched. If you use Recursive summarization (below), the summaries the model writes don't show up in search. The original messages do, while they're kept.
• The archive is capped on some storage. Redis and S3 keep the 1000 most recent archived events. The built-in Chat Memory keeps the number set by bytechef.ai.memory.session-max-archived-events.

Try it: the template

The fastest way to see this is the Chat that compacts its history template.

Three steps:

1. New Chat Request - a message arrives in the workflow chat.
2. AI Agent - Claude Sonnet as the model, Chat Memory (version 2) in its Memory slot.
3. Reply - the answer goes back to the chat.

Chat Memory is configured like this:

Field	Value
Conversation ID	${chat_1.conversationId}
Compaction Strategy	Turn window (by turns)
Max turns	3
Enable conversation search tool	on

The system prompt tells the agent what to expect:

Your visible history is short. If the user asks about something said earlier that you no longer see, use the conversation search tool to find it, and say that you searched.

Three turns is far too small for real use. It's set that low so compaction shows up after a few messages.

To run it:

1. Import Chat that compacts its history and add it to a project.
2. Add your Anthropic connection to both models: the agent's and the one on Chat Memory. That's the only setup.
3. Send four or five messages on different topics. Put something specific in the first one, like a name or a number.
4. Ask the agent what you said at the start.

By then the first turn is archived. The agent can't see it, so it calls conversation_search and tells you it searched.

That's a real test run. The code name was in the first message, which had already left the three-turn window, so the agent searched the archive to answer.

To watch it happen, open the Agent Playbook in the editor. Its tool-call cards show conversation_search running, with the keyword the agent picked and the messages it found.

Past the demo, raise Max turns to 10-20, or switch to Token count or Recursive summarization. The search tool works the same with any strategy.

Why there's a second model

Look at the template again. The Chat Memory element has its own model, Claude Haiku, separate from the agent's Claude Sonnet.

Only Recursive summarization uses it. With Turn window, old turns are archived and no model is called. Haiku is there so you can flip the dropdown to Recursive summarization without adding anything, and summaries come from a smaller, cheaper model than the one talking to your users.

Pick Recursive summarization with no model on Chat Memory and the run fails with a clear error.

Four ways to compact

Compaction Strategy is a dropdown. Each option shows only the fields it needs.

Strategy	Fields	What it does
None	-	Never compacts. The default.
Sliding window (by events)	Max events	Keeps the most recent Max events once the history exceeds it.
Turn window (by turns)	Max turns	Keeps the most recent Max turns once the history exceeds it.
Token count	Max tokens	Keeps the history within an estimated token budget.
Recursive summarization (LLM)	Max events, Max events to keep, Overlap size	Over Max events, a model summarizes older events and the most recent Max events to keep stay in full.

For Recursive summarization, ByteChef checks two rules when the run starts:

• Max events must be greater than Max events to keep.
• Overlap size must be less than Max events to keep.

Overlap size is how many of the kept events the summarizer also sees, so the summary connects to what comes after it.

Where the history lives

Chat Memory v2 ships as several components with the same fields. Only the storage differs:

• Chat Memory - built-in. Storage follows bytechef.ai.memory.provider: JDBC, Redis, AWS S3 or in-memory.
• JDBC Chat Memory - a relational database.
• Redis Chat Memory - Redis, with its own connection.
• AWS S3 Chat Memory - JSON objects in an S3 bucket, with its own connection.
• In Memory Chat Memory - server memory, gone after a restart.

The template uses the built-in one, so it needs no extra connection.

The S3 backend is ByteChef's own implementation of the Session library's storage interface. Each session is one JSON object, and writes use S3 conditional writes (If-Match on the ETag), so two writers can't silently overwrite each other's events.

What Spring AI gives you, and what ByteChef adds

The compaction strategies, triggers and conversation_search come from the Spring AI community Session library. In plain Java you build the memory advisor, pick a trigger and a strategy, and register the search tool yourself. ByteChef runs the library directly and adds:

• Fields instead of code. One dropdown, a few numbers and a toggle.
• A model picker for summaries. The summarizer is a child element on Chat Memory, so it can be any model ByteChef supports.
• Clear errors. Invalid Recursive summarization settings, or a missing summarizer model, stop the run with a message that names the problem.
• Storage choices. JDBC, Redis, in-memory and ByteChef's own S3 backend, as separate components with the same fields.
• A search tool Claude will actually call. The library's conversation_search also asks the model for a private "inner monologue", and Claude refuses the whole turn rather than fill it in. ByteChef registers the tool with only a keyword and a page number.

Short-term vs long-term memory

Chat Memory keeps one conversation coherent. It doesn't carry facts into the next one. For that, ByteChef has the Auto Memory Tool: the agent writes down what's worth keeping and reads it back in later chats. It works in deployed runs, not editor test runs.

Both can sit on the same agent: Chat Memory in the Memory slot, the Auto Memory Tool in the Tools slot.

Same rule as Part 1 of the series: load each kind of knowledge only when it's needed. Skills load procedures on demand. Compaction keeps old turns out of the prompt until the agent searches for them.

Wrapping up

Dropping old messages one at a time can cut a tool call in half. Chat Memory v2 compacts by whole turns, archives what it removes and lets the agent search the archive. You pick the strategy and the storage on the canvas.

Try it: import Chat that compacts its history, connect Anthropic, send five messages and ask what you said first.

ByteChef is open source (Apache 2.0) and built on Java and Spring. If this was useful, a star on GitHub helps a lot.