How to Add Conversational Memory to Your Claude Chatbot

#python #ai #claude #tutorial

3 min read · 612 words

Building a stateless LLM chatbot is easy. You send a prompt, you get a response, and the model instantly forgets everything that just happened. But real user applications demand context. Users expect the AI to remember what they said two messages ago.

In this tutorial, you’ll learn how to add robust conversational memory to a Python chatbot using the Anthropic SDK. We'll use claude-opus-5 to manage multi-turn conversations cleanly without bloating your context window.

Why LLMs Need Application-Level Memory

The Anthropic API is stateless. Every call to client.messages.create() is a completely fresh interaction. To create the illusion of memory, your application code must maintain a history array of alternating user and assistant messages and send the full history with every new request.

The Basic Chat Loop with Memory

Let's start by implementing a simple CLI chatbot that keeps track of the conversation in a local Python list.

Make sure you have the latest SDK installed:

pip install anthropic

Here is a working implementation:

import os
import anthropic

# Initialize the Anthropic client
# Make sure ANTHROPIC_API_KEY is set in your environment variables
client = anthropic.Anthropic()

def run_chat_loop():
    # Initialize message history
    messages = []

    print("🤖 Claude Chatbot Initialized. Type 'exit' to quit.\n")

    while True:
        try:
            user_input = input("You: ").strip()
        except (KeyboardInterrupt, EOFError):
            break

        if user_input.lower() in ["exit", "quit"]:
            print("Goodbye!")
            break

        if not user_input:
            continue

        # Append the user's message to history
        messages.append({"role": "user", "content": user_input})

        try:
            # Send the complete conversation history to Claude
            response = client.messages.create(
                model="claude-opus-5",
                max_tokens=4000,
                messages=messages
            )

            # Extract assistant's reply
            assistant_reply = response.content[0].text
            print(f"\nClaude: {assistant_reply}\n")

            # Append the assistant's reply to history
            messages.append({"role": "assistant", "content": assistant_reply})

        except Exception as e:
            print(f"\n⚠️ Error: {e}\n")

if __name__ == "__main__":
    run_chat_loop()

Handling Long Conversations (Pruning & Summarization)

Models like claude-opus-5 feature a massive 1M token context window, meaning you can carry out exceptionally long sessions. However, letting history grow infinitely increases latency and token costs.

For production apps, you should implement a rolling window or summarize old turns. Here is how you can cap your history to the last $N$ turns to keep things fast and cost-effective:

import anthropic

client = anthropic.Anthropic()

def chat_with_rolling_window(messages, new_user_message, max_history_turns=10):
    # Append the latest user message
    messages.append({"role": "user", "content": new_user_message})

    # Keep system instructions separate if you use them, 
    # but for raw message history, slice the tail.
    # Each turn consists of 1 user + 1 assistant message (2 items).
    max_items = max_history_turns * 2
    if len(messages) > max_items:
        # Always ensure the history starts with a 'user' message
        trimmed_messages = messages[-max_items:]
        if trimmed_messages[0]["role"] == "assistant":
            trimmed_messages = trimmed_messages[1:]
    else:
        trimmed_messages = messages

    response = client.messages.create(
        model="claude-sonnet-5", # Great choice for high-volume chat tasks
        max_tokens=4000,
        messages=trimmed_messages
    )

    assistant_reply = response.content[0].text

    # Update original history reference
    messages.append({"role": "assistant", "content": assistant_reply})

    return assistant_reply, messages

Pro-Tips for Production Chatbots

Drop Temperature/Top-P: Modern configurations omit temperature, top_p, and top_k. If you need to steer style or determinism, rely entirely on clear system prompts.
Use the Right Model: Use claude-opus-5 for deep reasoning tasks, and switch to claude-sonnet-5 for high-throughput, latency-sensitive applications.
Structured Outputs: If your chatbot needs to collect user data (like booking details) into memory, enforce JSON schema responses using output_config={"format": {...}} instead of attempting assistant-turn prefilling.

Conclusion

Adding memory to your Claude chatbot is as straightforward as maintaining a clean array of turns and passing it back on every request. By combining a rolling window strategy with the blazing speed and massive context of current Claude models, you can build seamless conversational experiences that scale.

What kind of AI assistant are you building? Let me know in the comments below!

DEV Community