I tested my memory-powered agent on the same customer twice — the difference surprised me

Mythri Gaddam — Tue, 29 Sep 2026 13:35:56 +0000

Most support bots have total amnesia. A customer can write twice in one week about the same order, and the bot asks them to explain it from scratch both times.

I built SupportMemory to fix that one thing. Not a fancier UI, not more integrations. Just: does the agent remember you?

What the system does

SupportMemory is a customer support agent for an e-commerce brand. It uses Groq running openai/gpt-oss-120b for generation and Hindsight as the memory layer. Each customer gets their own memory bank, seeded with sample order history, past tickets, and stated preferences.

Every message goes through the same loop:

Recall what Hindsight knows about this customer that relates to the current message.
Apply policy separately from memory, so the agent knows what it may and may not promise.
Generate a reply with both in context.
Retain a factual summary of the interaction.

Hindsight's agent memory handles storage and retrieval, so I didn't build an embedding pipeline or a vector store myself. My job was to call it in the right order and be careful about what I put in.

The test

I seeded a customer, Ananya, with four memories: a Bosch mixer grinder order, a cracked jar reported in August, a free replacement jar shipped two days later, and a note that she runs a home bakery and prefers replacements to refunds. She also has a second, more recent order for mixing bowls.

Then I opened a fresh session and typed the vaguest message I could:

Hi, I have an issue with my order.

A stateless bot asks for an order number. The agent instead replied:

I see you recently received the Bosch Series 6 mixer grinder (order #ORD-88213) and the stainless-steel mixing bowls (order #ORD-91765). You also experienced a cracked jar lid on the mixer grinder earlier, which we helped resolve. Could you let me know which order you're referring to now?

I hadn't given it any of that. With two orders on file, it asked which one, which is a better question than "what's your order number?"

Then I replied that the mixer grinder jar had arrived damaged again. The agent acknowledged the August replacement and the bakery, and asked for a photo of the damage and a delivery address "to help determine the appropriate next step." It didn't make her retell the story, and it didn't promise a replacement. That second part turned out to be the harder problem.

The part that took real work: memory is not authorization

My first version stored the whole exchange back into memory, including the agent's own reply. That's dangerous. If the agent ever says "I'll open a replacement request," that sentence becomes a memory. Next week, recall returns it as if it were a fact the company had actually committed to.

I saw this happen directly: an early run had the agent say a replacement was already being arranged, based only on a previous AI-generated reply that had said something similar. Nothing had actually been approved. That's the exact failure mode you don't want in a system a real business would run.

The fix has two parts. First, I keep support policy in a block of the prompt separate from memory. It lists what the agent may do (ask for a photo, confirm an address, explain next steps) and what it must not promise without a verified record (replacements, refunds, shipping, compensation). The prompt states it directly: memory is customer context, not authorization. Second, I stopped retaining the agent's reply at all:


python
retained_experience = (
    f"Customer {customer_name} ({customer_id}) reported: "
    f"\"{message}\". "
    f"Support interaction completed for this request. "
    f"No replacement, refund, shipping, compensation, or other "
    f"operational action should be treated as confirmed unless "
    f"separately verified."
)

hindsight.retain(bank_id=bank_id, content=retained_experience)
Only the customer's side and an explicit "nothing is confirmed" note go into memory.

Limitations
Policy is a hard-coded prompt block, not a live connection to an order or refund system. A production version would check real records before letting the agent promise anything.
Sample data: The customers and orders are sample data I wrote. I haven't tested this at scale or with real support volume.
Manual testing: I tested a small number of conversations by hand, not a benchmark, so I can't claim numbers.
Lessons learned
Scope memory per customer. One bank per customer removes any risk of cross-customer leakage and keeps recall focused.
Retain facts, not the agent's own words. Otherwise a hallucinated promise becomes a permanent memory.
Separate context from authority. What the customer history says and what the agent is allowed to do are different questions. Keep them in different places in the prompt.
Recall changes what you don't have to ask. The win isn't that the agent knows more. It's that the customer answers fewer questions before getting help.
The retain/recall loop is small, maybe 20 lines. Deciding what belongs in memory took most of my time. If you want to go deeper, the [Hindsight docs](https://hindsight.vectorize.io/) cover memory banks and reflection, which I haven't used yet.

DEV Community: Mythri Gaddam

I tested my memory-powered agent on the same customer twice — the difference surprised me

What the system does

The test

The part that took real work: memory is not authorization