DEV Community

Cover image for What I Learned Replacing Flat RAG with Dual-Bank Memory
Arpitha M V
Arpitha M V

Posted on

What I Learned Replacing Flat RAG with Dual-Bank Memory

#agents #llm #rag #vectordatabase

Why I Replaced Flat Support RAG with Dual-Bank Agent Memory Using Hindsight

Enterprise customer support systems fail for two predictable reasons: session amnesia and isolated silos. When a customer opens a ticket, standard stateless LLM wrappers have no memory of the user's operating system, subscription tier, or the troubleshooting steps they already ran forty-eight hours ago. Worse, when companies bolt on a naive per-user vector database, they create a secondary failure mode: if Customer A spends an hour helping engineering discover a workaround for a critical PDF export bug, the support agent talking to Customer B five minutes later has zero visibility into that resolution.

While building our stateful support engine (hindsight-customer-support-agent), I wanted to eliminate both failure modes without stuffing raw chat transcripts into a 128k context window. By pairing Vectorize agent memory with DeepSeek via OpenRouter, we designed a Dual-Bank Hierarchical Memory Architecture that separates private customer state from collective product learnings.

How the Dual-Bank System Hangs Together

Instead of treating memory as a single flat index, our core orchestrator in agent.py routes every incoming ticket through two distinct namespaces hosted on Hindsight Cloud (api.hindsight.vectorize.io) via the official hindsight-client SDK (v1.2.0):

  1. The Personal Episodic Bank (cust_001, cust_002, cust_003): Stores tenant-isolated facts—operating system, subscription tier, billing history, and explicit communication preferences (for example, whether the customer wants raw Linux CLI commands or beginner-friendly GUI steps).
  2. The Shared Hive-Mind Bank (global_product_learnings): Stores PII-stripped technical bug resolutions and product workarounds discovered across all support conversations.

When a ticket arrives, the system executes a deterministic Recall -> Reason -> Retain cognitive loop. You can explore the underlying memory primitives in the Hindsight GitHub repository.

Code Walkthrough: Dual-Bank Recall and Selective Retention

In agent.py, our generate_support_reply function queries both the customer's personal memory bank and the global learnings bank before constructing the system prompt for DeepSeek (deepseek/deepseek-v3.2):

import time
from datetime import datetime, timezone
from hindsight_client import Hindsight

hindsight = Hindsight(
    api_key=HINDSIGHT_API_KEY,
    base_url="[https://api.hindsight.vectorize.io](https://api.hindsight.vectorize.io)"
)

def recall_dual_bank_context(customer_id: str, query: str) -> tuple[str, str, int]:
    t0 = time.perf_counter()

    # 1. Recall private customer context (OS, tier, tone rules, past tickets)
    personal_resp = hindsight.recall(bank_id=customer_id, query=query)
    personal_context, _ = parse_hindsight_recall(personal_resp)

    # 2. Recall cross-customer technical bug workarounds
    global_resp = hindsight.recall(bank_id="global_product_learnings", query=query)
    global_context, _ = parse_hindsight_recall(global_resp)

    recall_ms = int((time.perf_counter() - t0) * 1000)
    return personal_context, global_context, recall_ms
Enter fullscreen mode Exit fullscreen mode

Once both memory blocks are retrieved, we inject them into distinct sections of the system prompt. This prevents cross-tenant data leakage while giving the reasoning model immediate access to verified product fixes:

system_prompt = f"""You are a Senior Technical Support Engineer.
Apply the verified longitudinal memory below without asking the customer to repeat known facts.

=== PERSONAL CUSTOMER MEMORY (Bank: {customer_id}) ===
{personal_context}

=== SHARED PRODUCT BUG LEARNINGS (Bank: global_product_learnings) ===
{global_context}

OPERATING RULES:
1. Match the customer's exact OS, plan tier, and communication style.
2. Skip Tier-1 checklists if prior tickets show those steps were already attempted.
3. Apply verified workarounds from global_product_learnings when relevant."""
Enter fullscreen mode Exit fullscreen mode

After the LLM generates a resolution, we don't blindly dump noisy greetings back into storage. We commit the interaction to the customer's personal bank with temporal metadata, and if a reusable technical workaround was identified, we commit a sanitized summary to global_product_learnings:

interaction_record = f"Customer ({customer_id}) asked: {user_message} | Agent resolved: {agent_reply}"
hindsight.retain(
    bank_id=customer_id,
    content=interaction_record,
    context="support_interaction",
    timestamp=datetime.now(timezone.utc).isoformat()
)
Enter fullscreen mode Exit fullscreen mode

Full details on configuring bank namespaces and retention parameters are available in the Hindsight documentation.

Concrete Behavior: Before vs. After Dual-Bank Memory

To see the architecture in action, consider two seeded customer profiles in our system:

  • Alex (cust_001): Senior Developer on Ubuntu Linux (Enterprise Tier). Hates generic troubleshooting checklists; wants direct bash commands. Previously resolved a PDF export crash caused by ad-blockers intercepting blob URLs.
  • David (cust_003): Small business owner on macOS (Starter Plan). Overwhelmed by technical jargon; needs numbered, plain-English UI instructions.

With Hindsight Memory Disabled (Stateless Mode):
When Alex messages "The export issue is happening again. How do I fix it?", the stateless model responds with a generic six-step checklist asking what operating system and browser he uses.

With Hindsight Dual-Bank Memory Enabled:

  1. Querying cust_001 recalls that Alex runs Ubuntu, is on the Enterprise tier, and previously hit the ad-blocker PDF export bug. The agent immediately replies with the CLI flag and Brave/uBlock blob whitelist command—zero clarifying questions asked.
  2. When Alex replies with a newly discovered fix ("Disabling hardware acceleration in v2.5 fixed the canvas renderer crash"), the agent retains that sanitized technical fact into global_product_learnings.
  3. Two minutes later, when David (cust_003) reports that his canvas is crashing, the agent recalls David's non-technical macOS persona from cust_003 alongside Alex's hardware acceleration fix from global_product_learnings—translating a developer's workaround into three gentle, click-by-click macOS steps for David.

Engineering Takeaways

  1. Separate Episodic Identity from Semantic Product Knowledge: Isolating cust_00X from global_product_learnings gives you strict multi-tenant privacy while allowing every customer ticket to make the entire support system smarter.
  2. Latency Budgets Matter: Decoupling memory retrieval via hindsight-client keeps prompt tokens compact and predictable compared to stuffing raw ticket histories into the context window.
  3. AI-Assisted Iteration: Using terminal-first agentic workflows and tools like Code.in alongside tight validation scripts (test_agent.py) allowed us to refactor our single-bank prototype into a dual-bank architecture without breaking existing recall assertions.

Top comments (0)