DEV Community

Cover image for RecallDesk: How Persistent Memory Turns Past Support Incidents into Reusable Solutions
Shivani Erlapally
Shivani Erlapally

Posted on

RecallDesk: How Persistent Memory Turns Past Support Incidents into Reusable Solutions

#react #python #ai #devops

RecallDesk: How Persistent Memory Turns Past Support Incidents into Reusable Solutions

When critical infrastructure fails, the fastest path to resolution often lies in the past. If an edge proxy rejects TLS connections after a certificate rotation, chances are a similar issue was diagnosed and resolved in staging months ago.

The tragedy of enterprise support is that this institutional knowledge rarely reaches the engineer handling the new ticket. It sits buried in closed tickets or chat logs. The customer must re-explain their architecture, and the support specialist spends hours re-diagnosing a solved problem.

In building RecallDesk, we set out to make historical support experience persistent, context-aware, and immediately reusable. Using Hindsight, an open-source persistent memory system for AI agents, RecallDesk surfaces relevant troubleshooting history the moment a ticket is opened.

Hindsight recall implementation

This article details how RecallDesk's memory retrieval and reuse pipeline works: how memories are scoped and queried, how vector facts become actionable suggestions, and how human review checks and approves proposed solutions before anything reaches the customer.

The Retrieval Problem

Support memory differs fundamentally from standard keyword search:

  • Vocabulary Drift: A customer reporting "our ingress gateway is throwing SSL_ERROR_UNKNOWN_CA_ALERT" uses different phrasing than an engineer noting "patched Vault agent ConfigMap from cert.pem to fullchain.pem". Keyword searches frequently miss these connections.
  • Context Fragmentation: Resolutions are scattered across initial incident logs, back-and-forth debugging, dead-end attempts, and final confirmations.
  • Relevance vs. Noise: Showing an engineer raw transcripts creates cognitive overload. The workspace must isolate the specific remedy that succeeded and the dead-ends to avoid.

Where Hindsight Fits

RecallDesk dashboard

RecallDesk delegates persistent memory storage and semantic recall to Hindsight. Rather than storing flat text in a generic vector table, Hindsight manages memories in a dedicated memory bank (recalldesk-support).

sequenceDiagram
    autonumber
    actor Engineer as Support Specialist
    participant UI as RecallDesk UI (React)
    participant API as FastAPI Backend
    participant Service as Hindsight Service
    participant Bank as Hindsight (recalldesk-support)

    Engineer->>UI: Select or Create Ticket (#conv_101)
    UI->>API: GET /conversations/{id} or POST /conversations
    API->>Service: recall_customer_memories(customer_id, query)
    Service->>Bank: arecall(query, tags=['customer:cust_001'], tags_match='any')
    Bank-->>Service: RecallResponse (facts, types, scores)
    Service-->>API: Recalled memory items
    API-->>UI: Conversation payload with recalled_memories
    UI->>UI: Categorize (What Worked / What Failed) & Propose Solution
    Engineer->>UI: Click "Use recalled solution" -> Human Review & Send
    Engineer->>UI: Click "Resolve & Retain"
    UI->>API: PATCH /conversations/{id}/status (resolved)
    API->>Service: retain_customer_conversation(...)
    Service->>Bank: aretain(content, doc_id="cust_001_conv_101", tags=[...])

When a ticket is resolved, RecallDesk retains a structured interaction record. Hindsight processes the retained record into searchable memory and returns relevant memories using its retrieval and ranking pipeline.

Building the Recall Query

Retrieval begins in backend/app/api/routes/conversations.py. When an engineer opens an existing ticket or submits a new one, the backend constructs a query from the subject line and customer problem:

query = conversation.subject
if conversation.messages:
    customer_msgs = [m for m in conversation.messages if m.sender_type == "customer"]
    if customer_msgs:
        query = f"{conversation.subject} - {customer_msgs[-1].text}"
Enter fullscreen mode Exit fullscreen mode

Before dispatching the query, RecallDesk scrubs sensitive tokens using sanitize_content() in backend/app/services/hindsight_service.py so credentials never enter the recall pipeline.

Customer-Scoped Retrieval

RecallDesk executes memory recall through HindsightMemoryService.recall_customer_memories():

recall_res: RecallResponse = await asyncio.wait_for(
    self._client.arecall(
        bank_id=self.bank_id,
        query=sanitized_query,
        tags=[f"customer:{customer_id}"],
        tags_match="any",
        max_tokens=max_tokens,
        budget=budget
    ),
    timeout=8.0
)
Enter fullscreen mode Exit fullscreen mode

Key configuration parameters govern this operation:

  • bank_id: Targets the shared memory bank (recalldesk-support).
  • tags & tags_match="any": Scopes retrieval to memories carrying the customer tag (e.g. customer:cust_001). This associates memories with Elena Rostova's account (Acme Cloud Infrastructure). Note that customer tags serve as an organizational query filter within the shared bank, not a strict security or tenant-isolation boundary.
  • max_tokens=2048 & budget="mid": Allocates token budget so Hindsight retrieves and ranks relevant memories.
  • Application Guardrail: Wrapped in asyncio.wait_for(..., timeout=8.0). If Hindsight experiences network latency, RecallDesk times out gracefully, logging a warning and returning an empty list so ticketing continues without blocking the user.

What Hindsight Returns

Hindsight returns a typed RecallResponse containing a list of memory objects:

  • id: Unique UUID assigned to the memory fact.
  • text: Synthesized natural-language fact (e.g., "Elena Rostova initiated an automated certificate rotation which caused mTLS failures due to Vault agent pointing to cert.pem instead of fullchain.pem").
  • type: Hindsight memory classification (fact, observation, or world).
  • document_id: Identifier of the source document (cust_001_conv_101).
  • tags: Metadata tags (customer:cust_001, status:resolved, tier:enterprise).
  • scores: Scoring breakdown including semantic, keyword, and ranking metrics.

FastAPI converts these items into Pydantic RecalledMemoryItem models and embeds them inside the Conversation response returned to the React frontend.

Turning Memories into Support Context

Hindsight delivers vector-scored factual statements, but raw lists do not tell an engineer how to resolve an incident. The frontend bridges this gap deterministically in frontend/src/components/CustomerContextPanel.jsx.

The client categorizes memories using keyword heuristics:

const whatWorkedItems = recalledMemories.filter(m => {
  const t = (m.text || '').toLowerCase();
  return t.includes('resolved') || t.includes('what worked') || t.includes('worked') ||
         t.includes('solution') || t.includes('fixed') || t.includes('patch') ||
         t.includes('fullchain.pem') || t.includes('succeeded');
});

const whatFailedItems = recalledMemories.filter(m => {
  const t = (m.text || '').toLowerCase();
  return t.includes('failed') || t.includes('what failed') || t.includes('dead-end') ||
         t.includes('error') || t.includes('reject') || t.includes('instead of') ||
         t.includes('omits') || t.includes('missing') || t.includes('alert');
});
Enter fullscreen mode Exit fullscreen mode
  • What Worked: Highlighted in emerald cards labeled Previously Successful Solution, showing past fixes recorded in memory with an option to reuse the solution.
  • What Failed: Displayed in rose cards labeled Known Dead-End (Avoid), warning engineers against repeating past missteps.
  • Relevant Memories: Renders the complete fact stream with memory types and document IDs.

From Memory to Suggested Solution

RecallDesk Memory Hub

When a recalled memory contains an actionable fix, ConversationView.jsx detects the top solution and displays a suggested solution banner:

const topSolutionMemory = hasRecalledMemories
  ? conversation.recalled_memories.find(m => {
      const t = (m.text || '').toLowerCase();
      return t.includes('resolved') || t.includes('fullchain.pem') ||
             t.includes('solution') || t.includes('worked') || t.includes('patch');
    })
  : null;
Enter fullscreen mode Exit fullscreen mode

Clicking "Use recalled solution" formats the memory into an actionable draft:

const formatSuggestedSolution = (memText) => {
  const textLower = (memText || '').toLowerCase();
  if (textLower.includes('fullchain.pem') || textLower.includes('envoy')) {
    return "Based on the previous certificate rotation issue, please verify that the Envoy secret mount references the fullchain.pem file rather than the standalone certificate.";
  }
  return `Based on previous customer experience: ${memText}`;
};
Enter fullscreen mode Exit fullscreen mode

This deterministic approach produces predictable guidance tailored to previously recorded support patterns rather than ungrounded LLM completions.

Human-in-the-Loop Review

The suggested solution does not send autonomously. Clicking "Use recalled solution" populates the active draft in the message composer under an explicit header:

Human Review & Edit: Verify solution before sending
Enter fullscreen mode Exit fullscreen mode

The support specialist reads the drafted message, verifies technical parameters against the customer's active environment, makes adjustments if needed, and clicks "Send Reply".

Memory accelerates the human engineer's diagnosis while preserving human judgment and accountability.

The Second Ticket: Testing Persistent Memory

The primary value of persistent memory is proven when a customer encounters a recurring problem:

  1. Ticket #conv_101: Elena Rostova reported an mTLS handshake error (SSL_ERROR_UNKNOWN_CA_ALERT) following automated certificate rotation. Support determined that Vault was mounting cert.pem instead of fullchain.pem. The ticket was resolved and retained.
  2. Ticket #conv_105: Hours later, Elena creates a new ticket:
    • Subject: "Production ingress rejecting client TLS certificates after rotation"
    • Message: "Our staging gateway is now throwing SSL_ERROR_UNKNOWN_CA_ALERT again after the automated rotation."
  3. Immediate Recall: In conversations.py, create_conversation() queries Hindsight using the new subject and initial message, filtered by customer:cust_001.
  4. Immediate Reuse: Hindsight matches the semantic concepts (SSL_ERROR_UNKNOWN_CA_ALERT, rotation) and retrieves the previous fullchain.pem experience as a relevant memory.
  5. Surfacing Previous Experience: Opening #conv_105 immediately displays the Suggested Solution from Previous Experience banner referencing fullchain.pem. The previous resolution is surfaced immediately for human review, allowing the engineer to inspect, edit, and send the response without re-investigating from scratch.

Retaining the New Experience

When the engineer clicks "Resolve & Retain", RecallDesk commits the resolution back to Hindsight using retain_customer_conversation():

clean_cust_id = customer_id[5:] if customer_id.startswith("cust_") else customer_id
clean_conv_id = conversation_id[5:] if conversation_id.startswith("conv_") else conversation_id
document_id = f"cust_{clean_cust_id}_conv_{clean_conv_id}"

tags = [
    f"customer:{customer_id}",
    f"conv:{conversation_id}",
    f"status:{status or 'open'}",
    f"tier:{tier.lower()}"
]
Enter fullscreen mode Exit fullscreen mode
  • Deterministic Document ID: Formatting IDs as cust_{id}_conv_{id} provides a deterministic document identifier that helps avoid duplicate document IDs when the same conversation is retained again. Status updates overwrite the existing document instead of polluting the bank with duplicates.
  • Structured Record: The retained payload organizes customer metadata, initial symptoms, recorded root-cause fixes, and chronological dialogue lines for semantic indexing.

Sanitization and Failure Handling

Support logs often contain credentials. Before content reaches Hindsight for retention or recall, sanitize_content() scrubs sensitive tokens via regular expressions:

SENSITIVE_PATTERNS = [
    (re.compile(r'(?i)(?:password|passwd|pwd|secret)\s*[:=]\s*([^\s\'";,]+)'), r'password=[REDACTED_SECRET]'),
    (re.compile(r'(?i)\bbearer\s+[a-zA-Z0-9_\-\.]{20,}\b'), r'[REDACTED_BEARER_TOKEN]'),
    (re.compile(r'(?i)(?:api[_-]?key|client[_-]?secret)\s*[:=]\s*([a-zA-Z0-9_\-]{16,})'), r'api_key=[REDACTED_API_KEY]'),
    (re.compile(r'\b(?:\d{4}[ -]?){3}\d{4}\b'), r'[REDACTED_CARD_NUMBER]'),
    (re.compile(r'(?i)\b(?:otp|one[- ]time code|pin|verification code)\s*[:=]?\s*\d{4,8}\b'), r'[REDACTED_OTP]'),
    (re.compile(r'-----BEGIN [A-Z ]+ PRIVATE KEY-----[\s\S]*?-----END [A-Z ]+ PRIVATE KEY-----'), r'[REDACTED_PRIVATE_KEY]')
]
Enter fullscreen mode Exit fullscreen mode

If the Hindsight API is unreachable, the backend catches the exception and returns an empty memory list, allowing core ticketing to function uninterrupted.

Current Limitations

Operating this pipeline highlighted several architectural boundaries:

  1. Shared Bank Scope: RecallDesk uses a single shared memory bank (recalldesk-support), relying on customer:{id} tags for scoping. Customer tags are an organizational index, not a cryptographic multi-tenant security boundary.
  2. Heuristic UI Categorization: Separating memories into "What Worked" and "What Failed" relies on string keyword matching rather than an autonomous classifier. While fast, memories with unconventional phrasing may remain in the general list.
  3. Historical Experience vs. Infrastructure Ground Truth: Recalled memories reflect what support teams documented during prior incidents. If an earlier ticket was resolved with an incorrect note, Hindsight will recall that note. Memory assists human engineers; it does not replace verification.

Engineering Lessons

  1. Memory Must Be Actionable: Storing embeddings is straightforward; presenting them usefully is difficult. Raw vector dumps cause distraction. Structuring memories into "What Worked" and editable drafts creates immediate value.
  2. Enforce Deterministic IDs: Predictable document IDs (cust_001_conv_101) eliminate duplicate records and enable safe re-retention when ticket statuses change.
  3. Keep the Human in the Loop: Full automation in mission-critical support carries high risk. Providing pre-filled suggestions for human review preserves reliability while significantly reducing resolution time.

Conclusion

By integrating Hindsight's persistent memory into RecallDesk's triage workflow, support teams transform static ticket archives into active organizational memory. When Elena Rostova's second certificate issue arrived, RecallDesk surfaced the previous resolution so the support specialist could review and reuse that experience instead of starting from scratch.

Resources

Top comments (0)