The Hard Part of AI Support Is Remembering What Failed

Ravi Kumawat — Sat, 07 Nov 2026 16:35:42 +0000

By Ravi Kumawat

A support agent can give a technically reasonable answer and still be unhelpful if it recommends the same fix a customer already tried. I built RECALL AI around a simple rule: a later response should account for the outcome of earlier troubleshooting, not just retrieve a transcript.

RECALL AI is a customer support system that carries useful context across conversations. It combines a React and TypeScript interface with a FastAPI service, Supabase for application records and conversation persistence, Groq for language-model responses, and Hindsight as a dedicated memory layer. The central engineering question was not how to store more chat text. It was how to preserve the difference between a suggestion, an attempted action, and a confirmed outcome—and make that distinction available before generating the next answer.

A transcript is not a support memory

Imagine a customer whose Bluetooth headphones disconnect every ten minutes. An agent suggests restarting Bluetooth. The customer tries it and reports that the disconnections continue. Later, the customer returns in a new conversation.

A transcript contains all the words, but the next agent still needs to infer several facts: the recurring symptom, the attempted restart, its unsuccessful outcome, and the fact that the issue remains open. Those facts have different operational meaning. Treating them as undifferentiated text makes it easy to recommend the same action again or to mistake an unconfirmed suggestion for a completed fix.

I designed RECALL AI to represent reported problems, attempted troubleshooting, failed attempts, successful fixes, preferences, confirmed resolutions, and unresolved issues as distinct memory. The distinction sounds small, but it determines whether memory can affect a support decision rather than merely make a conversation longer.

Where Hindsight fits

I use Hindsight, an open-source memory system on GitHub, as the long-term memory layer. Its retain, recall, and reflect operations provide a useful separation of concerns: the application decides what customer context to supply, while Hindsight manages the memory operations that make information available across sessions. The Hindsight documentation describes the system and its API, and Vectorize's overview of agent memory and how it works gives useful background on why a model's immediate context window is not a substitute for persistent memory.

The memory service uses Hindsight's asynchronous operations. Retention instructions classify information such as customer-reported issues, attempted steps, outcomes, preferences, and confirmed resolutions. That classification matters because “the agent suggested pairing again” is not equivalent to “the customer tried pairing again,” and neither statement means the issue was resolved.

I also wanted memory identifiers to be deterministic and scoped without putting raw customer IDs into bank names. RECALL AI derives an opaque identifier from the workspace, subject type, and canonical subject ID using HMAC-SHA256:

message = "\0".join(
    ("workspace", workspace_id, "subject", subject_type, canonical_id)
).encode("utf-8")

digest = hmac.new(
    self._bank_id_secret.encode("utf-8"),
    message,
    hashlib.sha256,
).hexdigest()

return f"{self.bank_prefix}-{digest}"

This gives the application a stable bank name for the same subject within its workspace while avoiding direct exposure of the customer identifier in that name. It is an identifier strategy, not authentication or access control; the application still has to verify identity and authorize access.

The Hindsight client is initialized from configured endpoint and credential settings, and the service can be unavailable when no API key is configured:

from hindsight_client import Hindsight
from app.core.config import Settings

def create_hindsight_client(settings: Settings) -> Hindsight | None:
    if not settings.hindsight_api_key:
        return None

    return Hindsight(
        base_url=str(settings.hindsight_base_url),
        api_key=settings.hindsight_api_key.get_secret_value(),
        timeout=20.0,
        max_attempts=1,
    )

I kept this boundary explicit. Conversation records belong to the application data layer; cross-session customer context belongs to the memory layer. That separation makes it easier to reason about what is persisted as an operational record and what is retrieved to inform a later response.

Recall before generation

The design decision with the biggest effect on response behavior is the ordering: retrieve memory before calling the language model. In SupportAgent.respond(), the service requests relevant memories using the verified workspace and subject identity together with the current message:

async with log_async_stage("hindsight_memory_recall", timing_correlation_id):
    recall_result = await self.memory_service.recall_memories(
        workspace_id,
        identity.subject_id,
        current_message,
        subject_type=subject_type,
    )

memory_context = self._memory_context(recall_result)

The application then checks provenance, identifies earlier failed actions, selects relevant memories, and bounds the context passed onward. Only after that preparation does it call Groq. Memory therefore participates in response generation; it is not attached after a response has already been composed.

This order also gives the application a place to apply domain rules. Retrieval alone does not guarantee a useful answer: a model can receive the right memory and still suggest an inappropriate step. The response path checks known failed-action constraints where applicable, then persists the conversation and retains relevant new information for later sessions.

At a high level, the workflow is:

Verify the customer's identity and persist conversation activity.
Recall relevant memories from Hindsight.
Prepare bounded context, prioritizing relevant history and troubleshooting provenance.
Generate a structured response with Groq.
Validate the response against known failed-action constraints.
Persist the exchange and retain relevant new information.

Supabase and Hindsight have different jobs here. Supabase handles application records and conversation persistence. Hindsight provides the memory operations that let a later interaction build on earlier outcomes.

What changes for the customer

The system's behavior is easiest to understand across three interactions. In the first, Sarah Mitchell, a synthetic customer used in the scenario, reports that her Bluetooth headphones disconnect about every ten minutes. Restarting Bluetooth has not helped, so the agent suggests forgetting the device and pairing it again.

In the second interaction, Sarah reports that forgetting and re-pairing also failed. The outcome is retained as a failed troubleshooting attempt. In a later session, she asks what different step she should try after the headphones disconnect again. The agent recalls the previous attempts and suggests a different diagnostic direction, including checking for headphone firmware and operating system updates.

The useful behavior is not that the system can quote an earlier conversation. It is that it can use the reported outcome to avoid treating the issue as new and avoid repeating a known unsuccessful step. In the support inbox, the operator can see the conversation alongside the retrieved memory results; the broader workspace also presents a reasoning graph for the demonstrated sequence of facts, failed attempt, retention rule, and next action.

What I learned building it

Memory quality depends on event distinctions. A recommendation is not an attempt, and an attempt is not an outcome. If the system collapses those states, later retrieval can confidently return the wrong operational conclusion.

Retrieval is necessary, but validation still matters. Persistent memory improves the context available to the model; it does not turn generation into a deterministic process. Application-level provenance checks and response validation provide a second line of defense for actions the application knows how to track.

Scope identifiers deliberately. Deterministic, opaque bank names help keep raw customer IDs out of memory identifiers, but they do not establish who may access a bank. Authentication and authorization remain separate responsibilities.

Keep the memory context bounded and relevant. Sending every prior message would recreate the same noise problem in a different form. The application selects memories associated with the current issue and prepares a bounded context before generation.

Treat confirmation as data. A customer-confirmed resolution should be represented differently from an agent's proposed fix. Future support decisions depend on that provenance, so the interaction should capture it explicitly.

Closing

RECALL AI is built around a narrow but consequential idea: support should remember not only what a customer said, but what the customer tried and what happened next. Hindsight provides the retain, recall, and reflect layer; the application adds identity scope, relevance selection, provenance checks, and response validation around it.

That architecture does not remove the need for careful support logic. It makes the history needed by that logic available at the right time—before the next answer is generated. For a customer who has already tried the obvious fix, that is the difference between another scripted conversation and a support interaction that actually moves forward.