The Missing Feedback Loop Behind Our Support Agent

The Missing Feedback Loop Behind Our Support Agent

An AI support agent can produce a convincing answer and still fail at one important job: learning from what actually happened.

That was the problem behind ResolveIQ. A support agent could analyze a customer issue and generate a recommendation, but a recommendation alone does not create useful experience. If an action succeeds, the system should be able to use that experience later. If it fails, that outcome should matter too.

We built ResolveIQ around that feedback loop: recall relevant support experiences, use them to ground a new recommendation, record the real outcome, and retain that experience for future cases.

The memory layer is powered by Hindsight by Vectorize.

The Problem: An Agent That Could Answer but Couldn't Learn

The first distinction we made was between context and experience.

A language model can reason over information in its current prompt. It can also produce a plausible support response without knowing whether a similar recommendation worked before.

Imagine a customer reports a 504 Gateway Timeout during a large analytics export. A model can suggest increasing a timeout, reducing the export size, checking infrastructure, or retrying the request. Those suggestions may be reasonable, but the model does not automatically know which approach worked for a previous customer with the same problem.

For support, that historical evidence matters.

ResolveIQ therefore treats previous support cases as usable experience rather than simply keeping them as an archive.

How ResolveIQ Works

ResolveIQ uses a React/Vite frontend and a Python/FastAPI backend.

The core workflow is:

Customer Issue
↓
Hindsight Recall
↓
Historical Support Context
↓
Groq Recommendation
↓
Action Taken
↓
Verified Outcome
↓
Hindsight Retain
↓
Future Recall

The frontend sends a customer issue to the backend resolution workflow. The backend recalls relevant experiences from Hindsight and provides that context to the recommendation agent.

After the support worker applies the recommendation, the outcome is recorded through the outcome workflow.

That second step is what closes the loop.

[Insert Screenshot 1: ResolveIQ main interface]

Adding Hindsight as the Memory Layer

We used the Hindsight Python client directly in the backend. The recall operation is intentionally small:

with Hindsight(base_url=base_url, api_key=api_key) as client:
response = client.recall(
bank_id=target_bank,
query=query
)

The returned memories are normalized into a predictable structure before being passed into the rest of the application.

This separation keeps memory operations inside the memory layer while allowing the resolution workflow to decide how retrieved experience should influence a new support case.

The memory bank is configurable through HINDSIGHT_BANK_ID, with resolveiq-support as the default.

For more background, see Hindsight documentation and Vectorize's agent memory guide.

Recall Before Reasoning

The most important part of the workflow happens before the LLM generates a recommendation.

ResolveIQ first retrieves relevant experiences:

recalled_memories = recall_similar_cases(
query=customer_issue,
bank_id=bank_id
)

context_text = _format_memories_for_prompt(
recalled_memories
)

Those memories are then included in the model's prompt alongside the current customer issue.

The recommendation workflow is instructed to use the supplied historical experiences as historical evidence and not invent previous support cases.

That changes the role of the model. Instead of asking the model to guess a solution from general knowledge, we ask it to reason over retrieved experience and communicate limitations when the evidence is insufficient.

Illustrative UI visualization of Hindsight recall; replace with a real runtime screenshot before publication.

A Real Example: The 504 Gateway Timeout

We used a 504 Gateway Timeout during a large analytics export as a representative support case.

ResolveIQ recalled five memories for the case. One recalled experience described an intermittent 504 problem during large analytics exports. The stored experience included a concrete resolution involving streaming CSV responses, increasing the reverse-proxy timeout, and verifying successful exports of 100,000 rows in 34 seconds.

That is more useful than simply retrieving a previous question containing the words “504 Gateway Timeout.”

The historical experience contains an action and a verification result.

The recommendation agent can use that context while still distinguishing historical evidence from its current recommendation.

[Insert Screenshot 3: Recommendation based on recalled experience]

Turning Outcomes Into Memory

Recall alone would not create a learning loop.

After a support worker takes an action, ResolveIQ records the action, outcome, and verification result. The backend turns those fields into a structured support experience:

memory_content = (
f"Customer Support Case Experience:\n"
f"Customer Issue: {customer_issue.strip()}\n"
f"Action Taken: {action_taken.strip()}\n"
f"Outcome: {outcome.strip()}\n"
f"Verification Result: {result.strip()}"
)

It then retains that experience in Hindsight:

response = client.retain(
bank_id=target_bank,
content=memory_content,
context="Customer Support Resolution Outcome",
)

We deliberately did not want to store only “Resolved.”

A future agent needs to know what happened, what was done, and why the result was considered successful. That makes the stored experience useful when another similar issue arrives.

Illustrative UI visualization of recording an outcome and retaining it; replace with a real runtime screenshot before publication.

What Changed With the Feedback Loop

Without the feedback loop, the architecture is roughly:

Issue → LLM → Recommendation

The recommendation ends with the interaction.

With ResolveIQ, the workflow becomes:

Issue
↓
Recall previous experience
↓
Recommendation
↓
Real-world action
↓
Verified outcome
↓
Retain experience
↓
Recall it later

The second design gives future requests access to evidence created by previous requests.

That is the part we found most interesting about Hindsight. The value is not simply that the system can remember text. The application can connect a recommendation to the result that followed it and make that experience available to later decisions.

Keeping the LLM Grounded

ResolveIQ uses Groq for recommendation generation, with the model configurable through GROQ_MODEL.

The recommendation prompt is organized around:

Understanding of the Issue

Relevant Previous Experience

Recommended Action

Why the Action is Recommended

Confidence & Limitations

Verification Note

Hindsight and the LLM therefore have different responsibilities.

Hindsight retrieves experience.

The LLM reasons over that experience.

The application then asks the support worker to verify the actual result.

That separation makes the learning loop easier to understand and test.

What We Learned

Memory is useful only when it changes a future decision

Storing previous interactions is not enough. The useful test is whether a later case can retrieve something relevant and use it to produce a better-grounded recommendation.

Outcomes are more valuable than unanswered conversations

A support conversation can tell us what someone tried. A verified outcome tells us what happened after they tried it. That distinction influenced our retention format.

Verification belongs in the memory

We store the verification result with the action and outcome. For technical support, “we changed the timeout” is weaker evidence than “we changed the timeout and verified successful exports.”

Retrieval should precede reasoning

We chose an explicit recall → context → reasoning workflow rather than asking the LLM to independently guess what might be relevant. This makes the source of historical context visible in the application.

Memory does not remove human verification

ResolveIQ does not treat recalled experience as unquestionable truth. A previous successful resolution can still be wrong for a new environment. That is why the workflow records the actual outcome separately.

Limitations and Next Steps

The current implementation is deliberately focused.

It does not automatically evaluate recommendation quality or decide whether a recorded outcome is trustworthy. The support worker supplies the action, outcome, and verification result.

The next improvements we would consider are automated evaluation of recommendation relevance and outcome quality, operator tools for correcting or annotating memories, stronger end-to-end testing, and production monitoring.

Those additions would help answer a harder question: not only whether an agent remembers, but whether its accumulated experience is actually improving its decisions.

Conclusion

The biggest change in ResolveIQ was not adding another prompt to the support agent.

It was adding a place for experience to go after the prompt was finished.

Hindsight gives the system the memory operations we needed: recall relevant experience before reasoning and retain verified outcomes after a support case.

That turns the architecture from:

Question → Answer

into:

Question → Experience → Recommendation → Outcome → New Experience

For us, that missing feedback loop was the difference between an agent that could answer support questions and an agent designed to learn from what happened next.

Hindsight on GitHub · Hindsight Docs · Vectorize Agent Memory