Making Recalled Memory Actually Influence LLM Recommendations
When we integrated Hindsight into PayEcho, retrieving a customer's history was not the difficult part.
The difficult part was getting the language model to actually use that history when generating a recommendation.
The model could see the recalled information in its context and still produce almost the same generic answer it would give to a customer with no history.
That became the main engineering problem I worked on:
How do you make recalled memory act as evidence for an LLM's recommendation instead of just additional context?
The initial approach
PayEcho is a payment-recovery and credit-decision agent.
For payment recovery, the system needs to consider things such as:
previous recovery attempts
communication channels
customer responses
payment outcomes
timing of previous follow-ups
The first version of the recommendation flow was straightforward:
Get the current invoice.
Recall the customer's previous history.
Give both to the language model.
Ask it for a recommendation.
The model produced a reasonable-looking answer.
But there was a problem.
For a customer with several months of recorded behavior, the recommendation could look almost identical to the recommendation for a customer the system had never seen before:
"Send a polite email reminder, and follow up in 3–5 business days if no response."
The historical information was present in the context.
But that does not guarantee that the generated response is based on those events.
The model can treat the recalled information as background context and fall back to a generic recommendation.
So the problem wasn't initially the memory retrieval itself.
It was the connection between retrieval and reasoning.
Making memory part of the reasoning
The change was surprisingly small.
Instead of simply asking the model to make a recommendation using the available history, the recommendation needed to cite the specific previous outcome that justified the recommendation.
suppose the recalled history showed that a customer:
ignored previous email reminders
responded to WhatsApp
completed payment after a follow-up three days later
The recommendation could then be:
"ABC previously ignored email reminders but responded to WhatsApp, and completed payment after a 3-day follow-up. Recommend WhatsApp outreach with a scheduled 3-day follow-up."
That is different from simply saying:
"The customer has previous payment history."
The first recommendation identifies the evidence that influenced the decision.
This made the memory load-bearing rather than decorative.
The important part is that the model itself does not change between these interactions.
What changes is the evidence available to it.
recall() retrieves previous recovery attempts and outcomes.
The current invoice is considered alongside that history.
The model then produces a recommendation involving things such as:
communication channel
timing
tone
The model simply wasn't required to use it.
Having memory is not the same as using memory
This was the distinction that became important in the implementation.
Top comments (0)