The goldfish thing is a myth, by the way. Studies suggest real goldfish can remember things for months.
Most recruiter bots I've seen can't manage a week.
I kept running into the same problem while building Recall.
A candidate could spend twenty minutes explaining what she wants from her next job. Three weeks later, the recruiter agent would see a new opportunity and behave as if that conversation had never happened.
At first, that felt like an intelligence problem.
It wasn't.
The model wasn't forgetting because it was bad at reasoning. It simply wasn't being given the old conversation.
So I stopped trying to make the model "remember" and started thinking about what should happen outside the model.
I built Recall, a recruiter memory agent that extracts useful information from candidate conversations, stores it somewhere durable, and retrieves the relevant context when a new opportunity appears.
Hindsight, the open-source agent memory layer from Vectorize, is what makes that loop possible.
The model is the goldfish, so I stopped asking it to remember
Once I accepted that the model wasn't going to remember previous conversations on its own, the architecture became much simpler.
I needed somewhere else to keep that history.
Recall ended up being a small loop: Groq extracts the pieces of a conversation that might matter later, Hindsight stores them, and when a recruiter needs that context again, Hindsight retrieves the relevant memories and Groq turns them into something the recruiter can actually use.
The rest is just the plumbing around that loop:
- Express serves the interface and runs the workflow.
-
Groq (
openai/gpt-oss-120b) extracts facts and generates recruiter-facing text. -
Hindsight provides long-term memory through
retain()andrecall(). - The browser UI lets me see the extracted facts, recalled memory, and generated response.
Groq doesn't store the candidate history. Hindsight doesn't decide what a recruiter should do.
The application connects the two.
The model extracts durable information, Hindsight stores and retrieves it, and the model uses that retrieved context to generate the next response.
That separation also made debugging much easier. When something went wrong, I could ask a concrete question:
Did the right memory come back, or did the model do something unexpected with the memory it received?
Step one: decide what is worth remembering
I didn't want to retain raw conversations forever.
Before anything is stored, Groq reduces the conversation to short pieces of information that might actually matter later: career goals, technical interests, location, work model, salary, work style, and constraints.
For Rashi Sharma, the first conversation gave us a handful of things worth keeping:
- She wanted to move deeper into backend engineering.
- Node.js was important to her.
- Hyderabad was her first choice.
- She was targeting ₹13–15 LPA and was currently earning ₹10 LPA.
- She cared about reasonable working hours and wanted to avoid another high-pressure environment.
None of these facts is particularly complicated.
The interesting part was making sure they were still available three weeks later, when a completely different recruiter interaction happened.
Once Groq extracted those facts, I retained them in Hindsight:
const importantFacts = response.choices[0].message.content;
await hindsight.retain(
BANK_ID,
`Important candidate information from a recruiter conversation
with Rashi Sharma:\n\n${importantFacts}`
);
I could have forced these into a rigid schema instead — salary_min, work_model, and so on. I didn't, because the things candidates actually say don't fit neatly into fields. "I'd rather not join another startup where weekends disappear" is a real dealbreaker, and there's no column for it. Keeping it as a short, natural bullet meant nothing got flattened out of it.
The test that made the difference obvious
Saying "memory makes the agent better" is easy. Proving it is harder, because it's tempting to change too many things at once and call the improvement "memory."
So I built the comparison to remove every other variable. Same Groq model. Same question. Same candidate. Same opportunity. The only thing that changes is whether Hindsight's recalled context is added to the prompt:
const context = withMemory && rememberedInformation
? `\n\nRecalled candidate context from Hindsight:\n${rememberedInformation}`
: "";
Without memory, I ask how to introduce a Backend Engineer opportunity to Rashi, and the agent writes something like:
"Hi Rashi, we have a Backend Engineer role in Hyderabad (₹14 LPA, hybrid, Node.js, predictable hours). Would you be open to learning more about this opportunity?"
That's a perfectly fine message. It's also a message that could go to anyone.
Once Hindsight has her history, the same question produces something else:
"Hi Rashi, just checking in — are you still prioritizing clear communication, reasonable working hours, and a strong work-life balance (and preferring to stay in Hyderabad without immediate relocation)?"
Nothing about the prompt template changed. The model didn't get smarter. The only new input is what she said three weeks earlier. That's the part I wanted judges — and honestly, myself — to see clearly: the improvement isn't coming from a cleverer prompt. It's coming from memory.
Where it actually matters for a recruiter
The real annoyance this solves is small but constant: "remind me, what's your salary range again?" asked for the third time.
The recalled memory holds her current pay and her target range, so when a ₹14 LPA role comes in, the brief can say directly that it fits. Nobody has to ask her again.
The brief itself has three parts — a summary, a list of matches, and a list of things to verify. That verify list is the part I care about most:
- Confirm the role doesn't involve a high-pressure startup environment or frequent overtime.
- Confirm the position offers clear communication channels and the level of backend ownership she's looking for.
A job description that says "predictable hours" is a claim, not a fact. Her stated preference is a reason to check, not proof the role delivers. Keeping those two things separate was a deliberate choice — I didn't want the brief to sound more certain than it actually is.
Asking for memory back, not everything
Storing information turned out to be the easy half. Retrieving the right thing at the right moment took most of my time.
I never ask Hindsight for "everything about Rashi." Each stage asks a question shaped for that stage. A follow-up asks about goals, location, work model, compensation, and constraints. A new opportunity asks about compensation range and related preferences.
const memory = await hindsight.recall(BANK_ID, query, {
includeSourceFacts: true,
maxSourceFactsTokens: 2000
});
Hindsight runs semantic, keyword, graph, and temporal search together, which is why a question phrased around a job can still surface a fact that was stored in completely different words. I didn't have to write any of that retrieval logic myself — that part was already solved for me.
The bug that showed up in the open
Here's something I didn't expect to end up in this article, but it's honest, so I'm keeping it in.
The interface shows the raw memory Hindsight returns, right above the generated brief. When I scrolled through Rashi's recalled memory during testing, I found the same fact repeated over and over — "prioritizes clear communication, reasonable working hours..." with slightly different dates, and her salary restated five or six different ways.
I'd been retaining the same test conversation every time I reran the app. Hindsight stored each one exactly as instructed — nothing was wrong on its end. The memory wasn't incorrect, just noisy, and the noise made the evidence harder to read than the brief it was supposed to support.
I only caught it because I chose to show the raw recall instead of hiding it behind a "memory used" badge. If I hadn't, memory would have kept quietly piling up with every test click, and I wouldn't have known. The fix was simple once I saw it: retain once per conversation, not once per interaction with the app.
What I learned
Retrieval is part of the product, not an implementation detail. Storing information is only half the work. The question you ask when retrieving it decides what actually comes back.
A fair comparison needs one variable, not five. Same model, same question, same inputs — memory on or off. Anything else and you're comparing prompts, not memory.
Show the evidence, don't just claim it. A badge that says "memory used" is a claim. Showing the actual recalled memory next to the response is proof — and it's also how I caught my own bug.
Preferences aren't decisions. Recall surfaces what seems to align and flags what to double-check. It never tells anyone who to hire.
Be deliberate about what you retain, and when. Memory that grows with every test click isn't memory — it's noise with a timestamp.
What I'd still want to fix
Candidate data is personal — salary expectations, current pay, reasons someone wants to leave a job. Any real version of this needs authenticated access, per-candidate isolation, explicit consent, and a straightforward way to correct or delete what's stored. None of that changes the core design, but it's not optional either.
The biggest shift for me, honestly, was dropping the phrase "an agent with memory." It sounds impressive but explains nothing about what actually happens when a new opportunity shows up. A better test is simpler: can the system retrieve the right earlier detail, explain why it matters now, and leave enough room for a person to verify it?
Hindsight gave Recall a place to keep that memory. Everything else — asking good questions, grounding generated text in what came back, keeping a person in the loop — was still on me to build.
If you're working on something similar, the Hindsight documentation is a good starting point, and Vectorize's write-up on agent memory and persistent context covers the ideas behind it well. The code for Recall is in the repository here.
Top comments (1)