Hindsight Recalled 126 Memories. I Kept 15.

The first time I asked Hindsight what it knew about a Payment Service connection timeout, it handed back 126 memory records. Plenty of them were the same sentence wearing a different date.

That is the problem this post is about: not whether an agent can remember an old outage, but what you do with everything it remembers before a model reasons over it. I built RECALLOPS to investigate production incidents. It recalls what Hindsight retained from earlier ones, filters that memory, and asks Groq to reason over what's left. Most of the interesting code turned out to live between the recall call and the model.

What it does, end to end

RECALLOPS is a FastAPI backend with a React front end. You submit an incident (service, problem, symptoms, severity) and the backend runs one loop:

Incident form (React)
        │
        ▼
FastAPI ── recall ──► Hindsight memory bank
   ▲                        │
   └── candidate memories ──┘
        │
        ▼
dedupe → classify → rank → group by incident ID
        │
        ▼
Groq: analysis citing [HM-n] memory references
        │
        ▼
SQLite record ── retain ──► Hindsight (for the next incident)

The memory layer is Hindsight. On retain, an LLM extracts facts, entities and temporal data. On recall, it runs semantic, keyword, graph and temporal search in parallel and reranks the merge. reflect reasons across memories instead of looking one up. That is what I want from agent memory, and I wrote none of it. Everything lives in one bank, recallops. SQLite holds the rows the dashboard needs, and Groq runs openai/gpt-oss-120b.

The dashboard: incident records come from SQLite, memory lives in Hindsight.

Retaining incidents like a postmortem

retain is one call, so the work is in what you hand it. I write each incident as a small postmortem: ID, service, symptoms, and separate fields for root cause, resolution and outcome.

# condensed from routes/incidents.py
incident_memory = f"""
NEW PRODUCTION INCIDENT
Incident ID: {incident.incident_id}
Service: {incident.service}
Problem: {incident.problem}
Symptoms: {incident.symptoms}

Confirmed Root Cause: {incident.root_cause or "Not yet identified"}
Confirmed Resolution: {incident.resolution or "Not yet applied"}
Confirmed Outcome: {incident.outcome or "Pending"}

AI Investigation: {analysis}

IMPORTANT: The AI investigation contains hypotheses
and recommendations. These are not confirmed facts
unless verified by an engineer.
"""
client.retain(bank_id=BANK_ID, content=incident_memory)

The incident ID is the only join key I get across memories, so it is always in the text. The defaults say "Not yet identified" instead of leaving a blank, so a missing root cause reads as unknown, not absent. The last paragraph is the part I would change. I retain the model's own analysis, hedged with that warning, which means guidance my agent wrote comes back later as "memory" next to real incidents.

126 recalled, 15 kept

Recall is one call, and what comes back is candidate evidence, not an answer. For one Payment Service query, the similarity endpoint reported 126 retrieved records. The cause is mundane: one retain becomes several extracted facts, and every analysis run retains the incident again. One outage turns into a dozen paraphrases, each ending in metadata like | When: 2026-11-06 | Where: Payment Service.

Before anything reaches Groq, prepare_historical_memories strips that metadata and the dates, drops exact and near-duplicates, tags and scores each survivor, and keeps the top 15.

# are_conservative_duplicates()
return (
    containment_score >= 0.90
    and (jaccard_score >= 0.75 or sequence_score >= 0.85)
)

# prepare_historical_memories()
if service and service in memory_lower:
    relevance_score += 5
if category == "HISTORICAL_INCIDENT":
    relevance_score += 3
if incident_ids:
    relevance_score += 2
...
selected_memories = processed_memories[:15]

The category (HISTORICAL_INCIDENT, INVESTIGATION_UPDATE or INVESTIGATION_GUIDANCE) comes from keyword lists. I kept the duplicate test conservative on purpose, because merging two records that only look alike can delete the one fact that differs.

This filter only runs on the similarity path. /analyze still hands raw recall to the model, which is why its memory panel looks like this:

Raw recall from /analyze, paraphrases included.

Making the model cite its sources

Filtering does not make the surviving 15 trustworthy, so the prompt does the rest. Each memory gets a stable label (HM-1 to HM-15), and records are grouped by incident ID with a regex on INC-.... Anything without an ID is labelled unlinked evidence, and the model is told never to count unlinked records as separate incidents. The evidence rules in the prompt include these:

1. Every historical factual claim must include its
   memory reference, such as [HM-1].
3. Never invent memory IDs or incident IDs.
4. Similar symptoms do not prove identical root causes.
7. A resolution is documented only when the memory
   explicitly states that it was applied.
9. If evidence is missing, write "Not documented".

Temperature is 0.1, and the prompt forbids recommending automatic production changes, so a human approves every fix. The UI turns each HM-n into an expandable evidence card, so an engineer can click any claim back to the memory behind it.

Each HM-n reference expands to the original Hindsight record.

Reflect: finding the pattern, not the record

Recall answers "what happened." reflect answers the question I actually cared about: what keeps happening?

# routes/incidents.py
result = client.reflect(bank_id=BANK_ID, query=request.query)
return {"success": True, "query": request.query, "answer": result.text}

I wired it to a Generate AI Insights button. Asked about the Payment Service, it laid out the pool exhaustion on November 6, the recurrence on November 7, and the fact that both times the fix was raising maxPoolSize and restarting. Its conclusion: those are temporary "quick-fixes" that mask the underlying root causes. That framing comes from reading across records instead of fetching one, and it is the observation I wanted from a memory layer in the first place.

Memory search over the Hindsight bank, above the SQLite incident records.

Generate AI Insights, backed by reflect.

Before and after

The test case: Payment Service, "Database Connection Timeout", payment failures, connection pool at its maximum.

The incident form.

Without memory, generate_incident_analysis gets the incident form and the fallback line No relevant historical incidents found. There is nothing to compare against and nothing to cite.

With memory, the 15 records that reached Groq were sorted before it said a word. Three linked to one confirmed incident, INC-1794015715580, from earlier that day. Ten were Payment Service records with no incident ID. One (HM-14) was guidance rather than an incident, and one (HM-15) belonged to an unrelated Authentication Service incident. Without grouping, that reads as fifteen outages. With it, the report named one confirmed incident and called the rest unlinked evidence that "cannot be assumed to be the same incident."

The similarity report: 126 records retrieved, 15 used, grouped by incident ID.

The analysis that followed refused to invent a root cause: "Not yet identified." It gave numbered hypotheses instead, each tied to history. For pool exhaustion: "Historically the same symptom has been caused by genuine pool saturation." For a connection leak, it pointed at the November 4–6 deployments.

The Groq analysis: known facts, an open root cause, numbered hypotheses.

What I'd tell you before you build this

1. Treat recall as candidates, not evidence. Dedupe, rank and cap before the model sees anything. 126 records in a prompt is mostly repetition.
2. Put a stable ID in every memory. It was the only join key I had. Ten of my 15 records were Payment Service evidence with no ID, so the model had no way to tell whether they described one outage or ten.
3. Make the model cite or abstain. [HM-n] references and "Not documented" are the cheapest hallucination check I found. A claim without a reference is easy to spot.
4. Do not feed your agent's guesses back as memory unlabelled. I hedge them in the retained text, but I still classify memories with keyword lists after recall, which is brittle. retain also accepts context and metadata, and I only pass content. Labelling AI output at write time is the first fix I would make.
5. Use reflect for patterns and recall for evidence. They answer different questions, and only recall gave me something I could cite.

What is still rough

• No feedback loop. I store outcomes, but nothing verifies that a recommended fix worked. The dashboard's Resolved counter sits at zero because "resolved" is a string match on the outcome field.
• Thin data. My bank is small and almost entirely one service, I have not benchmarked recall quality, and the dedupe thresholds are hand-set, not measured.

Analytics, computed from SQLite.

Try it

The code is on GitHub. If you are building on agent memory, start with the Hindsight repository and the Hindsight docs.

The rule I keep coming back to: every incident should leave behind knowledge that makes the next investigation easier, and every recalled memory should have to earn its place in the prompt.

Thanks to Code.in and the Vectorize team.