DEV Community: Pratheeksh S

I built an AI on-call agent that never forgets an outage

Pratheeksh S — Sat, 07 Nov 2026 15:56:21 +0000

It's 3 a.m. Your phone buzzes. The payments service is throwing KeyError: 'amount' after a deploy, and customers can't pay.

You open the logs, then Slack, then old postmortems. Forty minutes later you find it: a stale Redis cache key from the old version. Then someone says the words every on-call engineer hates: "Oh yeah, we had this exact problem in March."

That sentence is the reason I built DejaVu, an AI incident agent that remembers every outage your team has ever fixed.

The real problem: teams forget

Most outages are not new. The same few failures come back again and again: cache bugs, database connections running out, bad config, slow queues. They just look a little different each time. A different service, a different error message, a different person on call.

The fix usually exists somewhere. It's in an old Slack thread, a postmortem nobody reads, or the head of an engineer who is asleep or has left the company.

General AI chatbots don't solve this. They have never seen your systems. Ask one about your outage and it will give you a polite, generic guess. What on-call teams need is not a smarter guess. They need memory.

What DejaVu does

DejaVu works in three simple steps: recall, reason, retain.

Recall. A new error comes in. DejaVu searches its long-term memory for past incidents with the same failure pattern, even when the words are completely different.
Reason. An LLM (GPT-OSS 120B, running on Groq) reads only those recalled incidents and returns a root cause, a step-by-step fix, a time estimate, and the exact past incidents it used as proof. It is not allowed to cite anything memory didn't return.
Retain. When the engineer clicks Fix worked or Fix failed, that outcome is saved as a new memory. The next outage gets solved faster.

The memory layer is Hindsight, an open-source agent memory system by Vectorize. If you're new to the idea, Vectorize has a good explainer on what agent memory is and why it's different from simply stuffing documents into a prompt.

Same AI, with and without memory

The moment that convinced me this works is the side-by-side test in the app.

I simulate the cache KeyError outage and ask the same model twice:

Without memory: 20% confidence. "Likely a schema mismatch after deploy. Check the code and cache." A reasonable guess, but still a guess.
With Hindsight memory: 98% confidence. "Redis cache invalidation bug after the v3.2 deploy, leaving stale v1 keys." It gives the exact commands to flush the keys and restart the pods, and it cites 5 past incidents where this fix worked.

Same model. Same prompt. The only difference is memory.

Why keyword search isn't enough

In my dataset of 20 incidents, the cache bug happens 6 times, on 3 different services, with 6 different error messages: KeyError, TypeError on a cart object, HTTP 500s, "password reset tokens rejected as expired", and "promo codes ignored at checkout".

A keyword search for "KeyError" finds one or two of these. Hindsight's semantic recall finds all of them, because it matches on meaning, not on words. That's what lets DejaVu say "seen this 5 times, and this fix worked every time".

How Hindsight is used

Hindsight is the core of the project, not an add-on. DejaVu uses four of its operations:

create_bank makes a memory bank called dejavu-incidents, with a mission: remember each incident's error, root cause, exact fix and whether it worked.
retain stores each incident as plain English, with its real timestamp, a stable document_id (so re-runs update instead of duplicating), and metadata like the service name.
recall finds similar past incidents for a new error.
reflect answers big-picture questions across all incidents, like "What keeps breaking?"

Here is the recall step. Hindsight returns facts, and I group them by incident and rank them by the reranker's relevance score, not by recency, so an old but identical outage still wins:

def recall_similar(error_text, max_incidents=5):
    with memory_client() as client:
        response = client.recall(bank_id=BANK_ID, query=error_text, budget="mid")
    matches = {}
    for fact in response.results:
        incident_id = int((fact.metadata or {}).get("incident_id"))
        scores = fact.scores
        score = scores.reranker if scores.reranker is not None else scores.final
        entry = matches.setdefault(incident_id, {"incident_id": incident_id, "match": 0.0, "facts": []})
        entry["match"] = max(entry["match"], float(score or 0))
        entry["facts"].append(fact.text)
    return sorted(matches.values(), key=lambda m: m["match"], reverse=True)[:max_incidents]

The Hindsight docs cover retain, recall and reflect in detail. I ran the Hindsight server locally, with Groq as its LLM provider, so the whole stack is free to run.

Confidence you can trust

I didn't want the AI to just say it was 98% sure. So the confidence score comes from memory, not from the model's mood:

3 or more past incidents where the same fix worked: 90 to 98%
2 proven matches: 75 to 89%
1 proven match: 55 to 74%
Nothing proven: 30% or less, and it says "No similar incident in memory"

Under the score, the app shows the reason in plain words, like "5 of 5 matching past incidents were fixed this way." Engineers can check every claim.

It learns while you use it

The part I enjoy most is watching it learn. When I click Fix worked, the incident is retained in Hindsight in the background. A new green dot appears on the live memory map, and the next time the same alert fires, that fresh incident is often the best match.

The learning curve tells the story. Across the 20 incidents, the average time to fix fell from 44 minutes for the first 5 to 6 minutes for the last 5. That's 87% faster as the memory grew.

There's also a Lessons from memory panel. A team lead can ask "What keeps breaking?" or "Which fixes failed, and why?", and Hindsight's reflect reasons over every incident to answer, with incident numbers as proof.

What I learned building it

Memory beats a bigger model. The jump from 20% to 98% came from memory, not from a smarter LLM.
Rank by relevance, not recency. My first version preferred recent incidents, and it missed old outages that were identical. The reranker score fixed that.
Ground every claim. Forcing the model to cite only recalled incidents, and dropping any it made up, made the answers far more trustworthy.
Plan for free-tier limits. Groq's free tier allows about 8,000 tokens a minute, so the loader paces itself and the app has a fallback when reflect is busy.

Try it

DejaVu is open source and runs on a laptop:

Code: https://github.com/Pratheeksh-7S/dejavu-incident-agent
3-minute demo: https://youtu.be/Nu_sktguGMk
Hindsight: https://github.com/vectorize-io/hindsight

On-call will always be stressful. But it shouldn't mean solving the same outage twice. With memory, every outage you fix makes the next one easier.

Every outage, remembered.

Recall Desk: three AI support agents that share one memory, and the memory is a graph.

Pratheeksh S — Sat, 17 Oct 2026 19:21:51 +0000

Title
Recall Desk: three AI support agents that share one memory, and the memory is a graph.
Published
true
Description
How I used FalkorDB as the memory and coordination layer for a multi-agent support desk (Graph Hacks, Track 02).
Tags
Falkordb, AI, GraphDatabase, Hackathon
Cover image
https://github.com/Pratheeksh-7S/Recall-Desk-/raw/main/docs/graph-path.png
Every support chat I've been in has the same weak spot. You explain your problem to the first person, they transfer you, and you explain it again. Most AI agent demos copy this behaviour. Each agent has its own prompt and its own chat history, and "memory" is a long text log passed from one agent to the next.

For the WeMakeDevs × FalkorDB Graph Hacks hackathon (Track 02: Agent Memory and Coordination), I built Recall Desk, which tries something different. Three agents (triage, billing and technical) answer customers together, and they never hand context to each other in prompts. Everything an agent learns goes into a FalkorDB graph, and every other agent reads it from there, in the same session or a later one.

Live demo: https://recall-desk.onrender.com (free tier, so the first load can take a minute)
Code: https://github.com/Pratheeksh-7S/Recall-Desk-
Video: https://github.com/Pratheeksh-7S/Recall-Desk-/raw/main/docs/demo.mp4
The story the demo tells
Session 1. Priya, on the Team plan in eu-west, writes, "I was charged twice this month."

Triage reads her memory from the graph and opens a ticket.
The graph says (billing)-[:CAN_HANDLE]->(billing topic), so triage hands the ticket off. The handoff is a node, not a message.
The billing agent finds two paid $30 invoices for the same month, refunds the duplicate (a graph write), and stores a promise memory.
She then adds, "Bulk Export has been stuck for hours. " The technical agent follows a 7-hop path from her plan to an open incident in her region, finds the other affected customers and the other tickets with the same root cause, and links her ticket CAUSED_BY the incident.

Session 2. Priya comes back the next day. Before she types anything, triage says, "Welcome back." Last time: Billing issued a $30 refund for INV-5003…" Nobody asks her to explain again.

Every answer in the UI comes with the graph path behind it, so you can see exactly why the agent said what it said.

Why does the graph have to be there?
My rule for the hackathon was simple: if the app still works without FalkorDB, the graph is only decoration. Recall Desk has no SQL tables, no vector store and no chat history kept in Python. Remove FalkorDB, and nothing works.

Root cause with variable-length paths "Bulk export is stuck" is a question about connected data. The feature depends on services, which depend on other services, and somewhere down that chain there may be an incident in the customer's region:

MATCH (c:Customer {id:$customer_id})-[:IN_REGION]->(r:Region)
MATCH p=(c)-[:SUBSCRIBES_TO]->(:Plan)-[:INCLUDES]->(f:Feature)
-[:DEPENDS_ON*1..6]->(s:Service)<-[:ON]-(i:Incident {status:'open'})-[:IN_REGION]->(r)
WHERE toLower(f.name) = toLower($feature)
RETURN f.name AS feature, s.name AS root_service, i.id AS incident, length(p) AS hops, p
ORDER BY hops LIMIT 3
In SQL this would be a recursive CTE and several joins. In Cypher it's one pattern, and the path p it returns is what the UI draws.

The same idea answers "Is it just me?" (walk back from the incident to every customer whose plan reaches the broken service) and "Who else has this?" (tickets that share a transitive dependency). algo.pageRank('Service','DEPENDS_ON') shows which services matter most.

Memory as nodes, not text There are three kinds of memory, all in the graph:

Episodic: (:Session)-[:HAS_TURN]->(:Turn)-[:NEXT]->(:Turn), with FOLLOWED_BY linking one session to the next.
Semantic: (:Customer)-[:HAS_MEMORY]->(:Memory), plus LEARNED_IN, WRITTEN_BY and ABOUT edges, so every fact knows where it came from and which agent wrote it.
Procedural: (:Procedure {successes, failures}). When a customer marks a ticket "not resolved", the playbook's failure count goes up. The agent ranks playbooks by a smoothed success rate, so after enough failures it switches to a better one by itself:
MATCH (p:Procedure)-[:FOR_TOPIC]->(:Topic {name:$topic})
WITH p, (p.successes + 1.0) / (p.successes + p.failures + 2.0) AS rate
RETURN p.name AS name, round(rate * 100) AS success_rate
ORDER BY rate DESC LIMIT 1

Coordination with graph writes When triage passes a ticket to billing, it creates a Handoff node with FROM, TO and FOR edges, plus a CARRIES edge to each of the customer's memories. The specialist reads those memories instead of asking the customer again.

Two agents may try to claim the same ticket at the same moment. A single atomic statement settles it:

MATCH (t:Ticket {id:$tid})
SET t.owner = coalesce(t.owner, $agent)
WITH t MATCH (a:Agent {name:t.owner})
MERGE (t)-[:ASSIGNED_TO]->(a)
RETURN t.owner AS owner
I tested this with 30 agents racing for one ticket: one owner and one ASSIGNED_TO edge every time. Memory writes use MERGE, so a load test of 16 threads × 200 operations (about 390 ops/s) produced zero duplicate memories.

One graph per company Each tenant gets its own graph (recalldesk:acme, recalldesk:globex). I seeded the same customer email into both companies and checked that nothing leaks across.

Keeping the LLM honest
The LLM (Claude, GPT or Gemini, set by an environment variable, with a rules-only offline mode) does two jobs: it works out what the customer wants, and it words the reply. It never writes Cypher. It picks from a fixed catalogue of parameterised tools, and the customer id is always forced to the real customer.

Every reply is path-checked. If the LLM mentions an invoice, incident, article or ticket id that isn't in the graph results or the recalled memory, the reply is thrown away and an answer built straight from the graph rows is sent instead. The facts come from the graph, and the LLM only does the writing.

What I learned
Graphs make memory explainable. A memory node with a LEARNED_IN edge to the session and a WRITTEN_BY edge to the agent answers "Why do you think that?" for free.
Coordination is a data problem. Routing (CAN_HANDLE), handoffs and ticket ownership are rows and edges, not prompt engineering. That makes them testable.
Variable-length paths are the killer feature for support. Real root causes sit several hops away from the symptom.
Stack
FalkorDB (Docker locally, FalkorDB Cloud in production), Python + FastAPI, vanilla JS with vis-network for the graph view, and GitHub Actions running the end-to-end tests against a real FalkorDB. It's deployed on Render.

I built this with help from an AI coding assistant (Claude Code), as disclosed in the README.

If you try the live demo, pick Priya Raman and type "I was charged twice". Then open the Memory graph tab and watch the agents write to it.