I gave my on-call agents a graph instead of a bigger prompt

Yashas M — Sun, 18 Oct 2026 16:44:17 +0000

It is 2 a.m. and checkout is down.

Five alerts are firing across four services. checkout-service is throwing 500s, payments-service is slow, fraud-service is timing out and payments-db is almost out of connections. The on-call engineer opens Grafana, then the deploy log, then the service catalog, then Slack search ("didn't this happen before?"), then the wiki. Twenty minutes later they have rebuilt, in their head, a picture that already existed. It was just spread across ten tools.

I kept coming back to one thought while building for Graph Hacks (WeMakeDevs x FalkorDB): every question in that story is about how things connect.

Which alerting service is the cause and which ones are just victims? Follow the dependencies until the failures stop.
What changed recently? Look at deploys next to the failing node.
What breaks next? Walk the dependencies backwards.
Has this happened before, and who fixed it? Find incidents with the same root cause, then follow them to the person who resolved them.

A vector store gives you chunks that look similar. A graph gives you the relationships, and it can show you the path. So I built GraphOps, an incident commander where three AI agents work an outage through one FalkorDB graph.

Try it: graphops-gr4u.onrender.com (free hosting, so the first load after a quiet spell takes about 50 seconds)
Code: github.com/Yashasm18/graphops

What it does in 20 seconds

You type "Checkout is throwing 500s and customers can't pay" and click Investigate. About 20 seconds later GraphOps tells you:

The root cause is payments-db connection exhaustion, triggered by deploy DEP-981 that raised retry attempts from 3 to 10, 23 minutes ago. This is the same failure as incident INC-2041. Roll back and follow runbook RB-PAY-01. Page Diego (on call) and Nina (she fixed it last time). By the way, Diego warned against this exact change in #payments-eng, and the follow-up ticket from the last incident is still open.

That last sentence is my favourite part, and I will come back to it.

One graph for the whole company

Everything lives in a single FalkorDB graph per company:

(:Service)-[:DEPENDS_ON]->(:Service)
(:Service)-[:OWNED_BY]->(:Team)<-[:ON_CALL_FOR]-(:Person)
(:Service)-[:POWERS]->(:Capability {revenue_per_min})
(:Deploy)-[:CHANGED]->(:Service), (:Deploy)-[:AUTHORED_BY]->(:Person)
(:Alert)-[:FIRED_ON]->(:Service)
(:Incident)-[:ROOT_CAUSE|AFFECTED]->(:Service), -[:RESOLVED_BY]->(:Person)
(:Runbook {success_count, failure_count})-[:FOR]->(:Service)
(:Decision)-[:ABOUT]->(:Service), -[:MOTIVATED_BY]->(:Incident)
(:Deploy)-[:IMPLEMENTS]->(:Ticket)-[:PART_OF]->(:Project)
(:Message {channel})-[:MENTIONS]->(:Ticket|:Incident|:Decision)

The Capability nodes are a small idea that paid off a lot. Each one carries a revenue per minute, so "which services are affected" becomes "how much money are we losing right now".

Three agents that never talk to each other

Triage finds the root cause and the blast radius.
Historian recalls past incidents, runbooks and decisions, and traces why the suspect change was shipped.
Commander decides what to do and who to page.

Triage and Historian run at the same time. The design choice I am proudest of is that the agents never pass messages to each other. Each one writes Finding nodes into a shared investigation, linked to the evidence they are based on:

(:Investigation)-[:HAS_FINDING]->(:Finding)-[:EVIDENCE]->(payments-db, DEP-981, INC-2041)

The Commander starts by reading that subgraph back, and its decision is linked to every finding with BASED_ON edges. That one choice gave me three things for free:

Shared state without a message bus.
An audit trail. "Why did you page Nina?" is a two hop query from the decision to its findings to the evidence.
Memory. When the incident is resolved, the investigation turns into a new Incident node, and the next session finds it like any other incident.

Tools, not a text-to-Cypher free for all

My first instinct was to let Gemini write Cypher. It worked in the happy path and fell apart under pressure. So instead each agent gets small, named graph tools, and each tool is one tested query or FalkorDB algorithm. The model only decides which traversal to run.

Here is the heart of the root cause ranking:

MATCH (:Alert {status: 'firing'})-[:FIRED_ON]->(a:Service)
WITH collect(DISTINCT a) AS alerting
UNWIND alerting AS a
MATCH p = (a)-[:DEPENDS_ON*0..5]->(c:Service)
WITH c, collect(DISTINCT a.name) AS explains
RETURN c.name, explains

Every candidate is scored. The biggest weight goes to how many alerts it explains, because the simplest explanation that covers everything usually wins. After that come a few tie breakers: is it alerting itself, are its direct callers failing, is it the deepest failing node, and did a deploy land on it or right next to it.

FalkorDB's algorithms are tools too. PageRank and betweenness find the chokepoints, label propagation finds clusters of tightly coupled services, and shortestPath shows how a failure travelled from mobile-bff all the way down to payments-db.

Do not trust the model, ask the graph

LLMs are confident even when they are wrong. In early runs a weaker fallback model happily blamed payments-service because that is where the deploy happened, even though the thing actually failing was the database underneath it.

So before any decision is saved, it goes through a proof gate. FalkorDB has to return a causal path from every alerting service to the claimed root cause:

MATCH (a:Service {name: $alerting}), (r:Service {name: $root})
CALL algo.SPpaths({sourceNode: a, targetNode: r, relTypes: ['DEPENDS_ON'], pathCount: 1, maxLen: 6})
YIELD path
RETURN [n IN nodes(path) | n.name] AS path

On top of that, the suspect deploy has to be on the root or one hop from it and recent. A node that is not alerting itself also needs at least one failing caller, otherwise any shared hub like a Kafka cluster would look guilty. algo.maxFlow with a capacity of 1 on each edge tells us how many links you would have to cut to isolate the node.

The result is VERIFIED, WEAK or REJECTED. A rejected claim goes back to the Commander to fix. I also check citations: every id an agent mentions, like INC-2041 or DEP-981, is looked up in the graph, and the UI shows "4/4 cited nodes exist in the graph".

Simulate first, on a copy of the graph

Before anyone touches production, you can ask "what if Kafka goes down?" GraphOps copies the whole graph with GRAPH.COPY, runs the simulation on the copy, and deletes it afterwards. The real graph is never touched.

The containment planner goes further. Some dependency edges are marked as having a fallback mode, like "write payment events to a local outbox instead of Kafka". The planner tries every combination of up to three fallbacks on the copy, picks the smallest set that protects the most revenue, then actually deletes those edges on the copy and recomputes the impact to prove it.

The result surprised me the first time. In a Kafka outage, turning on two outbox fallbacks together keeps checkout up and protects $4,200 a minute. Either one on its own protects nothing, because checkout still reaches Kafka through the other path. That is exactly the kind of thing a human misses at 2 a.m.

GRAPH.COPY also solved a very practical problem. On the live site, every visitor gets their own private copy of the company graph, so nobody can wipe or pollute anyone else's memory.

The company brain moment

This is the part I mentioned at the start. The Historian does not just ask "what broke", it asks "why did this change ship?":

MATCH (d:Deploy {id: 'DEP-981'})-[:IMPLEMENTS]->(t:Ticket)-[:PART_OF]->(p:Project)
MATCH (m:Message)-[:MENTIONS]->(t)
RETURN t.id, p.name, m.text

It follows the deploy to ticket PAY-412 in the "Payments reliability Q4" project, and then to the chat thread about it. In that thread Diego wrote: "Careful: INC-2041 was a retry storm. ADR-022 says jitter first, and PAY-388 is still open." The deploy shipped anyway. PAY-388, the follow-up from the last outage, is still open.

No single document contains that story. It only exists in the connections between a deploy, a ticket, a project, a chat message, a design decision and an old incident. That is what a graph is for.

Real actions, with a human in the loop

After a decision, GraphOps proposes real actions: page the chosen people on Discord, and open a GitHub incident issue whose body is the full evidence trail. Nothing goes out until someone clicks Approve, and every approval is written back to the graph as (:User)-[:APPROVED]->(:Action). I did not want an agent paging people on its own.

It remembers

GraphOps keeps three kinds of memory, all in the graph:

Episodic: incidents, including the ones it learned itself.
Semantic: design decisions, docs and facts people teach it in chat.
Procedural: runbooks with success and failure counts. If a human corrects one, a new version is created with a SUPERSEDES edge, and every later session uses it.

Mark an incident as fixed, fire the same alerts again, and the Historian immediately cites the incident it learned a minute earlier.

Measured, not just shown

I did not want to only show a nice run, so I wrote a benchmark that needs no LLM and no API key. It injects random faults into GRAPH.COPY sandboxes, including silent root causes, unrelated noise alerts, and decoy deploys that are often newer than the real culprit.

200 random fault worlds	Result
Root cause found first	71.5% (blaming the newest deploy: 17.5%)
Root cause in the top 3	89%
Culprit deploy found	85% (newest deploy: 53.5%)
Proof gate verifies an unrelated wrong claim	2.2%

Then I tested it on real data. I wrote an importer that reads the public OpenTelemetry Demo compose files and its flagd fault flags, and builds a graph of 19 real services and 35 real dependencies. Every flag that breaks a service is replayed three ways.

OpenTelemetry Demo, 16 real fault flags	Found first	Top 3
The broken service alerts	100%	100%
The broken service is silent	14%	79%
Same fault again, after learning it	recalled 16 out of 16

The silent case is honestly hard. In that topology most services have a single caller, so "the caller broke" and "its silent dependency broke" look the same in the graph. GraphOps keeps the true cause in its top three and the gate says WEAK instead of pretending to know. I would rather show that number than hide it.

Things that broke along the way

Free tier quotas. Gemini's free tier gives about 20 requests per model per day. I ran out of my main model in the first hour. GraphOps now keeps a pool of models and keys with cooldowns, and if every model is out, each agent falls back to a scripted playbook that calls the same graph tools. The demo never stops working.
FalkorDB quirks. shortestPath has to live in WITH or RETURN, not MATCH. algo.maxFlow needed capacityProperty passed explicitly. Pattern checks inside RETURN gave a type error, so I rewrote them with OPTIONAL MATCH.
A crash I almost shipped. A connection timeout made the Node client emit an error event before I had a listener on it, which killed the whole server. I only found it while testing the "what if a judge forgets to start FalkorDB" case.
A stale clock. Visitor sandboxes copied old timestamps, so a few hours after seeding, the culprit deploy was no longer "recent" and silently disappeared from the answer. New sandboxes now refresh the scenario when they are created.

What I learned

Make the graph the protocol between agents. Shared state, memory and the audit trail all fell out of one design choice.
Small named tools beat raw text-to-Cypher. They are reliable, testable and easy to explain.
Let the graph check the model. A causal path is a much better judge than another prompt.
Store what happened, not just documents. "Nina fixed INC-2041" is an edge, and it is what makes "who should I page?" answerable.

Try it yourself

Open the live app, click Investigate, open the Why? tab, then try What-if, Containment plan on kafka-events. Or run it locally:

git clone https://github.com/Yashasm18/graphops.git && cd graphops
docker compose up --build

No API key is needed. The agents fall back to their graph playbooks without one.

Built for Graph Hacks: Context for AI Agents by WeMakeDevs and FalkorDB. I used Claude Code as an AI coding assistant while building it, and the agents use Google Gemini at runtime. Every decision you see is backed by FalkorDB.

If you have ever been woken up by a pager, I would love to hear what you would want an agent like this to do next.

DEV Community: Yashas M