Built for Graph Hacks: Context for AI Agents by WeMakeDevs and FalkorDB. Try the live app, read the source, or watch the 2:40 demo:
In November 2022 FDA inspectors walked into an Intas Pharmaceuticals plant in Sanand, Gujarat, and found quality records torn up and thrown in the scrap. The plant made more than half of America's generic cisplatin and carboplatin. Intas stopped shipping, cisplatin went short in February 2023, and by June an NCCN survey found 93% of 27 cancer centres short of carboplatin and 70% short of cisplatin.
Carboplatin is what you reach for when cisplatin runs out. It ran out too, for the same reason: it came out of the same plant.
Cisplatin and its standard substitute both trace back to one Intas plant under FDA import alert.
For Graph Hacks I built Provenance, an agent that checks this before a pharmacist makes the switch. This is how it works, what the data can and can't tell you, and what broke on the way.
Why a graph, and why vector search misses it
Ask a search over drug labels for a cisplatin substitute and it will say carboplatin, because the two labels read alike. That's a fine answer to "what is similar" and a dangerous answer to "what will still be on the shelf next month".
The fact that matters isn't written anywhere as a sentence. It's spread across four public records:
- A DailyMed drug label names the plants that make the product, each with a DUNS number.
- FDA's establishment registry maps that DUNS to an FDA FEI number and a country.
- FDA's Import Alert 66-40 lists plants barred from shipping drugs to the US.
- openFDA's shortage list says what is short today.
The graph model in FalkorDB: four FDA sources joined into one supply graph, plus one memory graph per hospital.
Join them and you get a graph: plants make products, products contain ingredients, ingredients sit in FDA pharmacologic classes, plants carry FDA actions. "Does this substitute share a plant with the drug that ran out" becomes one Cypher pattern in FalkorDB:
MATCH (i:Ingredient {name: $drug})-[:IN_CLASS]->(:Class)<-[:IN_CLASS]-(alt:Ingredient)
OPTIONAL MATCH (i)<-[:CONTAINS]-(:Product)<-[:MAKES]-(f:Facility)-[:MAKES]->(:Product)-[:CONTAINS]->(alt)
OPTIONAL MATCH (f)-[:FLAGGED]->(a:Action)
RETURN alt.name, collect(DISTINCT f.name), collect(DISTINCT a.date)
For cisplatin it returns carboplatin with three shared plants, Intas at FEI 3004011473 among them, flagged on 1 June 2023. It also turned up something I didn't know going in: a second Intas site in the same town makes cisplatin too, and it has its own import alert, from November 2023.
What the app shows for cisplatin: seven plants, three shared with carboplatin, two Intas sites under import alert.
The build
For data, I scoped to the 20 drugs with the most current shortage entries plus the three platinum drugs, then the members of their FDA classes as candidate substitutes: 75 substances, 6,251 products, 3,828 labels. Every label was read. The whole graph fits in memory with room to spare, which matters because FalkorDB keeps the graph in RAM.
The supply graph: 670 plant-drug links. Amber drugs are on today's FDA shortage list; red plants are under import alert.
To make the comparison fair, the "similar text" baseline also runs in FalkorDB: each drug's indications text is embedded with bge-small and stored in a FalkorDB vector index, and the baseline picks the nearest neighbour. Same database, same drugs, two ways of asking.
The agent's tools are Cypher queries: one check_substitute call returns the plants behind a drug, the plants it shares with a proposed substitute, any import alerts on them, and the same-class options with no shared plant. Others cover plant impact, the bridging plants and hospital memory. On the live demo, GPT-4.1 mini picks the tools through function calling, capped at 200 model calls a day; with no key, or past the cap, a rule-based planner runs the same tools. The system prompt says one thing above all: never name a plant, FEI number or date a tool didn't return. The page streams each tool call with its Cypher as it returns, then the answer as it's written, so rows show up in under half a second.
The trap query running live on FalkorDB Cloud: five rows in 27 ms.
FalkorDB's built-in algorithms do the systemic part. I project the graph to plant-drug edges in both directions, so every drug-to-drug path runs through the plants that make them, then run algo.betweenness on it. Plants with high betweenness tie separate drug supply chains together: Fresenius Kabi's FEI 3001833549 makes 15 of the graph's drugs across 14 classes. Intas ranks 33rd, a single point of failure for platinum drugs but not a bridge between classes. Two surprises: the algorithm samples 32 start nodes by default, so I set the sample to every node for exact scores; and FalkorDB 4.22 and 6.0.1 return different exact scores on the same 671 edges, with the same top three plants in a different order.
For memory, each hospital is its own graph on the same FalkorDB server, hospital:<name>, holding what it stocks and every substitution it decided. When a pharmacist says "we switched cisplatin to oxaliplatin", the agent writes a Decision node, links the old decision to the new one with REPLACED_BY, and points out that oxaliplatin also comes out of the Intas plant. When an FDA action hits a plant, each hospital's graph is read against it, so the same event lands as three different alerts: chemo drugs at the oncology centre, a sedative at the surgical hospital, a diuretic at the children's hospital.
Results
Across the 18 drugs on today's shortage list that the graph covers, the similar-text substitute shared a manufacturing plant with the drug in shortage 14 times. Six times it was itself on the shortage list. For 15 of the 18, the graph found a same-class drug with no shared plant.
Measured across the 18 drugs on today's FDA shortage list.
For dextrose, dopamine, hydromorphone and lorazepam, the similar-text pick shared no plant at all. For carboplatin, dextrose and rocuronium, every same-class alternative shares a plant with the drug, so the honest answer is "there is no independent option in this class".
Then I scored the agent the same way. A script asks the live agent one plain question per shortage drug and checks the answer against FalkorDB. In 18 of 18 answers it cited no FEI number or alert date the graph doesn't hold. In 17 of 18 it named every flagged plant behind the drug with its date; the miss said "in 2023" for clonazepam instead of the two dates. Median answer: 2.3 seconds.
What broke
Repackers looked like factories. Labels list repackers and relabellers next to manufacturers. My first count said 94% of similar-text picks shared a plant, and most of that was repackers like Bryant Ranch Prepack. I split the edge into MAKES (manufacture, sterilise) and HANDLES (pack, relabel, test). The real number is 78%.
One firm had seven sites. My first matcher attached both Intas import-alert entries to one plant. They're two sites with two dates. The matcher now scores every registered site of a firm by shared address words and keeps the best. A test pins it.
DailyMed cut me off at label 1,739. The fetcher now paces requests, honours Retry-After, and records every response so a rerun resumes from where it stopped.
The model skimmed past the one fact that mattered. The first agent eval caught GPT-4.1 mini telling a pharmacist that no plant behind furosemide was under import alert. Both Intas sites make furosemide. The flagged plant sat in the middle of a 27-row list and the model read past it. check_substitute now returns flagged plants as their own field, and the prompt requires each one with its date. The smaller GPT-4.1 nano was worse: it invented import alerts on midazolam's substitutes, so it's out.
FalkorDB 4.22 dropped a hop. On the cloud instance, OPTIONAL MATCH (q)<-[:MAKES]-(bad:Facility)-[:FLAGGED]->(:Action) bound every plant that makes the product, flagged or not, so clobazam showed 11 plants under import alert when it has none. 6.0.1 gets it right. Moving the second hop into WHERE (bad)-[:FLAGGED]->(:Action) works on both, and a test pins it. The minimal repro is in the README.
Every new connection to the cloud cost 410 ms. Opening a fresh FalkorDB Cloud connection per query made a simple answer take 2.1 seconds. Now one pooled connection opens at startup and the agent's tool calls run in parallel, and the same answer takes about 70 ms locally.
There is no FalkorDB 6 Docker image yet. The Rust rewrite ships as a module, while falkordb/falkordb:latest is 4.22.1. CI now runs every test on both, and the eval numbers match.
What the data can't tell you
Labels name the plant that makes the finished drug, not the plant that makes the active ingredient, and many shortages start at the ingredient. FDA doesn't publish market share either, so "share of US labels" undersells Intas badly: public labels show it making 2 of 19 cisplatin products, while it supplied over half of US cisplatin. And "same class" is a pharmacology grouping, not a statement that two drugs are interchangeable for a patient. A pharmacist decides.
What I learned
The most useful fact in this project lives between records. No FDA document says carboplatin and cisplatin share a factory; you only see it by walking from a label to a plant to a registry entry to an import alert. That walk is what a graph is for, and it's why vector search, which reads one document at a time, kept picking the risky substitute.
Cleaning the data turned out to be modelling. The repacker fix lives in the graph itself: "makes this drug" and "puts a label on this drug" became different edges. Once the graph said what each plant actually did, the numbers became honest.
Score the AI against the graph. The demo looked right every time I tried it by hand. A script that asked about all 18 shortage drugs and checked each answer against FalkorDB found the furosemide error in its first run.
Test on the engine you deploy on. My tests passed on FalkorDB 6.0.1 locally while the cloud instance on 4.22 gave a different answer to the same query. CI now runs both.
Try it
Live app: https://provenance-imzk.onrender.com (free hosting, so the first visit can take a minute to wake). Pick cisplatin, then click the red Intas plant in the chain.
Code, data snapshot, the Cypher and every number above: https://github.com/tanayvasishtha/provenance
Top comments (0)