Catch the Risk Before the First Dose - Building an AI Drug Safety Reviewer That Remembers

Soha patel — Sun, 08 Nov 2026 05:53:55 +0000

💡 In one line: I built TrialSleuth AI, an agent that reviews a new drug before its first human trial and remembers every past drug result, so it gets smarter with each one.

🚀 Try the live demo · 💻 See the code

🧪 The first dose is the scariest moment

Every new medicine eventually reaches a day when it is given to a human for the first time. By then it has passed lab tests and animal studies. Yet people in early trials still sometimes get hurt.

When you read about those cases afterwards, many of them look predictable:

What went wrong	Was it known before?
A similar molecule damaged the liver	✅ Yes, in an earlier drug of the same class
A chemical feature affected heart rhythm	✅ Yes, a well-known warning sign
A common medicine pushed drug levels 2.5× higher	✅ Yes, seen in a past trial
Six volunteers were dosed on the same morning	✅ Yes, a lesson learned the hard way

The knowledge existed. It was just scattered across old reports, papers and people's memories.

So I asked myself: what if a tool could remember every past drug result and check each new drug against all of it before anyone takes the first dose?

📋 What TrialSleuth does

A pharma team describes their drug the way they would in a planning meeting:

🧬 What it is: composition and chemical features
💊 How much: the dose and how it goes up
👥 Who gets it: which patients are allowed in
🗓️ How: the trial design
🔗 With what: which other medicines are allowed
🔬 What we know: lab results so far

TrialSleuth then gives back a clear review:

An overall risk level (High, Medium or Low)
Risks by body system (liver, heart, kidneys…), each with the evidence it used
Suggested changes: lower the starting dose, exclude a patient group, ban an interacting medicine, dose one person first
Tests to run before anyone is dosed
Limits of the review: what memory can't tell us

👉 Click to see a sample TrialSleuth report

Risk level: High - changes needed before human dosing

System	Risk	Why (IDs)
Liver	2–3× higher exposure in Child-Pugh B patients; interaction with strong CYP3A4 inhibitors	HB-02, HB-03, PC-101, PC-102
Heart	Possible hERG block and QT prolongation (aniline + basic amine core)	PC-702, HB-05
Whole group	Six people dosed on the same morning	PC-801

Suggested changes

Exclude Child-Pugh B patients until liver data exists (HB-02)
Ban strong CYP3A4 inhibitors with a 14-day washout (PC-102)
Dose one sentinel participant first and wait before the rest (PC-801)
Run a hERG assay and add ECG monitoring (PC-702)

This is an example of the report format. It flags known risks; it never declares a drug safe.

⚠️ The rule I never broke: TrialSleuth never says a drug is "safe" or "will work". No software can promise that. Its job is to make sure known risks are designed out before human dosing, so experts start from a better plan.

🤖 Why not just ask ChatGPT?

A normal AI chatbot only knows what you paste into the chat, and forgets it when the chat ends. Ask it to review a drug and you get:

"Monitor liver function. Watch for drug interactions."

That sounds sensible, but it's generic. It isn't linked to any real past case, and it never learns.

A real safety expert works differently. They remember what went wrong before, and what fixed it. That's exactly what agent memory gives an AI. If the idea is new to you, Vectorize has a great explainer: What is agent memory?

🧠 Giving the agent a memory with Hindsight

I built the memory layer on Hindsight, an open-source agent memory system by Vectorize. I used Hindsight Cloud with its Python client, and the Hindsight docs got me from zero to a working memory quickly.

The agent uses four simple operations:

Operation	What TrialSleuth does with it
🏦 Create bank	One memory bank, `drug-safety-memory`, told to keep IDs, doses and percentages exact
💾 Retain	Saves body knowledge, past drug outcomes (with the fix that worked), and a summary of every review
🔎 Recall	For each new drug, pulls back the relevant body facts, similar past drugs and earlier reviews
🪞 Reflect	A "What have you learned?" button that sums up lessons across all memory

vectorize-io / hindsight

Hindsight: Agent Memory That Learns

Documentation • Integrations • Cookbook • Benchmarks • Paper • Hindsight Cloud

What is Hindsight?

Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.

hindsight-learning-demo.mp4

It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.

🆕 Introducing Hindsight Models: decision, rerank and embedding models built for agent memory. Open weights on Hugging Face, or hosted on Hindsight Inference. Read the post →

Contents

Memory Performance & Accuracy
Quick Start — server · clients · platforms · embedded
Adding Hindsight to Your Agent — LLM Wrapper · integrations · coding agents · MCP
Core Concepts — memory types · retain / recall / reflect · observations · mental models & knowledge…

View on GitHub

Every memory has an ID, like HB-02 for a body fact or PC-102 for a past drug. The language model (Groq, running openai/gpt-oss-120b) must cite the ID for every claim. That makes each point traceable, and the app shows exactly what was recalled in a side panel, so reviewers can check the evidence themselves.

✨ The moment it clicked

Our demo drug is a made-up cancer medicine called HLV-772. Its plan hides a few quiet problems: patients with weak livers are allowed in, six people are dosed on the same day, and no other medicines are banned.

😶 Memory OFF: a generic answer with no evidence.
🧠 Memory ON: it flagged a liver risk, citing HB-02 (weak livers can mean 2–3× more drug in the blood) and PC-102 (a similar drug where certain common medicines raised levels 2.5×). It suggested banning those medicines before the first dose, which is exactly the fix that worked last time.

Then I taught it one new result: a sister drug that affected heart rhythm in animal tests (PC-702). I ran the same review again and a brand-new ❤️ Heart row appeared, citing PC-702 and asking for ECG monitoring and heart tests before dosing.

🌱 One new result, saved once, and every future review gets better. That learning loop is the whole point.

🛠️ What I learned building it

1. Better memory beats more memory.
At first I sent everything recalled to the model. The prompt grew with every saved review until the model refused it. Recalling a tighter set of the most relevant records fixed the errors and made the reports sharper.

2. Show the evidence.
The side panel of recalled memories became the most convincing part of the app. People trust a review more when they can see what it's based on.

3. Honest framing is a feature.
"Here are the known risks and how to reduce them" is more useful, and far more responsible, than a confident yes or no.

🔭 What's next

Right now the memory holds a small, curated set of example records. A real version would be filled from a company's own trial history, published safety data and regulator reports, with access controls and expert sign-off on every review. I'd also love reviewers to mark which suggestions they accepted, so the agent learns which advice actually helps.

🔗 Links

🚀 Live demo: trialsleuthai.streamlit.app
💻 Source code: github.com/Sohapatel-26/trialsleuth-production
🧠 Hindsight on GitHub: github.com/vectorize-io/hindsight
📚 Hindsight docs: hindsight.vectorize.io
📖 What is agent memory? vectorize.io/what-is-agent-memory

If you're building AI agents that should get better with experience instead of starting from zero every time, give agent memory a try. I'd love to hear what you build! 🙌

DEV Community: Soha patel