DEV Community

Neelam Geethika Reddy
Neelam Geethika Reddy

Posted on Edited on

How I Built TrialSleuth AI: Simulating Clinical Trial Safety Before Exposing Human Participants

#ai #python #llm #streamlit

TL;DR: TrialSleuth AI reviews a new drug candidate before its first human trial. It remembers how the human body handles drugs, past trial outcomes, and safety bounds using Hindsight agent memory. Every new lab result taught to the agent makes future reviews smarter and keeps human trial participants safe.

🔗 Try it: trialsleuthai.streamlit.app · Code: GitHub

The problem, in plain words

Before a new medicine reaches the public, it is tested on volunteers in clinical trials. Labs and animal studies come first, but trial participants can still suffer preventable adverse events.

The sad part? Many of those safety risks were already seen before:

  • A similar drug compound had already caused drug-induced liver injury.
  • A specific chemical structure was known to block hERG channels and alter heart rhythm.
  • A common co-administered drug was known to raise exposure levels dangerously high.

The trial data existed, but it was buried across hundreds of dense medical pages, amendment logs, and expert heads.

What if an AI could remember all of that, stress-testing every new trial protocol before anyone takes the first dose?

Why standard LLMs risk patient safety

A regular stateless AI chatbot has no long-term memory. It evaluates context turn-by-turn, suffering from context decay across 100+ page protocol documents. When evaluating complex criteria, it generates generic advice like "monitor the liver" without retaining exact numerical bounds or historical drug contraindications.

A clinical safety auditor works differently: they remember what happened in past trials, exact lab cutoffs, and what changes fixed the issue.

That is exactly what persistent agent memory unlocks for AI agents. (Vectorize has a detailed overview: What is agent memory?)

How TrialSleuth AI works

TrialSleuth AI Interface: Pre-trial drug safety review using Hindsight Memory

Figure 1: TrialSleuth AI auditing candidate HLV-772 against recalled human body knowledge (HB-03 Liver) and earlier safety reviews stored in Hindsight memory.

TrialSleuth AI acts as a pre-flight safety audit layer across four core steps:

Step What happens
1. Submit The team inputs drug details: active composition, dosing strategy, patient eligibility, trial design, allowed medications, and preliminary lab data.
2. Remember Hindsight recalls matching facts about human physiology (e.g., HB-03), past drug outcomes (PC-102), and earlier reviews.
3. Review The model writes a safety report: risks categorized by body system, required protocol changes, and mandatory pre-dosing tests—citing every memory ID used.
4. Learn When a new lab result comes in, it is saved once to memory, making every subsequent review smarter.

One non-negotiable design rule: the agent never declares a drug "safe". Its role is to highlight hidden risks early so human safety experts can fix the trial protocol before participants are exposed.

Enforcing safety bounds with Hindsight memory

I integrated Hindsight, an open-source memory system for AI agents by Vectorize. It provides four fundamental operations:

  • Create a memory bank: A dedicated storage unit for trial safety protocols.
  • Retain: Persist critical facts, such as past drug toxicity data or lab cutoffs.
  • Recall: Fetch relevant safety constraints dynamically during a drug review.
  • Reflect: Synthesize lessons learned across historical records.

Here is a simplified Python implementation from our codebase:

Here is a simplified Python implementation from our codebase:

Here is a simplified Python implementation from our codebase:

import os
from hindsight_client import Hindsight

client = Hindsight(
    base_url=os.getenv("HINDSIGHT_URL"),
    api_key=os.getenv("HINDSIGHT_API_KEY")
)

# 1. Retain a past trial safety outcome and the fix that worked
client.retain(
    bank_id="drug-safety-memory",
    content="Past drug outcome PC-102: KRS-110 caused grade-3 liver injury at 800mg. Change that worked: cap dose and ban CYP3A4 inhibitors.",
    tags=["past-drug", "liver-toxicity"],
    timestamp="2025-06-03"
)

# 2. Recall safety rules relevant to a new drug candidate before trial setup
results = client.recall(
    bank_id="drug-safety-memory",
    query="Safety risks for HLV-772, an oral KRAS inhibitor evaluated in patients with liver impairment...",
)

# 3. Reflect across memory banks to extract safety takeaways
lessons = client.reflect(
    bank_id="drug-safety-memory",
    query="What critical safety precautions must be enforced for lipophilic basic amine compounds?",
)
Enter fullscreen mode Exit fullscreen mode

The Tech Stack:

  • 🧠 Memory Engine: Hindsight Cloud (Vectorize)
  • 💬 Language Model: Groq (openai/gpt-oss-120b)
  • 🖥️ Interface: Streamlit
  • 🐍 Language: Python

Impact & Results: Memory ON vs. Memory OFF

TrialSleuth AI Safety Report with Hindsight Memory Citations

Figure 2: TrialSleuth AI generating an explicit safety report flagging liver injury risks and citing recalled memory IDs (HB-03, HB-02, PC-101).

Our demonstration evaluates a hypothetical cancer drug candidate, HLV-772. Its initial protocol hides significant risks: enrolling patients with moderate liver impairment, simultaneous cohort dosing, and no restrictions on concomitant medications.

  • With Memory OFF: The LLM output was generic, recommending vague monitoring without citing specific risk thresholds or prior trial data.
  • With Memory ON: The agent flagged a High Risk level for liver toxicity, backing it up with recalled memory IDs:
    • HB-02: Patients with Child-Pugh B liver impairment experience 2–3× higher drug exposure.
    • PC-102: A past sister compound caused grade-3 liver injury when combined with strong CYP3A4 inhibitors.

Then, we taught the agent one new result: animal study data showing a sister compound (PC-702) caused hERG channel blocking and QT prolongation. When re-running the evaluation, a new Heart risk category appeared automatically, citing PC-702 and mandating baseline ECG monitoring before human dosing.

Key Takeaways

  1. Focused recall beats prompt dumping: Recalling only high-relevance safety memories prevents context dilution and keeps LLM reasoning sharp.
  2. Citation builds trust: Displaying recalled memory IDs (HB-03, PC-102) alongside safety recommendations allows clinical teams to verify the reasoning behind every warning.
  3. AI as a pre-trial sandbox: Simulating trial risks in software provides a vital layer of protection before human participants are ever dosed.

Links & Resources

Top comments (0)