This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content
Real-time clinical trial matching and safety verification using the Sanity Context MCP and deterministic GROQ queries.
What I Built
A friend’s dad has stage 4 non-small cell lung cancer. In late September, his first-line carboplatin stopped working, and the clinic oncologist told the family to look for Phase 2 trials on ClinicalTrials.gov.
If you have ever tried navigating ClinicalTrials.gov during a family emergency, you know how painful it is. The website has roughly half a million studies, but whether someone actually qualifies is buried inside massive PDF attachments.
I wanted to see what ChatGPT and Gemini would do with his diagnosis:
58-year-old female, Stage IV NSCLC, EGFR Exon 20 insertion, progressed after carboplatin/pemetrexed, seeking recruiting Phase 2 trials in Texas.
Both models gave me two specific study IDs: NCT04847387 and NCT04746654.
I checked the federal registry. Neither ID existed. The models made up eight-digit numbers on the spot.
Plain keyword search didn't help much either. Searching "EGFR" and "chemo" pulled 69 studies. But when I read the PDF attachments, 31 had strict exclusion clauses banning patients who previously took chemotherapy. Worse, it pulled kidney failure trials because the kidney blood test eGFR has almost the same letters as EGFR.
Cancer trial matching is pure Boolean filtering. You have to match exact alleles, past drug lines, and washout dates. LLMs make up trial numbers, and keyword search misses negative exclusions.
That's why I built TrialMatch using Sanity Content Lake and the Sanity Context MCP. The AI doesn't touch the database. It just pulls four parameters from clinical notes, and our backend passes those parameters into fixed GROQ queries against Sanity schemas and protocol rules.
System Architecture
sequenceDiagram
autonumber
actor Clinician as Clinician / Patient
participant UI as TrialMatch Web App (Next.js 16)
participant Agent as Claude Haiku 4.5 (Schema Extractor)
participant MCP as Sanity Context MCP (trialmatch)
participant Lake as Sanity Content Lake (Project 6xsr2k42)
participant KB as Sanity Knowledge Base (trialmatch-kb)
Clinician->>UI: Enter Patient Note ("58yo NSCLC, EGFR Exon 20, prior chemo, TX")
UI->>Agent: Extract Clinical Primitives
Note over Agent: Converts narrative into:<br/>condition="Lung", biomarker="EGFR",<br/>chemo="ALLOWED", location="TX"
Agent->>MCP: Dispatch GROQ Query with Bound Parameters
MCP->>KB: Check Washout & Hierarchy Rules
KB-->>MCP: Rule Verified (Chemo permitted post-progression)
MCP->>Lake: Execute Deterministic GROQ Filter
Lake-->>MCP: Return 2 Verified Recruiting Protocols
MCP-->>UI: Return Match Set + Full GROQ Audit Trace
UI-->>Clinician: Render Verified Cards + Side-by-Side Hazard Analysis
The Duel Arena in action: Live Sanity Context MCP status beacon, real-time safety scorecard, and 8 clinical scenario presets.
Demo
- Live Application: trialmatch-oncology.vercel.app
- 3-Arm Benchmark Dashboard: trialmatch-oncology.vercel.app/benchmark
- Hosted Sanity Studio: trialmatch-oncology.sanity.studio
The Duel Arena: Side-by-Side Reality
To see the difference in practice, I built the app as a split-screen duel where you can test any custom clinical note or pick from 8 preset patient profiles.
The left column runs the Sanity Context Agent. It displays only active, verified trials that meet all protocol criteria, showing biomarker targets, participating hospital sites, and the exact GROQ query executed against Sanity.
The right column shows naive keyword search results. A red hazard banner highlights disqualifications the text search missed, such as prior chemo exclusions, active liver metastases, or serving mutant-targeted drugs to wild-type patients.
Because keyword searches often return dozens of loose hits while the agent returns 1 or 2 verified matches, each column paginates independently (6 cards per page) so you can review the keyword results without losing your place.
Inspecting the generated GROQ query on the left alongside the naive search failure warnings on the right.
Independent column pagination keeps 69 keyword matches manageable while displaying verified hospital locations.
Code
- GitHub Repository: github.com/IshekKhal/trialmatch
Project Layout
trialmatch/
├── app/
│ ├── page.tsx # The Duel Arena (Sanity Agent vs Keyword Baseline)
│ ├── benchmark/page.tsx # 3-Arm Evaluation Dashboard
│ ├── layout.tsx # Root layout with Geist font tokens
│ ├── globals.css # Medical obsidian styling (#080C14)
│ └── api/
│ ├── duel/route.ts # Live duel execution endpoint
│ ├── benchmark/route.ts # 3-arm benchmark evaluation API
│ └── trials/route.ts # Normalized clinical trial fetcher
├── components/
│ ├── DuelArena.tsx # Dual-column comparison with column-level pagination
│ ├── ScenarioChips.tsx # 8 pre-configured oncology clinical scenarios
│ ├── BenchmarkRunner.tsx # Interactive benchmark runner & case inspector
│ ├── BenchmarkTable.tsx # 10-patient audit breakdown table
│ └── Scoreboard.tsx # Real-time comparative metric display
├── studio/
│ ├── sanity.config.ts # Sanity Studio v3 configuration
│ └── schemas/
│ ├── clinicalTrial.ts # Core schema: biomarkers, therapy rules, facilities
│ ├── protocolRule.ts # Knowledge base schema: exclusion overrides & washouts
│ └── index.ts # Schema registry
├── lib/
│ ├── agent.ts # Claude Haiku 4.5 schema extractor & GROQ builder
│ ├── sanity.ts # Sanity client & live Context MCP bindings
│ ├── protocol_auditor.ts # Protocol-schema clinical safety audit engine
│ ├── eval_cases.ts # 10 gold-standard oncology test profiles
│ ├── eval_runner.ts # Automated 3-arm benchmark execution engine
│ └── types.ts # TypeScript domain interfaces
├── scripts/
│ ├── ingest_trials.ts # ClinicalTrials.gov API v2 data ingestion pipeline
│ └── run_eval.ts # CLI benchmark evaluation runner
└── data/
├── trials_normalized.json # Curated oncology trials across 30 countries
├── eval_results.json # Raw 3-arm benchmark evaluation outputs
└── eval_summary.md # Comprehensive 419-line benchmark report
How I Used Sanity
1. Modeling Oncology Beyond Text Blobs
You can't treat clinical trial protocols like blog posts or raw text fields. A single protocol has specific rules about gene mutations, previous treatments, and organ involvement.
In Sanity Studio (studio/schemas/clinicalTrial.ts), I split each protocol into structured fields. Biomarkers are stored as a typed string array (targetBiomarkers) covering oncogenes like EGFR, KRAS, and BRAF. Prior treatments (priorTherapyRules) use explicit status flags (REQUIRED, ALLOWED, EXCLUDED, ANY) for chemo, immunotherapy, and targeted agents. Trial sites (locations) track facilities across cities, states, and countries, alongside structured eligibilityCriteria.
Clinical Trial Protocol Schema (studio/schemas/clinicalTrial.ts)
import {defineField, defineType} from 'sanity'
export default defineType({
name: 'clinicalTrial',
title: 'Clinical Trial',
type: 'document',
fields: [
defineField({
name: 'nctId',
title: 'NCT ID',
type: 'string',
validation: (Rule) => Rule.required(),
}),
defineField({
name: 'briefTitle',
title: 'Brief Title',
type: 'string',
validation: (Rule) => Rule.required(),
}),
defineField({
name: 'recruitmentStatus',
title: 'Recruitment Status',
type: 'string',
options: {
list: [
{title: 'Recruiting', value: 'RECRUITING'},
{title: 'Active, Not Recruiting', value: 'ACTIVE_NOT_RECRUNTING'},
],
},
}),
defineField({
name: 'primaryCondition',
title: 'Primary Condition',
type: 'string',
validation: (Rule) => Rule.required(),
}),
defineField({
name: 'targetBiomarkers',
title: 'Target Biomarkers',
type: 'array',
of: [{type: 'string'}],
options: {
list: ['EGFR', 'KRAS', 'BRAF', 'HER2', 'ALK', 'BRCA1', 'BRCA2', 'Exon 20', 'G12C', 'V600E'],
},
}),
defineField({
name: 'priorTherapyRules',
title: 'Prior Therapy Rules',
type: 'object',
fields: [
defineField({
name: 'chemotherapy',
type: 'string',
options: { list: ['REQUIRED', 'ALLOWED', 'EXCLUDED', 'ANY'] },
}),
defineField({
name: 'immunotherapy',
type: 'string',
options: { list: ['REQUIRED', 'ALLOWED', 'EXCLUDED', 'ANY'] },
}),
],
}),
defineField({
name: 'locations',
title: 'Trial Locations',
type: 'array',
of: [{
type: 'object',
fields: [
{ name: 'facility', type: 'string' },
{ name: 'city', type: 'string' },
{ name: 'state', type: 'string' },
{ name: 'country', type: 'string' },
],
}],
}),
],
})
A lot of clinical rules have caveats—for example, a protocol might ban prior chemo unless there was at least a 30-day washout period. To handle those edge cases, I set up a second schema called protocolRule.ts in our Sanity Knowledge Base:
Protocol Guidance Rule Schema (studio/schemas/protocolRule.ts)
import {defineField, defineType} from 'sanity'
export default defineType({
name: 'protocolRule',
title: 'Clinical Protocol Interpretation Rule',
type: 'document',
fields: [
defineField({ name: 'ruleId', title: 'Rule ID', type: 'string' }),
defineField({ name: 'title', title: 'Rule Title', type: 'string' }),
defineField({
name: 'category',
type: 'string',
options: {
list: [
{title: 'Eligibility Hierarchy', value: 'ELIGIBILITY_HIERARCHY'},
{title: 'Washout Periods', value: 'WASHOUT_PERIODS'},
{title: 'Biomarker Specificity', value: 'BIOMARKER_SPECIFICITY'},
],
},
}),
defineField({ name: 'priority', title: 'Priority (1-10)', type: 'number' }),
defineField({ name: 'summary', title: 'Rule Summary', type: 'text' }),
defineField({ name: 'clinicalRationale', title: 'Clinical Rationale', type: 'text' }),
],
})
Sanity Studio managing 100 oncology protocols with explicit prior therapy rules, biomarker tags, and multi-center site registries.
2. Sanity Context MCP Endpoints
We hooked the Sanity Content Lake up to two Context MCP endpoints:
-
trialmatch: handles parameterized GROQ queries against the live trials dataset. -
trialmatch-kb: handles semantic search over the protocol rules.
When a clinician enters a note, the agent translates the extracted values into this GROQ query:
*[_type == "clinicalTrial" &&
recruitmentStatus == "RECRUITING" &&
primaryCondition match $condition &&
targetBiomarkers[] match $biomarker &&
(locations[].state match $location ||
locations[].city match $location ||
locations[].country match $location ||
locations[].facility match $location) &&
(priorTherapyRules.chemotherapy == "ALLOWED" ||
priorTherapyRules.chemotherapy == "REQUIRED" ||
priorTherapyRules.chemotherapy == "ANY")] {
nctId,
briefTitle,
phase,
primaryCondition,
targetBiomarkers,
priorTherapyRules,
"matchingLocations": locations[state match $location || city match $location || country match $location || facility match $location]
}
Because priorTherapyRules.chemotherapy is an explicit schema field, Sanity excludes protocols that forbid prior chemotherapy before anything reaches the UI.
Behind the Build: 6 Edge Cases and Engineering Fixes
Real clinical datasets break code in weird ways. Here are the headaches I hit while writing this and how the schema saved the build:
1. The "Chemo-Naive" Substring Collision
Substring bugs almost tanked trial matching right off the bat. Many trials demand a patient be chemo-naive, meaning they never had chemotherapy. My earliest script just ran text.includes("chemo"). Naturally, since "chemo-naive" literally has the word "chemo" sitting right inside it, the parser assumed the patient received prior chemotherapy and threw out the match. Talk about embarrassing. I had to overhaul protocol_auditor.ts so it checks negative prefixes like "no prior" and "chemo-naive" long before evaluating drug strings.
2. The eGFR vs EGFR Alphabet Trap
Then came the alphabet collision: eGFR vs EGFR. EGFR is the lung tumor gene mutation I was hunting for, whereas eGFR is just a routine blood panel test tracking kidney function. Two completely different worlds, but dumb substring search treated them as identical. It started matching kidney disease trials for lung cancer patients. That is obviously unacceptable. Restricting targetBiomarkers to a typed Sanity array killed this bug dead, because an oncology biomarker field simply will not accept metabolic lab acronyms.
3. The React Preset State Leak
React state caused another subtle leak. Clicking a quick preset filled in the search filters nicely. But if a doctor started typing a brand new patient note into the box, those preset tags stuck around in the background payload. Old tags secretly merged with new text. A simple onChange hook on the textarea solved it by purging the active preset the second a user hits a key.
4. Ungrounded LLM Trial ID Drift (21 vs 22 vs 20)
Unconstrained LLMs also drifted in funny ways. Running bare Gemini Flash across ten test cases without a database produced 21 fake NCT IDs on the first attempt, 22 on the second, and 20 on the third. Why? Temperature variance. A single probabilistic roll makes the model drop or tack on an extra bullet. Not one of those IDs existed in real life.
5. Why the Agent is an Extractor, Not a Query Writer
I quickly gave up on letting LLMs write GROQ queries. Whenever you ask a model to produce database queries, it routinely drops exclusion criteria or adds sloppy OR clauses that pull in half the database. We flipped the architecture: the LLM only pulls out four simple parameters, and hardcoded query builders do the rest.
6. The Vercel Serverless Read-Only Filesystem
Finally, serverless hosting gave me headaches on day one. Locally, my benchmark suite was dumping evaluation logs to disk with fs.writeFileSync. Vercel serverless functions, of course, run on read-only filesystems outside /tmp and crashed immediately with 500 errors. I had to rip out disk writes, compute the stats entirely in memory, and stream JSON directly down to the client.
The 3-Arm Benchmark: Empirical Findings
To measure how structured content impacts discovery accuracy, I built an automated evaluation runner (lib/eval_runner.ts) and tested ten gold-standard oncology profiles across three search approaches:
| Evaluation Metric | Arm 1: Structured Sanity Agent | Arm 2: Naive Keyword Search | Arm 3: Bare LLM (Gemini 3.8 Flash) | Notes |
|---|---|---|---|---|
| Medical Precision | 100% | 60% | 0% | Sanity matched safe protocols; other arms included disqualified trials |
| Safety Violations | 0 | 153 | 21 | Keyword search missed negative criteria; Gemini bypassed protocol rules |
| Hallucinated NCT IDs | 0 | 0 | 21 (100% fake) | Raw LLM invented non-existent trial IDs with no registry records |
| Auditability Rate | 100% | 0% | 0% | Sanity logs the exact GROQ query and rule check for every match |
| Avg Returned Trials | 1.2 | 41.5 | 2.1 | Sanity returns 1-2 verified trials instead of flooding the user with 40+ bad hits |
The 3-Arm Benchmark Dashboard showing 100% medical precision and 0 hallucinations for the Sanity agent versus 153 safety violations in keyword search and 100% fake NCT IDs from the bare LLM.
Real Clinical Failure Modes
Here is what happens when you pit these search methods against real clinical profiles:
1. Chemotherapy Exclusion (Case TC-01)
A 58-year-old with EGFR Exon 20 non-small cell lung cancer had already progressed on carboplatin. Keyword search naively spat out 69 studies, but 31 of them (including NCT07799935) explicitly forbid anyone who has had systemic chemotherapy. Meanwhile, ungrounded Gemini hallucinated two fictional studies, NCT04847387 and NCT04746654. Our Sanity agent checked priorTherapyRules.chemotherapy directly in GROQ, immediately tossing out the 31 invalid studies and keeping the only 2 trials that actually permit prior platinum therapy.
2. Liver Metastases Exclusion (Case TC-02)
Organ metastases in TC-02 trip up naive text matching completely. We tested a 62-year-old with KRAS G12C colorectal cancer and active liver metastases. Keyword search flagged NCT05286814 simply because the word "metastases" appeared in the description, totally missing the protocol exclusion disqualifying liver involvement. Gemini just made up two random trial IDs, NCT04793958 and NCT04685141. The Sanity knowledge base caught the liver contraindication in the protocol rules and dropped the trial.
3. HER2 Expression Cutoffs (Case TC-03)
Biomarker cutoffs in TC-03 require strict numerical boundaries. A 51-year-old patient with HER2-low breast cancer (IHC 1+ or 2+) needs trials tailored to low expression. Keyword search dumped 72 trials, mostly high-HER2 protocols like NCT04281641 that strictly demand 3+ overexpression. Raw LLMs hallucinated non-existent trials. Sanity’s schema filtered out the 3+ requirements and matched antibody-drug conjugate studies specifically enrolling HER2-low cohorts.
4. Phase 3 Requirement (Case TC-04)
Phase filtering in TC-04 proves why exact metadata matters. A 47-year-old with Stage IIIC BRAF V600E melanoma specifically needed confirmatory Phase 3 trials in New York. Plain search returned 30 studies, but 26 were early Phase 1 dose-escalation cohorts. Gemini fabricated trials out of thin air. Sanity ran a simple phase == "PHASE3" check, instantly filtering the noise down to the single matching trial.
Detailed case audit: inspecting why trial NCT07799935 violates chemotherapy rules, while Gemini invents fictional identifiers.
Sanity Project Details
-
Project ID:
6xsr2k42 -
Dataset:
production - Hosted Studio: trialmatch-oncology.sanity.studio
-
Sanity GROQ MCP Endpoint:
https://api.sanity.io/v1/context/organizations/oz3yptidu/mcp/trialmatch -
Sanity Knowledge Base MCP Endpoint:
https://api.sanity.io/v1/context/organizations/oz3yptidu/mcp/trialmatch-kb - Public Sample Query (Live JSON): Live 3-Trial Sanity Query Endpoint
Agent Session
Below is a representative excerpt from the agent's interaction loop with Sanity Context MCP:
Clinician:
"We have a 58yo patient with NSCLC harboring an EGFR Exon 20 insertion.
She progressed on carboplatin/pemetrexed. Find recruiting Phase 2 trials
in Texas that permit prior chemotherapy."
Agent (Claude Haiku 4.5):
Extracting clinical parameters:
- Condition: "Lung" (NSCLC)
- Target Biomarker: "EGFR", "Exon 20"
- Chemotherapy: "ALLOWED" or "REQUIRED"
- Location: "TX"
- Recruitment Status: "RECRUITING"
Tool Call -> Sanity Context MCP (trialmatch):
GROQ: *[_type == "clinicalTrial" && recruitmentStatus == "RECRUITING"
&& primaryCondition match "Lung"
&& targetBiomarkers[] match "EGFR"
&& (priorTherapyRules.chemotherapy == "ALLOWED" ||
priorTherapyRules.chemotherapy == "REQUIRED")
&& (locations[].state match "TX" || locations[].facility match "TX")]{nctId, briefTitle, phase}
MCP Response:
[
{
"nctId": "NCT05376891",
"briefTitle": "Phase 2 Study of Targeted EGFR Exon 20 Inhibitor",
"phase": "PHASE2"
},
{
"nctId": "NCT06234137",
"briefTitle": "Targeted Kinase Therapy for Relapsed EGFR Mutations",
"phase": "PHASE2"
}
]
Verification: 2 active matches found. 0 safety exclusions violated. 0 hallucinations.
Why This Matters
A lot of people assume AI accuracy is something you fix by picking a bigger model or spending weeks tweaking system prompts. In healthcare, that doesn't work. A patient either meets trial inclusion criteria or they don't. An 80% accurate search isn't helpful if that remaining 20% margin of error sends a family to a clinic where they get rejected at the door.
Splitting up the responsibilities solved this cleanly. We let Claude do what it handles best: parsing unstructured clinical notes and pulling out core patient parameters. Then we hand those parameters to Sanity to do what it handles best: executing deterministic GROQ queries against structured schemas and rule engines.
Most teams treat structured content as something you only use for marketing blogs and CMS landing pages. But if you are building autonomous agents where a hallucinated number could ruin someone's care plan, having a typed schema is the only thing keeping the model grounded.
Try It Yourself
Explore the live duel or run the benchmark yourself:
- 🌐 Live Arena: trialmatch-oncology.vercel.app
- 📊 Live Benchmark: trialmatch-oncology.vercel.app/benchmark
- 💻 GitHub: github.com/IshekKhal/trialmatch
If you have feedback, ideas for clinical trial filtering, or experience with Sanity MCP in production, I would love to hear your thoughts in the comments below!
Top comments (1)
I built this because flat keyword search in clinical trials leads to dangerous safety blind spots. How are you handling deterministic guardrails for LLM agents in your own production architectures?