The problem shape
Clinical trial patient matching is a good case study in a class of problem that shows up constantly and gets modelled wrong: matching prose against prose, where the cost of a false positive and a false negative are wildly different.
A trial protocol's eligibility section says things like:
No prior systemic therapy for metastatic disease. ECOG performance status 0–1. Adequate hepatic function.
Those are conditions, not fields. "Adequate hepatic function" expands into lab thresholds defined elsewhere in the same 40-page document. Meanwhile the evidence that answers them is split across structured data (labs, diagnosis codes, meds) and clinician notes:
...tolerated first-line well, though notes ECOG appears stable at 1. Path report pending re-review.
That sentence probably answers a criterion. No SQL query finds it.
Why "just embed everything" underperforms
The tempting move is to embed the protocol and the chart and check similarity. It works badly, for a specific reason: eligibility is conjunctive and includes negations. A patient who matches 9 of 10 criteria and fails the 10th is not a 90% match, they're ineligible. Cosine similarity has no opinion about that.
What works better is decomposing the protocol into individual criteria first, then evaluating each one separately against retrieved evidence. Slower, more calls, dramatically more useful — because now you can say which criterion failed, which is the thing a human actually needs.
Unknowns are first-class results
The design decision that mattered most for us: a criterion evaluates to one of four states, not two.
-
MET— evidence found supporting it -
NOT_MET— evidence found contradicting it -
UNKNOWN— no evidence either way in the record -
STALE— evidence exists but is outside a clinically relevant window
Most systems collapse UNKNOWN into NOT_MET and silently discard candidates. That's the expensive mistake. "ECOG not documented in the last 90 days" is not a rejection — it's a five-minute task for a coordinator that might surface an eligible patient. Merging unknown into no throws away exactly the cases where a human adds the most value.
Corollary: don't emit a single confidence score. A 0.73 tells a reviewer nothing actionable. A per-criterion breakdown with the source sentence quoted tells them precisely where to look. Scores are for ranking the queue; the breakdown is what makes review possible.
Retrieval is per-criterion, not per-patient
Naive version: retrieve "the patient's record" and evaluate all criteria against it. This blows context and buries evidence.
Better: each criterion drives its own retrieval. The ECOG criterion searches for performance-status mentions; the prior-therapy criterion searches treatment history. Same patient, different retrievals, each scoped to what that condition needs. Costs more calls, and the precision difference is not close.
Asymmetric costs shape the whole design
- False positive: a coordinator spends twenty minutes reviewing someone ineligible. Annoying, recoverable, and it's exactly the job.
- False negative: a patient who might have qualified is never seen by anyone. Invisible, unrecoverable, and nobody ever learns it happened.
So the system should be tuned to over-surface with evidence attached, not to be precise. That inverts the usual instinct to optimize precision, and it's correct here.
The hard boundary
The agent screens; it does not determine eligibility. A coordinator or clinician confirms every candidate, and nothing enrolls anyone. In a research context the audit trail is also load-bearing rather than decorative — an IRB or sponsor asking "how were these candidates identified?" needs a better answer than "the model suggested them."
We build this pattern at IntelliBooks Studio. Happy to get into the criterion decomposition or the four-state evaluation model in the comments.
Top comments (0)