How I combined semantic retrieval, LLM-based eligibility evaluation, PostgreSQL pgvector, and tenant-isolated microservices to build a clinical trial pre-screening platform.
Introduction
Clinical trial matching is not simply a matter of finding patients whose medical records contain similar keywords to a trial protocol.
A patient may have a high semantic similarity score with a clinical trial and still be ineligible because of one critical exclusion criterion. Conversely, a patient whose notes use different terminology may satisfy the actual eligibility requirements.
That distinction became the central engineering challenge behind MedMatch V2.
MedMatch is an AI-assisted clinical trial pre-screening platform designed to help research coordinators identify potentially relevant oncology trials from structured and unstructured patient information.
The goal was to build more than a vector-search application. I wanted to combine semantic retrieval, structured eligibility extraction, LLM-based reasoning, tenant isolation, and asynchronous processing into a complete system.
Important: MedMatch is a research prototype. It uses synthetic patient records only and has not been clinically validated. It is not intended to make autonomous medical decisions or replace clinical judgment.
1. The Architecture
MedMatch V2 uses a microservices architecture with separate authentication and AI services.
The frontend communicates directly with both backend services:
- Spring Boot handles authentication, registration, and JWT issuance.
- FastAPI handles document processing, embeddings, trial retrieval, and eligibility evaluation.
- PostgreSQL with pgvector stores application data and embeddings.
- Redis and Celery support background processing.
- Gemini handles structured trial extraction and eligibility evaluation.
The authentication service signs RS256 JWTs using its private key. The AI service verifies those tokens locally using the corresponding public key, avoiding an authentication-service network request for every AI operation.
Architecture diagram
flowchart TD
UI["React 19 Frontend<br/>TypeScript / Vite"]
AUTH["Spring Boot Auth Service<br/>Java 21 / Spring Security<br/>RS256 JWT Issuer"]
AI["FastAPI AI Service<br/>Python 3.12<br/>JWT Verification"]
PG[("PostgreSQL 17<br/>pgvector")]
REDIS[("Redis 8")]
WORKER["Celery Worker"]
EMBED["SentenceTransformers<br/>all-MiniLM-L6-v2"]
GEMINI["Gemini 2.5 Flash"]
UI -->|"Login / Register"| AUTH
AUTH -->|"Signed JWT"| UI
UI -->|"Bearer token + AI requests"| AI
AI -->|"Tenant-scoped queries"| PG
AI -->|"Dispatch background tasks"| REDIS
REDIS --> WORKER
WORKER -->|"Generate embeddings"| EMBED
EMBED -->|"Store vectors"| PG
WORKER -->|"Protocol criteria extraction"| GEMINI
AI -->|"Eligibility evaluation"| GEMINI
Technology stack
| Layer | Technology |
|---|---|
| Frontend | React 19, TypeScript, Vite 8 |
| Styling | Tailwind CSS 4 |
| Authentication | Java 21, Spring Boot 4.1, Spring Security |
| Token security | Nimbus JOSE + JWT, RS256, BCrypt |
| AI backend | Python 3.12, FastAPI, Pydantic v2 |
| LLM | Google Gemini 2.5 Flash |
| Embeddings | SentenceTransformers, all-MiniLM-L6-v2 |
| PDF processing | PyMuPDF |
| Database | PostgreSQL 17, pgvector |
| Background processing | Redis 8, Celery |
| Migrations | Flyway, Alembic |
| Deployment configuration | Docker, Docker Compose, Kubernetes/Kustomize |
2. Turning Clinical Trial PDFs into Structured Data
Clinical trial protocols are often lengthy documents containing eligibility criteria, treatment information, study objectives, and other protocol details.
The first challenge was converting these documents into data that the matching pipeline could actually use.
The document-processing workflow is:
- A research coordinator uploads a trial protocol PDF.
- The service validates the PDF signature.
- PyMuPDF extracts the document text.
- Gemini 2.5 Flash identifies relevant trial information.
- Pydantic schemas validate the extracted structure.
- The trial and its inclusion/exclusion criteria are stored in PostgreSQL.
The extracted information includes:
- Trial identifiers and descriptions
- Inclusion criteria
- Exclusion criteria
- Relevant clinical conditions
- Treatment-related requirements
One implementation detail worth mentioning: the current PDF pipeline extracts text using PyMuPDF. It does not have an active OCR fallback, so scanned PDFs without an extractable text layer are not supported by the current workflow.
This was an important distinction to preserve. A PDF upload feature does not automatically mean every kind of PDF can be processed.
3. Semantic Retrieval with SentenceTransformers and pgvector
Once trial information is structured, the next challenge is finding relevant trials for a patient.
I used the all-MiniLM-L6-v2 SentenceTransformer model to generate embeddings.
The model produces normalized 384-dimensional vectors for patient notes and trial-related text.
These embeddings are stored in PostgreSQL using pgvector.
For retrieval, the system uses cosine distance:
embedding <=> :query_embedding
The smaller the cosine distance, the closer the vectors are in embedding space.
However, there was an important architectural decision here.
Semantic similarity is used for candidate discovery, not final eligibility determination.
A patient note can be semantically similar to a trial protocol while still violating a strict exclusion criterion.
For example, a patient may have a cancer diagnosis and treatment history that closely resemble a trial's target population. However, progression on a specific treatment may make the patient ineligible under that protocol.
A vector score alone cannot reliably resolve that distinction.
Optimizing candidate retrieval
The retrieval query uses Common Table Expressions (CTEs) and window functions to rank candidate trials and their criteria.
The implementation:
- Filters trials using
hospital_id. - Computes vector distances to identify relevant candidates.
- Ranks criteria within each trial.
- Uses
ROW_NUMBER()to select representative criteria. - Limits the initial candidate set.
- Fetches the complete criteria for selected trials in a subsequent batch.
A simplified representation of the ranking logic:
WITH ranked_trials AS (
SELECT
t.id AS trial_id,
t.hospital_id,
te.embedding <=> :patient_embedding AS distance
FROM trials t
JOIN trial_embeddings te
ON te.trial_id = t.id
WHERE t.hospital_id = :hospital_id
ORDER BY distance, t.id
LIMIT :top_k
),
ranked_criteria AS (
SELECT
rt.trial_id,
tc.id AS criterion_id,
ROW_NUMBER() OVER (
PARTITION BY rt.trial_id
ORDER BY
tc.embedding <=> :patient_embedding,
tc.id
) AS criterion_rank
FROM ranked_trials rt
JOIN trial_criteria tc
ON tc.trial_id = rt.trial_id
)
SELECT *
FROM ranked_criteria
WHERE criterion_rank = 1;
This illustrates the ranking strategy rather than reproducing the entire production query.
The important optimization was to avoid repeatedly loading related criteria while retrieving candidate trials.
4. Eligibility Evaluation with Gemini
After retrieving candidate trials, MedMatch evaluates the patient's information against the complete inclusion and exclusion criteria.
Gemini 2.5 Flash is used for structured eligibility evaluation.
The model receives relevant patient information and the trial's criteria. The output is validated against a structured schema.
The system produces three possible outcomes:
| Status | Meaning |
|---|---|
ELIGIBLE |
Available information supports the eligibility criteria. |
INELIGIBLE |
At least one exclusion or unmet requirement makes the patient unsuitable. |
POSSIBLY_ELIGIBLE |
More information or clinical review is needed. |
The evaluation also includes:
- Confidence information
- Explanations for the result
- Supporting criterion references
- Missing information that could affect the decision
The purpose is not to let an LLM make an unchecked decision. It is to structure the evaluation and make the reasoning easier for a human reviewer to inspect.
Validating LLM responses
One issue with LLM-based workflows is that a model may return incomplete, duplicated, or unexpected results.
For example, if five trials are submitted for evaluation, the service must not silently accept responses for only four trials.
I added explicit validation to ensure that each expected trial receives exactly one evaluation.
The implementation checks:
- Each result contains exactly one trial ID.
- No unexpected trial IDs are returned.
- Duplicate trial evaluations are rejected.
- Missing trial evaluations are detected.
A simplified version of the validation logic:
def _validate_llm_trial_results(
self,
llm_results: list[EligibilityResponse],
expected_trial_ids: set[str],
) -> None:
returned_trial_ids = []
for result in llm_results:
if len(result.trial_ids_evaluated) != 1:
raise ValueError(
"Each eligibility result must correspond "
"to exactly one clinical trial."
)
returned_trial_ids.append(
result.trial_ids_evaluated[0]
)
returned_trial_id_set = set(returned_trial_ids)
unexpected_trial_ids = (
returned_trial_id_set - expected_trial_ids
)
if unexpected_trial_ids:
raise ValueError(
"LLM returned an evaluation for an unexpected "
"clinical trial."
)
if len(returned_trial_ids) != len(returned_trial_id_set):
raise ValueError(
"LLM returned duplicate clinical trial evaluations."
)
missing_trial_ids = (
expected_trial_ids - returned_trial_id_set
)
if missing_trial_ids:
raise ValueError(
"LLM did not return an eligibility evaluation for "
"every retrieved clinical trial."
)
This validation is a small but important part of making an LLM-powered pipeline more predictable.
The model generates the evaluation, but the application remains responsible for checking whether the response satisfies the expected contract.
5. Multi-Tenancy and Role-Based Access Control
Clinical research platforms may serve multiple hospitals or research institutions.
That introduces an important security requirement: one hospital must not be able to access another hospital's data.
MedMatch uses a shared PostgreSQL database with hospital_id as a tenant discriminator on relevant domain records.
The authorization design includes six roles:
SYSTEM_ADMINHOSPITAL_ADMINRESEARCH_COORDINATORPHYSICIANTRIAL_SPONSORPATIENT
The implementation uses multiple layers of protection:
- Spring Security handles authentication and role-based authorization.
- JWTs contain identity, role, and hospital context.
- FastAPI verifies JWT signatures using the RSA public key.
- Repository queries apply hospital-level filters.
- Ownership checks validate that requested resources belong to the authenticated user's hospital.
- Celery task polling also checks tenant ownership.
I treated tenant isolation as a backend responsibility rather than relying on frontend visibility or filtering.
Audit logs are also designed to avoid storing raw patient narrative content.
This is an important distinction: implementing access controls and audit logging does not, by itself, establish regulatory compliance or certification.
6. Asynchronous Processing with Redis and Celery
Document extraction and LLM evaluation can take longer than ordinary API requests.
Keeping every operation synchronous would make the user experience less reliable and increase the risk of request timeouts.
I integrated Redis and Celery to support background processing.
The workflow is:
- The API receives a processing request.
- A background task is dispatched through Celery.
- Redis acts as the message broker and task-state backend.
- A Celery worker processes the task.
- The client polls for task status and results.
- Tenant ownership is checked when retrieving task information.
This separates the API request lifecycle from longer-running AI operations.
The current worker configuration uses a solo pool with concurrency set to one. Higher-concurrency behavior has not yet been load-tested.
7. Performance Optimization
Building the complete workflow was only part of the work. I also wanted to understand where the system was spending time.
One optimization focused on repeated database queries while loading trial criteria.
Reducing database round trips
| Metric | Before | After |
|---|---|---|
| Database queries | 5 | 1 |
| Mean DB loading latency | 121.55 ms | 25.34 ms |
This represents:
- An 80% reduction in database query count.
- Approximately 79.2% lower mean database loading latency.
These measurements concern database loading, not the complete LLM evaluation pipeline. Gemini latency was not measured in this particular experiment.
pgvector retrieval benchmark
I also benchmarked vector retrieval using PostgreSQL sequential scans on the test environment.
| Number of vectors | Retrieval latency |
|---|---|
| 100 | 0.59 ms |
| 1,000 | 0.95 ms |
| 10,000 | 4.67 ms |
| 50,000 | 19.83 ms |
These are database vector-scan measurements, not end-to-end application latency or throughput.
The benchmark used sequential scans. It should not be interpreted as evidence of HNSW or IVFFlat index performance.
8. Testing and Verification
A system that combines authentication, tenant isolation, background workers, database migrations, and LLM calls needs testing at multiple levels.
The backend test results were:
| Test suite | Passed |
|---|---|
| Spring Boot | 85 |
| FastAPI | 116 |
| Integration | 23 |
| Security | 15 |
| Live reliability | 7 |
| Total | 246 |
The reported backend test run completed with zero failures and zero skipped tests.
The frontend was separately checked through:
- TypeScript compilation
- Vite production build
- ESLint
There is no frontend component unit-test runner configured in the current project.
Local deployment verification
The application was also verified using Docker Compose.
The local staging verification included:
- Six active runtime containers passing health checks.
- Sixteen Flyway migrations applied.
- Eight Alembic schema versions applied.
- Eighteen live end-to-end checks passed.
- Five service health checks passed for PostgreSQL, Redis, authentication, AI, and frontend services.
The Celery worker does not expose a native HTTP health endpoint.
Kubernetes manifests were authored and validated using kustomize build. However, the application has not been deployed to a live Kubernetes cluster or cloud environment.
9. Evaluation on Synthetic Data
I evaluated the matching pipeline using a small synthetic oncology dataset.
The evaluation consisted of:
- 30 synthetic patient cases
- 6 clinical trial protocols
The results were:
| Metric | Result |
|---|---|
| Recall@5 | 100% |
| Mean Reciprocal Rank (MRR) | 0.8883 |
| Epistemic reasoner accuracy | 100% |
| Baseline vector classifier accuracy | 36.67% |
| Epistemic false-positive eligibility | 0% |
| Baseline false-positive eligibility | 4.76% |
The results illustrate why separating candidate retrieval from eligibility reasoning can be useful.
However, these numbers must be interpreted carefully.
The evaluation dataset is small and synthetic. It does not establish clinical accuracy, safety, generalization to real patient populations, or readiness for clinical deployment.
Larger datasets, external benchmarks, expert review, and clinical validation would be necessary before drawing those conclusions.
10. What Is Still Missing?
Building a functional prototype also means being honest about what it does not yet support.
The current limitations include:
- No active OCR fallback for scanned PDFs.
- No external clinical-trial benchmark corpus integrated into the evaluation pipeline.
- No real patient data or hospital deployment.
- No clinical validation, IRB approval, or FDA SaMD clearance.
- No live cloud or Kubernetes deployment.
- Prometheus-compatible metrics are exposed, and scrape/alert configuration exists, but Prometheus and Alertmanager servers are not deployed.
- Grafana dashboards and Flower are not implemented.
- Celery concurrency and large-scale workload behavior need further testing.
- Final matching decisions are returned to the application but are not persisted in the
matchestable.
These are not minor details to hide behind a successful demo. They define the next engineering and validation milestones.
11. Key Engineering Lessons
Building MedMatch V2 reinforced several lessons for me.
1. Retrieval and reasoning solve different problems.
Vector search helps identify potentially relevant candidates. It does not replace rule-based or criterion-level eligibility evaluation.
2. LLM output needs application-level validation.
Structured prompts and schemas help, but the application must still validate completeness, uniqueness, and expected identifiers.
3. Multi-tenancy must be enforced below the UI.
Tenant isolation belongs in authentication, authorization, repository queries, and background-task access checks.
4. Performance measurements need clear boundaries.
Database latency, embedding generation time, LLM latency, and end-to-end response time are different metrics. Reporting them separately avoids misleading conclusions.
5. A working prototype is not a production-ready clinical system.
Container health checks and automated tests demonstrate engineering progress. They do not establish clinical validity, regulatory compliance, or real-world safety.
Conclusion
MedMatch V2 started as an attempt to solve a practical problem: helping research coordinators discover potentially relevant clinical trials without relying entirely on manual document review.
It evolved into a system combining:
- PDF-based protocol ingestion
- Structured information extraction
- Semantic retrieval with SentenceTransformers
- PostgreSQL and pgvector
- LLM-based eligibility evaluation
- JWT-based authentication
- Role-based access control and tenant isolation
- Redis and Celery background processing
- Automated testing and deployment verification
The most important takeaway was that building an AI-powered application is not just about integrating an LLM. It is about designing the system around the model, validating its outputs, securing the data, measuring performance, and being transparent about its limitations.
The project is still evolving, but it has been a valuable opportunity to work across AI engineering, backend architecture, database optimization, security, and deployment.
Project Repository
GitHub: https://github.com/snehapriy958/medmatch
AI disclosure: This article was drafted with substantial assistance from ChatGPT. I supplied the project context and repository-backed technical details, reviewed the article against the implementation and verification results, and take responsibility for its published content.
MedMatch is a research prototype that uses synthetic patient records only. It is not a medical device, does not provide medical advice, and is not intended to replace clinical judgment.
Top comments (0)