Multi-Collection Vector Search for Clinical Data

You are testing your clinical search system and ask it to find patients with lab profiles similar to a reference case for chronic kidney disease. The results come back with full discharge summaries and progress notes that mention creatinine only in passing. But not a single lab record appears in the top ten. This happens because of how the system is built.

If you use just one embedding model trained on general medical text, it cannot distinguish between a clinical note that says “elevated creatinine” and a lab record showing a creatinine level of 4.2 mg/dL. In a vector space, where data is turned into arrays of numbers, both end up close together because they use similar words. But one is a doctor’s note, and the other is an actual measurement, and your search system can’t tell them apart.

This article builds a multi-collection vector search system for clinical data using VectorAI DB, FastAPI, and Docker. By the end, you will have six collections, one per clinical data type, queried in parallel through a single FastAPI gateway that runs entirely on-premises, with no patient data leaving your infrastructure

Why Clinical Data Breaks Single-Collection Search

When teams build a vector search system for clinical data, they often start with a single collection and a single embedding model. In an EHR system, though, different data types have different structures, vocabularies, and search patterns.

Data type	Example record	What breaks in a shared collection
Clinical notes	Discharge summary: "Patient presented with chest pain radiating to the left arm. EKG showed ST elevation."	Notes dominate search results because they contain the most natural language. Other data types are buried.
Imaging records	Radiology report metadata: modality CT, body region thorax, finding "2.3 cm pulmonary nodule", associated image stored in a Picture Archiving and Communication System (PACS)	Imaging metadata fields such as modality and Digital Imaging and Communications in Medicine (DICOM) Study UID do not belong in a schema designed for free-text notes.
Lab results	Creatinine: 4.2 mg/dL, reference range 0.7 to 1.3, collected 2027-01-15, ordering provider: nephrology	Numeric values with units are poorly suited to text embedding models. A creatinine result of 4.2 mg/dL and one of 0.8 mg/dL may land close together in the embedding space because the surrounding text looks similar, even though clinically they are very different.
Medication records	Metformin 500mg, twice daily, start date 2025-03-01, prescriber: endocrinology, RxNorm (a standardized vocabulary for medications) code 860974	Drug names share vocabulary with clinical notes and discharge summaries. Medication searches return narrative mentions instead of actual medication history.
Structured EHR	International Classification of Diseases, 10th revision (ICD-10) code E11.9 (Type 2 diabetes without complications), encounter date 2026-04-15, visit type: outpatient	Coded data in ICD-10 and SNOMED carry meaning only within their own vocabulary systems. General-purpose embedding models lack a reliable way to represent that structure, so coded records yield poor similarity results when mixed with free text.
Knowledge base	PubMed abstract on chronic kidney disease management, institutional guideline on antibiotic prescribing, clinical reference entry on drug-drug interactions	Without its own collection, clinical reference material mixes with patient records, making it impossible to separate evidence-based citations from patient-specific data at query time.

If you put all six data types into one collection, your embedding models end up handling tasks they were not built for. Clinical notes take over because they have the most natural language. Lab values, medication records, and coded diagnoses get lost because numbers and coding systems like ICD-10 and SNOMED do not integrate well with unstructured text. As a result, search accuracy drops for every data type except the one that best matches the model.

Lee et al. showed this in npj Digital Medicine. In a study of over 400,000 emergency department visits, they found that embedding EHR data domains separately before combining them performed better than using a single mixed-embedding model for every clinical prediction task. If you combine everything into one set of vector embeddings, you lose the unique value of each data type. Using a better embedding model will not fix this, but separating your collections will.

Multi-Collection Search

With multi-collection vector search, each clinical data type has its own separate collection and embedding model. When you send a search query, it is turned into a query vector and sent to all the relevant collections at once. Each collection searches independently using Approximate Nearest Neighbor (ANN) algorithms, which quickly narrow down results from millions of records.

This architecture splits clinical data across six collections, each designed for a specific data type:

• clinical_notes: discharge summaries, progress notes, and consult notes
• imaging_records: radiology and pathology report metadata; image files stay in a separate PACS system
• lab_results: numeric lab values, reference ranges, and Logical Observation Identifiers Names and Codes (LOINC) codes
• medications: medication names, doses, durations, and RxNorm codes
• ehr_structured: coded diagnosis and procedure records using ICD-10 and SNOMED (Systematized Nomenclature of Medicine Clinical Terms) codes
• knowledge_base: clinical reference content from PubMed and institutional guidelines

Since each collection has its own embedding model, you cannot directly compare similarity scores between collections. To solve this, the system merges results using Reciprocal Rank Fusion (RRF). RRF ranks items based on their positions in each result list, not on their raw scores, and produces a single combined ranking.

But before you deploy this architecture in a clinical environment, there is a regulatory layer you need to understand.

The On-Premises Compliance Requirement

HIPAA, the Health Insurance Portability and Accountability Act, is the United States federal law that governs the privacy and security of health data. Any data that can identify a patient and relates to their health condition, treatment, or payment is Protected Health Information (PHI), including names, dates of birth, diagnoses, lab results, and prescriptions.

In this architecture, the vector database, embedding service, and API gateway all store, process, or transmit PHI, which means each component falls under HIPAA's Security Rule. Three requirements affect this architecture directly:

• Data residency: PHI cannot be sent to a cloud service without a Business Associate Agreement (BAA), a contract that makes the vendor legally responsible for protecting PHI under HIPAA. Sending PHI to a cloud-hosted vector database without a BAA in place is a HIPAA violation.
• Encryption: Under the current HIPAA Security Rule, encryption of PHI at rest is an addressable specification, meaning covered entities must assess whether it is reasonable and appropriate for their environment. The 2025 Notice of Proposed Rulemaking (NPRM) proposes making encryption mandatory for all systems that access PHI. As of this writing, that requirement is still proposed.
• Audit logging: Under 45 CFR section 164.312(b), all PHI access must be logged. Under 45 CFR section 164.316(b)(2)(i), these logs must be retained for six years.

VectorAI DB runs on-premises fully using Docker, so PHI always stays within your own infrastructure. The database uses AES-256 encryption for data at rest and lets you manage your own encryption keys. This removes the need for a BAA at the vector database layer itself. Any external services or managed infrastructure that handles PHI elsewhere in your stack will still require a BAA and a compliance review.

Check with your organization’s compliance and legal teams before making any architectural decisions based on these requirements.

The Full Stack

The system has five components:

• VectorAI DB: Stores all six clinical collections on your own servers and handles similarity search using Hierarchical Navigable Small World (HNSW) indexing, a graph-based structure that makes ANN search fast at scale
• FastAPI: Receives search requests from clinical applications, validates them, checks caller permissions, and forwards the query gRPC: The communication protocol between FastAPI and VectorAI DB; uses a binary format for internal service communication
• Docker: Packages VectorAI DB and the FastAPI service into isolated, reproducible containers that run consistently on any server
• Embedding service: Converts incoming queries and clinical records into vector embeddings locally: no external API calls, no PHI leaves the server

The recommended hardware is Ubuntu 22.04 LTS, 16 GB of RAM, an 8-core CPU, and a 200 GB NVMe SSD. A GPU can speed up embedding generation, but it is not needed for search.

Note: This tutorial covers only dense vector search. Adding hybrid search, which mixes dense ANN search with keyword matching, can improve recall for queries with specific drug names, LOINC codes, or ICD-10 codes. Once your base system works, this is the next recommended step. Check out the VectorAI DB documentation for details on hybrid search.

Setting Up the Collections

Before you begin, make sure Docker Desktop is installed. Install Python 3.10 or later, and confirm that pip is available.

Project structure

Your project will have the following structure, which keeps the collection setup, ingest, search, and audit steps separated by responsibility:

clinical-search/
├── docker-compose.yml
└── src/
    ├── clinical_collections.py
    ├── ingest.py
    ├── search.py
    ├── audit.py
    ├── evaluate.py
    └── main.py

Stage 1: Start VectorAI DB with Docker

Create a docker-compose.yml file in your project root:

services:
  vectorai:
    image: actian/vectorai:latest
    container_name: vectorai
    ports:
      - "6573:6573"
      - "6574:6574"
      - "6575:6575"
    volumes:
      - ./local_data:/var/lib/actian-vectorai
    environment:
      - ACTIAN_VECTORAI_ACCEPT_EULA=YES
    restart: unless-stopped

Start the container:

docker compose up -d

Confirm it is running:

docker ps --filter name=vectorai --format 'Status: {{.Status}}  Ports: {{.Ports}}'

Open http://localhost:6575 in your browser. A System Health of 100% confirms that the database is ready.

Stage 2: Connect with VectorAIClient

Install the Python SDK:

pip install actian-vectorai-client

Verify the connection:

from actian_vectorai import VectorAIClient

with VectorAIClient("localhost:6574") as client:
    info = client.health_check()
    print(f"Connected: {info}")

Expected output:

Connected: {'title': 'Actian VectorAI DB', 'version': 'Actian VectorAI DB 1.0.1 / VDE 1.0.1'}

Stage 3: Create the six collections

Each collection uses 768-dimensional vectors with cosine similarity, includes a primary key field, and carries five compliance metadata fields: patient_id, site_id, encounter_id, data_sensitivity_class, and created_at.

These fields enable HIPAA-compliant filtering and audit logging across every collection. This dense-only schema is enough for storing dense vectors today. If you later extend to multi-vector hybrid search, it can also support additional vector fields, raw text descriptions, and sparse-vector fields for full-text search.

Set up src/clinical_collections.py to define and initialize all six collections:

from actian_vectorai import VectorAIClient
from actian_vectorai.models import VectorParams, Distance

HOST = "localhost:6574"

COLLECTIONS = [
    {"name": "clinical_notes", "size": 768},
    {"name": "imaging_records", "size": 768},
    {"name": "lab_results", "size": 768},
    {"name": "medications", "size": 768},
    {"name": "ehr_structured", "size": 768},
    {"name": "knowledge_base", "size": 768},
]

def create_collections():
    with VectorAIClient(HOST) as client:
        for col in COLLECTIONS:
            try:
                client.collections.create(
                    col["name"],
                    vectors_config=VectorParams(
                        size=col["size"],
                        distance=Distance.Cosine,
                    ),
                )
                print(f"Created: {col['name']}")
            except Exception as e:
                print(f"Skipped {col['name']}: {e}")

if __name__ == "__main__":
    create_collections()

All six collections are now visible in the Local UI under Collections.

Stage 4: Ingest a sample clinical record

The embedding service generates vector embeddings from each record's text before insertion. In production, you should use your own clinical embedding model instead of get_mock_embedding. You can also use precomputed dense vectors if embeddings are produced upstream. The client.points.upsert method adds each record to its collection, creating a new entry if one does not exist or updating it if one does.

Add src/ingest.py with a mock embedding function and a synthetic record for each collection.

from actian_vectorai import VectorAIClient, PointStruct
import hashlib

HOST = "localhost:6574"

def get_mock_embedding(text: str, size: int = 768) -> list[float]:
    hash_bytes = hashlib.sha256(text.encode()).digest()
    return [((hash_bytes[i % len(hash_bytes)] / 255.0) * 2 - 1) for i in range(size)]

SYNTHETIC_RECORDS = [
    {
        "collection": "clinical_notes",
        "id": 1,
        "text": "Patient TEST-001 presented with elevated creatinine levels and reduced urine output. Assessment consistent with acute kidney injury stage 2. Recommended nephrology consult and IV fluid resuscitation.",
        "payload": {
            "patient_id": "TEST-001",
            "site_id": "DEMO-SITE",
            "encounter_id": "ENC-2027-001",
            "data_sensitivity_class": "PHI",
            "created_at": "2027-01-15T09:30:00Z",
            "note_type": "progress_note",
            "author": "DR-DEMO-001",
        }
    },
    {
        "collection": "lab_results",
        "id": 2,
        "text": "Creatinine 4.2 mg/dL reference range 0.7-1.3 mg/dL LOINC 2160-0 status abnormal critical",
        "payload": {
            "patient_id": "TEST-001",
            "site_id": "DEMO-SITE",
            "encounter_id": "ENC-2027-001",
            "data_sensitivity_class": "PHI",
            "created_at": "2027-01-15T08:00:00Z",
            "loinc_code": "2160-0",
            "value": 4.2,
            "unit": "mg/dL",
            "reference_range": "0.7-1.3",
            "status": "critical",
        }
    },
    {
        "collection": "medications",
        "id": 3,
        "text": "Metformin 500mg twice daily oral type 2 diabetes management RxNorm 860974",
        "payload": {
            "patient_id": "TEST-001",
            "site_id": "DEMO-SITE",
            "encounter_id": "ENC-2027-001",
            "data_sensitivity_class": "PHI",
            "created_at": "2027-01-15T09:00:00Z",
            "rxnorm_code": "860974",
            "dose": "500mg",
            "frequency": "twice daily",
            "route": "oral",
        }
    },
    {
        "collection": "ehr_structured",
        "id": 4,
        "text": "ICD-10 E11.9 type 2 diabetes without complications outpatient encounter",
        "payload": {
            "patient_id": "TEST-001",
            "site_id": "DEMO-SITE",
            "encounter_id": "ENC-2027-001",
            "data_sensitivity_class": "PHI",
            "created_at": "2027-01-15T09:00:00Z",
            "icd10_code": "E11.9",
            "description": "Type 2 diabetes mellitus without complications",
            "encounter_type": "outpatient",
        }
    },
    {
        "collection": "imaging_records",
        "id": 5,
        "text": "CT abdomen pelvis contrast enhanced bilateral renal cortical thinning consistent with chronic kidney disease no acute obstruction",
        "payload": {
            "patient_id": "TEST-001",
            "site_id": "DEMO-SITE",
            "encounter_id": "ENC-2027-001",
            "data_sensitivity_class": "PHI",
            "created_at": "2027-01-15T10:00:00Z",
            "modality": "CT",
            "body_region": "abdomen_pelvis",
            "finding": "bilateral renal cortical thinning",
            "pacs_reference": "PACS-DEMO-001",
        }
    },
    {
        "collection": "knowledge_base",
        "id": 6,
        "text": "Acute kidney injury staging KDIGO criteria stage 2 creatinine 2-2.9x baseline or urine output less than 0.5 mL/kg/h for 12 hours management includes nephrology consult fluid resuscitation and nephrotoxin avoidance",
        "payload": {
            "patient_id": "NONE",
            "site_id": "DEMO-SITE",
            "encounter_id": "NONE",
            "data_sensitivity_class": "PUBLIC",
            "created_at": "2027-01-01T00:00:00Z",
            "source": "KDIGO_Guidelines_2024",
            "topic": "acute_kidney_injury",
        }
    },
]

def ingest_records():
    with VectorAIClient(HOST) as client:
        for record in SYNTHETIC_RECORDS:
            vector = get_mock_embedding(record["text"])
            point = PointStruct(
                id=record["id"],
                vector=vector,
                payload=record["payload"],
            )
            client.points.upsert(record["collection"], [point])
            print(f"Ingested record {record['id']} into {record['collection']}")

if __name__ == "__main__":
    ingest_records()
    print("All synthetic records ingested successfully.")

Open the clinical_notes collection in the Local UI to confirm the record was ingested and the metadata fields are visible.

Note: When you move to production, replace get_mock_embedding with a domain-specific model suited to each collection. ClinicalBERT works well for clinical_notes and imaging_records, PubMedBERT for lab_results and knowledge_base, and SapBERT for medications and ehr_structured. Each model is available via Hugging Face Transformers.

Building the Search Gateway

The FastAPI gateway handles all clinical search requests. When a query comes in, the gateway checks it, embeds the query text, sends the resulting vector to each target collection via gRPC, and then merges the results with RRF before returning a ranked response.

Each collection search call accepts only one query data item, so separate search requests are needed for each target collection.

Install the dependencies:

pip install fastapi uvicorn pydantic

Start with the function that searches all requested collections in turn. It converts the incoming query into a vector embedding, runs the search against each collection sequentially, and merges the results using RRF.

Add src/search.py with the multi-collection search function and the result formatter:

from actian_vectorai import VectorAIClient, reciprocal_rank_fusion, Field, FilterBuilder
from ingest import get_mock_embedding

HOST = "localhost:6574"

def search_collections(query: str, collections: list[str], limit: int = 5, site_id: str = None):
    query_vector = get_mock_embedding(query)
    all_results = []

    with VectorAIClient(HOST) as client:
        for collection in collections:
            filter_ = None
            if site_id:
                filter_ = (
                    FilterBuilder()
                    .must(Field("site_id").eq(site_id))
                    .build()
                )

            results = client.points.search(
                collection,
                vector=query_vector,
                limit=limit,
                filter=filter_,
                with_payload=True,
            )
            all_results.append(results)

    # RRF ranks results by position across lists, not raw score.
    # This is necessary because scores from different embedding
    # models are not directly comparable across collections.
    merged = reciprocal_rank_fusion(all_results, limit=limit)
    return merged

def format_results(results):
    formatted = []
    for i, result in enumerate(results):
        formatted.append({
            "rank": i + 1,
            "id": result.id,
            "score": round(result.score, 4) if result.score is not None else 0.0,
            "payload": result.payload,
        })
    return formatted

The format_results function normalizes the merged RRF output into a consistent structure, assigning each result a rank, ID, score, and payload regardless of which collection it came from.

Within each collection, vector distance is computed locally, so raw scores are not compared across collections; instead, RRF is used.

With the search function in place, wire it into the FastAPI endpoint. The endpoint receives incoming search requests, validates them using Pydantic, a Python library that checks that request data has the correct structure before the code processes it, and triggers an audit log entry on every query.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from search import search_collections, format_results
from audit import emit_audit_event
import uuid

app = FastAPI(title="Clinical Search Gateway")

class SearchRequest(BaseModel):
    query: str
    collections: list[str]
    limit: int = 5
    site_id: str = "DEMO-SITE"
    principal_id: str = "USER-DEMO-001"

class SearchResponse(BaseModel):
    request_id: str
    query: str
    results: list[dict]
    collections_queried: list[str]
    total_results: int

@app.post("/search", response_model=SearchResponse)
def multi_collection_search(request: SearchRequest):
    request_id = str(uuid.uuid4())

    try:
        raw_results = search_collections(
            query=request.query,
            collections=request.collections,
            limit=request.limit,
            site_id=request.site_id,
        )
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

    formatted = format_results(raw_results)

    emit_audit_event(
        principal_id=request.principal_id,
        site_id=request.site_id,
        request_id=request_id,
        collections_queried=request.collections,
        retrieved_record_ids=[r["id"] for r in formatted],
        query=request.query,
    )

    return SearchResponse(
        request_id=request_id,
        query=request.query,
        results=formatted,
        collections_queried=request.collections,
        total_results=len(formatted),
    )

@app.get("/health")
def health():
    return {"status": "ok"}

Start the gateway:
python3 -m uvicorn src.main:app --host 127.0.0.1 --port 8000

Test it with a clinical sample query:

curl -X POST http://127.0.0.1:8000/search \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "elevated creatinine acute kidney injury",
    "collections": ["lab_results", "clinical_notes", "knowledge_base"],
    "limit": 3,
    "site_id": "DEMO-SITE",
    "principal_id": "USER-DEMO-001"
  }'

The endpoint returns a ranked JSON response with results from all three collections. It should look like this:

Audit Logging for HIPAA Compliance

Treat every search query against this system as a PHI access event and log it accordingly. A clinician querying for patients with similar lab profiles is retrieving identifiable health information, and under 45 CFR section 164.312(b), every access must be logged. The query text itself should not appear in the log because it may contain PHI; hash it instead.

Each audit record must capture the principal ID, site ID, request ID, collections queried, retrieved record IDs, a SHA-256 hash of the query text, a timestamp, and the embedding model version. The emit_audit_event function writes each record as a JSON line to an append-only log file after every search.

In src/audit.py, add the function that writes each access event to the log:

import hashlib
import json
from datetime import datetime, timezone

AUDIT_LOG_PATH = "audit.log"

def hash_query(query: str) -> str:
    return hashlib.sha256(query.encode()).hexdigest()[:16]

def emit_audit_event(
    principal_id: str,
    site_id: str,
    request_id: str,
    collections_queried: list[str],
    retrieved_record_ids: list[int],
    query: str,
    model_version: str = "mock-embedder-v1",
):
    record = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "principal_id": principal_id,
        "site_id": site_id,
        "request_id": request_id,
        "collections_queried": collections_queried,
        "retrieved_record_ids": retrieved_record_ids,
        "query_hash": hash_query(query),
        "model_version": model_version,
    }

    with open(AUDIT_LOG_PATH, "a") as f:
        f.write(json.dumps(record) + "\n")

    return record

Each search event is written to the log with the query hashed and all required fields present:

Under 45 CFR section 164.316(b)(2)(i), these logs must be retained for six years. Feed the output into your organization's existing Security Information and Event Management (SIEM) system to manage retention. The retention mechanism itself is outside the scope of this tutorial.

Measuring Search Quality

In clinical search, missing a relevant result is more serious than returning an irrelevant one. If you miss a lab record, it could lead to a missed diagnosis. An irrelevant result is just extra noise. These mistakes are not the same, so your quality standards should reflect this difference.

Recall shows what percentage of all relevant records the system found. Precision shows the percentage of returned records that were relevant. For clinical search, Recall is the most important metric.

The evaluation script below calculates Recall@10 and MRR (Mean Reciprocal Rank) against a labeled test set. Replace the synthetic test cases with clinician-labeled queries before running this in production.

Add src/evaluate.py with the Recall@10 and MRR evaluation functions.

def recall_at_k(retrieved_ids: list, expected_ids: list, k: int = 10) -> float:
    retrieved_set = set(retrieved_ids[:k])
    expected_set = set(expected_ids)
    if not expected_set:
        return 0.0
    return len(retrieved_set & expected_set) / len(expected_set)

def mean_reciprocal_rank(retrieved_ids: list, expected_ids: list) -> float:
    expected_set = set(expected_ids)
    for rank, rid in enumerate(retrieved_ids, start=1):
        if rid in expected_set:
            return 1.0 / rank
    return 0.0

The evaluation script prints Recall@10 and MRR scores for each test case, which looks like this:

Set these goals before launching: Recall@10should be at least 0.95 for safety-critical queries. MRR should be above 0.6, with 0.7 as your target. Citation coverage, which is the percentage of results shown to a clinician that include a traceable source record, should be above 0.9. These thresholds are recommended starting points; you can calibrate them against your own clinician-labeled test set.

Recall@10 of 0.95 or higher is the launch target. Treat 0.90 as the floor: if your clinician-labeled test set falls below 0.90, do not ship. Add a hybrid search that mixes dense vector search with keyword matching, and include a clinical reranker before expanding further.

Wrapping Up

At this point, you have a working multi-collection clinical search system:

• Six collections, separated by data type, each with its own embedding model and schema
• A FastAPI and gRPC gateway that fans queries out across collections and merges results using RRF
• VectorAI DB running on-premises via Docker with HIPAA-aligned configuration and AES-256 encryption at rest

To go deeper into the imaging collection, see the medical image similarity search tutorial.

For collection configuration details and the current API reference, see the VectorAI DB documentation and the Python SDK reference. You can also check out our Community Edition to get started.