DEV Community

wellallyTech
wellallyTech

Posted on

Stop Googling Your Lab Results: Build a Medical RAG Pipeline with Unstructured.io and Qdrant

#ai #rag #vectordatabase #python

We’ve all been there: you receive a 20-page PDF medical report filled with cryptic tables, nested rows, and handwritten notes. You want to ask, "Has my LDL cholesterol trended down since last year?" but your data is trapped in a non-searchable pixel prison. 🏥

In this tutorial, we are building a Medical RAG (Retrieval-Augmented Generation) system. We will leverage Unstructured.io for complex PDF layout analysis, Qdrant as our high-performance vector database, and LangChain to glue it all together. This setup allows you to transform messy lab reports into a structured, queryable personal medical knowledge base.

By the end of this guide, you’ll master Unstructured.io OCR, vector indexing, and how to handle multi-modal document parsing for real-world messy data. 🚀

The Architecture: From Pixels to Insights

Handling medical reports is tricky because the data is usually trapped in tables. A simple text extraction won't work; we need layout-aware parsing.

graph TD
    A[PDF Lab Reports] --> B[Unstructured.io Partitioning]
    B --> C{Layout Analysis}
    C -->|Tables| D[HTML/Structured Data]
    C -->|Text| E[Text Chunks]
    D --> F[LangChain Document Transformer]
    E --> F
    F --> G[OpenAI Embeddings]
    G --> H[(Qdrant Vector Store)]
    I[User Query] --> J[Query Embedding]
    J --> H
    H --> K[Contextual Retrieval]
    K --> L[GPT-4o Medical Insight]
Enter fullscreen mode Exit fullscreen mode

Prerequisites

Before we dive in, ensure you have the following tech_stack ready:

  • Unstructured.io: For "hi_res" partitioning (requires poppler and tesseract).
  • Qdrant: Our vector database (running locally via Docker or Cloud).
  • LangChain: The orchestration framework.
  • OpenAI Embeddings: To turn medical text into math. 
pip install unstructured[pdf] qdrant-client langchain openai tiktoken
Enter fullscreen mode Exit fullscreen mode

Step 1: Parsing Complex Tables with Unstructured.io

Standard PDF loaders often mangle tables. Unstructured.io uses computer vision models to detect document elements (Title, Table, List).

from unstructured.partition.pdf import partition_pdf

# We use the 'hi_res' strategy to trigger OCR and layout detection
# This is crucial for medical reports with complex grids
elements = partition_pdf(
    filename="my_lab_report.pdf",
    strategy="hi_res",
    infer_table_structure=True,
    model_name="yolox"
)

# Extracting tables specifically as HTML for better LLM understanding
tables = [el.metadata.text_as_html for el in elements if el.category == "Table"]
print(f"Detected {len(tables)} tables in the report!")
Enter fullscreen mode Exit fullscreen mode

Step 2: Vector Indexing with Qdrant

Once we have our chunks, we need to store them in a way that captures semantic meaning. Qdrant is perfect for this due to its speed and payload filtering capabilities. 🥑

from langchain_community.vectorstores import Qdrant
from langchain_openai import OpenAIEmbeddings
from langchain.schema import Document

# Convert Unstructured elements into LangChain Documents
docs = []
for el in elements:
    metadata = el.metadata.to_dict()
    metadata["source"] = "lab_report_2023"
    docs.append(Document(page_content=str(el), metadata=metadata))

# Initialize Qdrant and upload embeddings
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Qdrant.from_documents(
    docs,
    embeddings,
    location=":memory:", # Local testing, or use URL/API Key for production
    collection_name="medical_knowledge_base",
)
Enter fullscreen mode Exit fullscreen mode

Step 3: Querying the Knowledge Base

Now we can ask complex questions. The RAG pipeline will find the relevant table or text chunk and provide the context to the LLM.

from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-4o", temperature=0)
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

query = "What was my Glucose level and is it within the normal range?"
response = qa_chain.invoke(query)
print(f"Result: {response['result']}")
Enter fullscreen mode Exit fullscreen mode

💡 Pro-Tip: Advanced Medical Patterns

Building a local RAG is a great start, but when dealing with high-stakes medical data, you need to implement Hybrid Search and Reranking to ensure 100% accuracy.

If you are looking for production-ready patterns, such as handling multi-vector retrieval or integrating medical ontologies (like SNOMED CT), I highly recommend checking out the advanced guides on WellAlly Tech Blog. They provide deep dives into how to optimize vector search for specialized domains like healthcare and finance.

Conclusion

By combining Unstructured.io's layout awareness with Qdrant's vector capabilities, we've transformed a static PDF into a dynamic assistant. You no longer need to manually scan rows of data; your RAG pipeline does the heavy lifting.

What's next?

  1. Add Chain-of-Thought prompting to interpret the medical trends.
  2. Implement a front-end using Streamlit to upload your own PDFs.
  3. Explore wellally.tech/blog for more insights on building robust AI agents.

Have you tried parsing medical data before? What was your biggest challenge? Let me know in the comments! 👇 💻

Top comments (0)