Beck_Moulton

Posted on Oct 8

Stop Guessing Your Health Stats: Building a Local Medical RAG System with Pinecone & LangChain

#ai #machinelearning #python #dataengineering

We’ve all been there: you get your annual health checkup results, and it looks like a collection of random numbers and cryptic abbreviations. Your first instinct is to Google "What does a high ALT mean?", and three clicks later, you're convinced you have three weeks to live. 😅

The problem isn't the data; it's the interpretation. While Large Language Models (LLMs) are great at explaining things, they are notorious for "hallucinating" clinical facts when they don't have the specific context.

In this tutorial, we are building a Personal Medical RAG (Retrieval-Augmented Generation) System. This allows you to feed your unstructured PDFs (checkup reports, medical papers, lab results) into a local vector pipeline so the AI can provide answers based only on your actual data. We’ll be using Pinecone, LangChain, Unstructured.io, and Sentence-Transformers to create a system that prioritizes evidence over imagination.

The Architecture: From Raw PDF to Medical Insights 🧬

Before we dive into the code, let’s look at the data flow. We need to transform messy, unstructured PDF tables into searchable mathematical vectors.

graph TD
    A[Unstructured PDF/Medical Paper] --> B[Unstructured.io Partitioning]
    B --> C[Recursive Character Text Splitting]
    C --> D[Sentence-Transformers Local Embedding]
    D --> E[(Pinecone Vector Database)]
    F[User Query: 'Is my glucose high?'] --> G[Embedding Engine]
    G --> H[Vector Similarity Search]
    E --> H
    H --> I[Context-Aware Prompt]
    I --> J[LLM: Grounded Health Advice]
    J --> K[Final Answer with Citations]

Prerequisites 🛠️

To follow along, you'll need:

Python 3.9+
Pinecone API Key (Serverless index works great here!)
Unstructured.io API Key (or use the local library)
Tech Stack: Pinecone, LangChain, Unstructured, Sentence-Transformers.

Step 1: Parsing the Messy Stuff with Unstructured.io

Medical reports are usually PDFs filled with tables and weird layouts. Standard PDF loaders often fail here. We'll use Unstructured.io to extract text while maintaining the hierarchy.

from unstructured.partition.pdf import partition_pdf

# Partitioning the PDF into elements
elements = partition_pdf(
    filename="health_report_2023.pdf",
    strategy="hi_res",  # Uses OCR to handle tables and images
    infer_table_structure=True,
    chunking_strategy="by_title",
    max_characters=1000,
    new_after_n_chars=800,
)

# Join the chunks for the RAG pipeline
docs = [el.text for el in elements]
print(f"✅ Successfully parsed {len(docs)} chunks from your report.")

Step 2: Local Embeddings for Privacy 🔒

When dealing with medical data, privacy is king. Instead of sending your raw text to an external API for embeddings, we use Sentence-Transformers to run the embedding model locally on your machine.

from langchain_community.embeddings import HuggingFaceEmbeddings

# Using a high-quality, lightweight medical-friendly model
model_name = "sentence-transformers/all-MiniLM-L6-v2"
model_kwargs = {'device': 'cpu'} # Switch to 'cuda' if you have a GPU!
encode_kwargs = {'normalize_embeddings': False}

embeddings = HuggingFaceEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)

print("🚀 Local embedding engine initialized.")

Step 3: Vector Storage with Pinecone

Now we need a place to store these vectors so we can perform "Semantic Search." Pinecone is the gold standard for high-speed retrieval.

from pinecone import Pinecone, ServerlessSpec
from langchain_pinecone import PineconeVectorStore

pc = Pinecone(api_key="YOUR_PINECONE_API_KEY")

index_name = "medical-rag-index"

# Create index if it doesn't exist
if index_name not in pc.list_indexes().names():
    pc.create_index(
        name=index_name,
        dimension=384, # Match the dimension of MiniLM
        metric='cosine',
        spec=ServerlessSpec(cloud='aws', region='us-east-1')
    )

# Sync our documents to the cloud
vectorstore = PineconeVectorStore.from_texts(
    docs, 
    embeddings, 
    index_name=index_name
)

Step 4: The "Official" Way to Handle Medical Data 💡

While this DIY setup is great for learning and personal use, production-grade medical AI requires much stricter adherence to clinical standards and data normalization (like HL7 FHIR).

For those looking to scale this into a production-ready health app or looking for advanced patterns in Medical AI Architectures, I highly recommend checking out the deep-dive articles at WellAlly Blog. They have some fantastic resources on handling high-compliance data and advanced RAG strategies that helped me refine my chunking logic for this project.

Step 5: The Grounded RAG Chain

Finally, we link everything together. When you ask a question, the system finds the relevant parts of your health report and feeds them to the LLM as "Context."

from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA

llm = ChatOpenAI(model_name="gpt-4o", temperature=0)

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3}),
    return_source_documents=True
)

query = "My LDL is 140 mg/dL. Based on my report, is this a concern?"
result = qa_chain.invoke({"query": query})

print(f"🤖 AI Response: {result['result']}")
print(f"📍 Sources used: {result['source_documents']}")

Why this works 🥑

By using this RAG approach, the LLM isn't just "remembering" what it learned during training in 2023. It is actively reading the document you just uploaded.

Zero Hallucinations: If the info isn't in the PDF, the AI can be prompted to say "I don't know" rather than guessing.
Evidence-Based: You get source citations (e.g., "See Table 2 in your Lab Results").
Local Control: Using local embeddings ensures your sensitive text is processed on your machine before being converted into anonymous numbers (vectors).

Conclusion 🚀

Building a personal medical assistant isn't about replacing doctors; it's about becoming a more informed patient. By leveraging LangChain and Pinecone, we've turned a static PDF into a dynamic, conversational knowledge base.

Next Steps:

Try adding a "Self-Correction" layer where the AI double-checks its answers against a trusted source like PubMed.
Explore more production-ready examples over at wellally.tech/blog.

What are you building with RAG lately? Drop a comment below! 👇

DEV Community