Goodbye Cloud! Build a Privacy-First Medical Summarizer with WebLLM, WebGPU, and React

#ai #react #security #discuss

Handling medical data is a privacy nightmare. Every time you upload a sensitive PDF to a cloud-based AI service, you're essentially handing over your most personal information to a third party. But what if you could harness the power of Large Language Models (LLMs) without the data ever leaving your machine?

In this tutorial, we are building a Privacy-First Medical Record Summarizer. By leveraging WebGPU performance, WebLLM, and React, we can run powerful models like Llama-3 or Mistral directly in your browser. This approach ensures high-speed local LLM processing and privacy-preserving AI workflows that are both cost-effective and secure. If you've been looking for a way to integrate browser AI into your next project, you're in the right place!

The Architecture: Why WebGPU?

Traditional browser-based AI relied on WebAssembly (WASM) or CPU-bound processing, which was painfully slow for LLMs. WebGPU changes the game by giving the browser direct access to the device's graphics card. Using TVM (Tensor Virtual Machine), WebLLM compiles model weights to run efficiently on this hardware.

Here is how the data flow looks:

graph TD
    A[User Uploads Medical PDF] --> B[PDF.js Extraction]
    B --> C{Text Chunking}
    C --> D[WebLLM Engine]
    D --> E[WebGPU Compute]
    E --> D
    D --> F[LLM Inference: Llama-3/Mistral]
    F --> G[Summarized Medical Report]
    G --> H[React UI Display]
    style E fill:#f9f,stroke:#333,stroke-width:2px

Prerequisites

To follow along, make sure you have:

A browser with WebGPU support (Chrome 113+, Edge, or Arc).
Node.js and npm/yarn installed.
The following tech stack: React, WebLLM, and PDF.js.

Step 1: Setting Up the WebLLM Engine

First, we need to initialize the WebLLM engine. Unlike standard APIs, we don't need an OPENAI_API_KEY. We are downloading the model weights into the browser's Cache API.

import * as webllm from "@mlc-ai/web-llm";

// We'll use a small but powerful model for medical summaries
const selectedModel = "Llama-3-8B-Instruct-v0.1-q4f16_1-MLC";

async function initializeEngine(onProgress: (p: number) => void) {
    const engine = await webllm.CreateWebWorkerEngine(
        new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }),
        selectedModel,
        {
            initProgressCallback: (report) => {
                onProgress(Math.round(report.progress * 100));
            },
        }
    );
    return engine;
}

Step 2: The React Component

We want a clean interface where users can drag and drop their medical records. We use a simple state machine to track the loading, processing, and result phases.

import React, { useState } from 'react';
import { initializeEngine } from './llmService';

const MedicalSummarizer = () => {
    const [summary, setSummary] = useState("");
    const [loading, setLoading] = useState(false);
    const [progress, setProgress] = useState(0);

    const handleSummarize = async (text: string) => {
        setLoading(true);
        const engine = await initializeEngine(setProgress);

        const messages = [
            { role: "system", content: "You are a medical assistant. Summarize the following medical record clearly, highlighting diagnoses and medications." },
            { role: "user", content: text }
        ];

        const reply = await engine.chat.completions.create({ messages });
        setSummary(reply.choices[0].message.content);
        setLoading(false);
    };

    return (
        <div className="p-8 max-w-2xl mx-auto">
            <h2 className="text-2xl font-bold mb-4">🏥 Local Med-Summary</h2>
            {loading && <div className="mb-4">Loading Model: {progress}%</div>}
            <textarea 
                className="w-full p-4 border rounded" 
                placeholder="Paste de-identified medical text here..." 
                onChange={(e) => handleSummarize(e.target.value)}
            />
            {summary && (
                <div className="mt-6 p-4 bg-blue-50 border-l-4 border-blue-500">
                    <h3 className="font-bold">Summary:</h3>
                    <p>{summary}</p>
                </div>
            )}
        </div>
    );
};

Going Beyond the Basics

While running a basic summary is great, production-ready healthcare apps require structured data extraction (e.g., extracting ICD-10 codes) and complex RAG (Retrieval-Augmented Generation) pipelines—all while maintaining that strict local-only privacy.

For more production-ready examples and advanced patterns on deploying local AI models, I highly recommend checking out the deep-dive guides at WellAlly Blog. They cover everything from optimizing WebGPU memory buffers to fine-tuning local models for specific domains like clinical healthcare.

Step 3: Handling Large Files (PDF Parsing)

Medical records are usually PDFs. To keep it local, use pdfjs-dist. Never send the file to a server!

import * as pdfjs from 'pdfjs-dist';

async function extractTextFromPDF(file) {
    const arrayBuffer = await file.arrayBuffer();
    const pdf = await pdfjs.getDocument(arrayBuffer).promise;
    let text = "";
    for (let i = 1; i <= pdf.numPages; i++) {
        const page = await pdf.getPage(i);
        const content = await page.getTextContent();
        text += content.items.map(item => item.str).join(" ");
    }
    return text;
}

Why This Matters 🥑

Zero Latency: Once the model is cached, inference happens at the speed of your GPU. No network round-trips.
Zero Cost: You aren't paying per token. Your user provides the compute.
Absolute Privacy: The "Holy Grail" of healthcare tech. Data stays in the RAM, then it's gone.

Conclusion

The era of "Cloud-Only AI" is ending. With WebGPU and WebLLM, we can now build applications that respect user privacy without sacrificing the "magic" of LLMs. Whether you are building a personal health tracker or a secure document analyzer, the browser is becoming a first-class citizen for AI deployment.

What are you building next with WebGPU? Let me know in the comments below! 👇

For more advanced tutorials on Edge AI and privacy-preserving tech, visit the official *WellAlly Technology Blog*. 💻✨

DEV Community