Local & Private: Building a Real-time ECG Analyzer in Chrome with WebLLM and WebGPU 🚀

Privacy in healthcare isn't just a "nice-to-have"—it's a legal and ethical mandate. When dealing with sensitive Electrocardiogram (ECG) reports, sending raw data to a cloud provider can trigger a mountain of compliance hurdles. This is where Edge AI and Privacy-first AI local inference change the game. By leveraging WebLLM and WebGPU, we can now run massive Large Language Models (LLMs) directly in the browser at near-native speeds.

In this tutorial, we will explore how to build a high-performance, browser-based ECG analyzer. We'll use TVM (Apache's compiler stack) to run Wasm-accelerated models that interpret ECG data extracted via PDF.js. By the end of this guide, you'll understand how to implement a fully client-side inference pipeline that keeps patient data where it belongs: on their own device. 💻

---

The Architecture: Why WebLLM? 🏗️

Traditional web-based AI relies on heavy API calls. However, for real-time medical report analysis, we need low latency and zero data leakage. The stack below utilizes WebGPU for hardware acceleration and TVM to bridge the gap between complex ML models and the web's WebAssembly (Wasm) runtime.

Data Flow Diagram

graph TD
    A[User Uploads ECG PDF] --> B[PDF.js: Text/Image Extraction]
    B --> C{WebLLM Engine}
    C --> D[WebGPU: Hardware Acceleration]
    D --> E[TVM: Compiled Wasm Runtime]
    E --> F[Llama-3/Phi-3 Model Weights]
    F --> G[Instant ECG Interpretation]
    G --> H[Render Results to UI]
    style C fill:#f96,stroke:#333,stroke-width:2px
    style E fill:#bbf,stroke:#333,stroke-width:1px

---

Prerequisites 🛠️

To follow this advanced guide, you should have a basic understanding of TypeScript and React/Vue.

• Tech Stack:
  • WebLLM: The core engine for running LLMs in-browser.
  • TVM (Apache): Used for compiling models to Wasm.
  • WebGPU: A modern API for high-performance graphics and data-parallel computation.
  • PDF.js: For parsing the ECG reports.
• Browser: Chrome 113+ or any browser with WebGPU enabled.

---

Step 1: Initializing the WebLLM Engine

First, we need to set up the MLCEngine. This engine manages the model lifecycle, including downloading the sharded weights into the browser's Cache API and executing the inference via WebGPU.

import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm";

// We'll use a quantized version of Llama-3-8B for better performance
const selectedModel = "Llama-3-8B-Instruct-v0.1-q4f16_1-MLC";

async function initializeEngine(onProgress: (p: number) => void) {
  const engine = await CreateMLCEngine(
    selectedModel,
    { 
      initProgressCallback: (report) => {
        onProgress(report.progress);
        console.log(report.text);
      }
    }
  );
  return engine;
}

Step 2: Extracting Data from ECG PDFs 📄

ECG reports are often generated as PDFs. Before the LLM can analyze them, we need to extract the textual data (findings, heart rate, PR interval, etc.) using PDF.js.

import * as pdfjsLib from 'pdfjs-dist';

async function extractECGText(fileArrayBuffer) {
  const loadingTask = pdfjsLib.getDocument({ data: fileArrayBuffer });
  const pdf = await loadingTask.promise;
  let fullText = "";

  for (let i = 1; i <= pdf.numPages; i++) {
    const page = await pdf.getPage(i);
    const content = await page.getTextContent();
    const strings = content.items.map(item => item.str);
    fullText += strings.join(" ");
  }

  return fullText;
}

Step 3: Local Inference & Prompt Engineering

Once we have the text, we pass it to our local model. The key is to provide a "System Prompt" that constrains the model to medical interpretation logic without hallucinating.

async function analyzeECG(engine: MLCEngine, rawData: string) {
  const messages = [
    { role: "system", content: "You are a specialized cardiology AI assistant. Analyze the provided ECG text and summarize: 1. Heart Rate, 2. Rhythm, 3. Critical Flags. Be concise." },
    { role: "user", content: `Analyze this ECG report: ${rawData}` }
  ];

  const reply = await engine.chat.completions.create({
    messages,
    temperature: 0.2, // Keep it deterministic
  });

  return reply.choices[0].message.content;
}

---

The "Official" Way: Advanced Patterns 🥑

While running LLMs in the browser is revolutionary, managing large binary model files and ensuring cross-browser GPU compatibility can be tricky. For production-grade implementations and optimized TVM compilation patterns, we highly recommend checking out deeper architectural deep-dives.

For more production-ready examples and advanced Edge AI patterns, visit the official blog at https://www.wellally.tech/blog. They provide extensive resources on optimizing Wasm runtimes for high-concurrency healthcare environments.

---

Performance Optimization (TVM + Wasm) ⚡

The magic behind WebLLM is TVM. It compiles machine learning models into highly optimized WebAssembly and WGSL (WebGPU Shading Language).

1. Quantization: We use q4f16_1, which reduces model size by 4-bit quantization, making an 8B parameter model fit into ~4GB of VRAM.
2. Streaming: Instead of waiting for the whole response, we use the stream API to show results to the user word-by-word, enhancing the UX.

// Streaming implementation for real-time feel
const chunks = await engine.chat.completions.create({
  messages,
  stream: true,
});

for await (const chunk of chunks) {
  const content = chunk.choices[0]?.delta?.content || "";
  process.stdout.write(content); // Update your UI state here
}

---

Conclusion: The Future is Local 🌟

By combining WebLLM, WebGPU, and TVM, we've built a system that analyzes medical reports without a single byte of patient data ever leaving the client's machine. This is a massive win for privacy and latency.

As WebGPU becomes the standard across all mobile and desktop browsers, the browser will evolve from a simple document viewer into a powerful, AI-driven computational engine.

Ready to dive deeper?

• Check out the WebLLM GitHub repo.
• Don't forget to visit wellally.tech/blog for more insights on Edge AI.

What are you building with Local LLMs? Let me know in the comments below! 👇