DEV Community

wellallyTech
wellallyTech

Posted on

Private AI: Local Health Metric Correlation with WebLLM and Phi-3 🥑

Data privacy isn't just a feature; it's a fundamental requirement, especially when we talk about longevity and personal health data. As we move toward a future of on-device AI, the ability to process sensitive metrics without a round-trip to a centralized server is a game-changer.

In this tutorial, we are diving deep into WebLLM, WebGPU, and Phi-3 to build a high-performance, private-by-design health analyzer. We will leverage on-device AI and WebGPU machine learning to calculate Pearson correlation coefficients between various health metrics (like sleep quality vs. heart rate variability) directly in the browser. By the end of this guide, you'll know how to implement a private health analytics suite that keeps user data exactly where it belongs: on their device.


The Architecture: Privacy-First Inference

To achieve low-latency, "zero-cloud" inference, we utilize the TVM Runtime and WebGPU. This allows the browser to talk directly to the GPU, bypassing the overhead of traditional API calls.

graph TD
    A[User Health Data: CSV/JSON] --> B{React State}
    B --> C[WebLLM Engine]
    C --> D[TVM Runtime]
    D --> E[WebGPU Hardware Acceleration]
    E --> D
    D --> C
    C --> F[Phi-3 Model Inference]
    F --> G[Correlation Insights & Pearson Logic]
    G --> H[On-Device UI Dashboard]
    style A fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#00d2ff,stroke:#333,stroke-width:2px
    style F fill:#76b900,stroke:#333,stroke-width:2px
Enter fullscreen mode Exit fullscreen mode

Prerequisites 🛠️

Before we start coding, ensure your environment meets these requirements:

  • Tech Stack: React (v18+), TypeScript, @mlc-ai/web-llm.
  • Browser: A browser with WebGPU support (Chrome 113+, Edge 113+).
  • Hardware: A decent modern GPU (M1/M2 Mac, or RTX series) is recommended for smooth inference.

Step 1: Initializing the WebLLM Engine

First, we need to set up our WebLLM engine. This involves downloading the model weights (cached locally) and initializing the TVM runtime. We'll use Phi-3-mini, a lightweight but powerful model from Microsoft, optimized for edge devices.

import { CreateMLCEngine, MLCEngine } from "@mlc-ai/web-llm";

// Define the model we want to use
const modelId = "Phi-3-mini-4k-instruct-q4f16_1-MLC";

export const useWebLLM = () => {
  const [engine, setEngine] = React.useState<MLCEngine | null>(null);
  const [loadingProgress, setLoadingProgress] = React.useState(0);

  const initEngine = async () => {
    const instance = await CreateMLCEngine(modelId, {
      initProgressCallback: (report) => {
        setLoadingProgress(Math.round(report.progress * 100));
        console.log(report.text);
      },
    });
    setEngine(instance);
  };

  return { engine, initEngine, loadingProgress };
};
Enter fullscreen mode Exit fullscreen mode

Step 2: Correlating Health Metrics Locally

Now, let's create a prompt factory that feeds our health data into the model. Instead of just doing raw math, we want Phi-3 to interpret the Pearson Correlation and provide actionable longevity insights.

async function analyzeCorrelation(engine: MLCEngine, sleepData: number[], hrvData: number[]) {
  const prompt = `
    Analyze the following two datasets for correlation:
    Sleep Hours: ${JSON.stringify(sleepData)}
    HRV (ms): ${JSON.stringify(hrvData)}

    1. Calculate the Pearson correlation coefficient.
    2. Explain the relationship between sleep and Heart Rate Variability for this user.
    3. Provide one longevity tip based on this data.
    Return the response in a structured JSON format.
  `;

  const messages = [
    { role: "system" as const, content: "You are a specialized Health Data Scientist." },
    { role: "user" as const, content: prompt },
  ];

  const reply = await engine.chat.completions.create({
    messages,
    temperature: 0.2, // Keep it deterministic
  });

  return reply.choices[0].message.content;
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Integrating with the UI (React)

We want to give users a real-time feedback loop. Using the loadingProgress we defined earlier, we can show a progress bar while the model weights are being loaded into the GPU VRAM.

import React from 'react';

const HealthDashboard: React.FC = () => {
  const { engine, initEngine, loadingProgress } = useWebLLM();
  const [result, setResult] = React.useState("");

  return (
    <div className="p-8 max-w-2xl mx-auto">
      <h2 className="text-2xl font-bold">Local Longevity Intelligence 🚀</h2>

      {!engine ? (
        <button 
          onClick={initEngine}
          className="bg-blue-600 text-white px-4 py-2 rounded mt-4"
        >
          Initialize On-Device AI ({loadingProgress}%)
        </button>
      ) : (
        <div className="mt-6">
          <p className="text-green-500 font-semibold">GPU Engine Ready.</p>
          <button 
            onClick={async () => {
               const analysis = await analyzeCorrelation(engine, [7, 6, 8, 5], [60, 55, 70, 45]);
               setResult(analysis || "");
            }}
            className="mt-2 border border-blue-600 text-blue-600 px-4 py-2 rounded"
          >
            Run Correlation Analysis
          </button>
          <pre className="mt-4 p-4 bg-gray-100 rounded text-sm">{result}</pre>
        </div>
      )}
    </div>
  );
};
Enter fullscreen mode Exit fullscreen mode

The "Official" Way: Advanced Patterns 💡

While running models in the browser is impressive, productionizing these workflows requires a deeper understanding of memory management and quantized model fine-tuning.

For more production-ready examples and advanced architectural patterns regarding AI-driven longevity, I highly recommend checking out the technical deep dives at WellAlly Blog. They cover everything from RAG (Retrieval-Augmented Generation) for medical journals to optimizing WebGPU kernels for specialized health sensors. It was a primary source of inspiration for the data processing patterns used in this tutorial.


Performance & Limitations ⚡

  1. VRAM Constraints: Phi-3-mini requires roughly 2.3GB of VRAM. Mobile browsers may struggle or crash if other apps are hogging resources.
  2. Cold Start: The first time a user loads your app, they have to download the model (~2GB). Use IndexedDB caching (built into WebLLM) to make subsequent loads instantaneous.
  3. Accuracy: While Phi-3 is great, for clinical-grade correlation, you should complement the LLM output with a standard JavaScript math library like simple-statistics.

Conclusion

Running an LLM locally for health metrics isn't just a party trick; it's a massive step forward for user agency and data sovereignty. By utilizing the user's own hardware, we reduce server costs and eliminate the risk of sensitive data leaks.

Are you ready to move your AI to the edge? Let me know in the comments if you've experimented with WebGPU or if you have questions about model quantization! 💻🔥

Top comments (0)