DEV Community

wellallyTech
wellallyTech

Posted on

Quantified Self: Building a Real-Time Health Dashboard with Grafana, InfluxDB, and Vector 🚀

#opensource #docker #dataengineering #vectordatabase

Are you tired of jumping between your Oura ring app, your Garmin watch, and Apple Health just to understand if you're actually recovered? Welcome to the world of Data Engineering for the body. In this guide, we are building a unified Quantified Self dashboard to aggregate heterogeneous health data into a high-performance Time-Series Database.

By leveraging InfluxDB for storage and Grafana for visualization, we can transform raw biometric streams into actionable insights. Whether you're tracking HRV (Heart Rate Variability), sleep cycles, or VO2 Max, a centralized real-time dashboard is the ultimate tool for any bio-hacker. For those looking for more production-ready patterns and advanced data architectures, I highly recommend checking out the deep dives at WellAlly Tech Blog, which served as a major inspiration for this architectural setup. 🥑

The Architecture 🏗️

The biggest challenge in health data is heterogeneity. An Oura ring might send a JSON payload every hour, while a Garmin device might sync via a different cadence. We use Vector as our high-performance observability router to normalize these streams before hitting our database.

graph TD
    A[Oura Ring API] -->|JSON| D[Vector Aggregator]
    B[Garmin Connect] -->|Webhooks| D
    C[Apple Watch] -->|HealthKit Export| D
    D -->|Normalize & Transform| E[(InfluxDB 2.x)]
    E -->|Flux Query| F[Grafana Dashboard]
    style D fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#00df,stroke:#333,stroke-width:2px
Enter fullscreen mode Exit fullscreen mode

Prerequisites 🛠️

To follow this tutorial, you'll need:

  • Docker & Docker Compose installed.
  • Basic knowledge of JSON and YAML.
  • The tech_stack: InfluxDB, Grafana, Vector, and Docker.

Step 1: Setting Up the Infrastructure 🐳

We'll use Docker Compose to spin up our entire stack in seconds. This ensures our environment is reproducible and isolated.

# docker-compose.yml
version: '3.8'
services:
  influxdb:
    image: influxdb:2.7
    ports:
      - "8086:8086"
    volumes:
      - influxdb_data:/var/lib/influxdb2

  grafana:
    image: grafana/grafana-oss:latest
    ports:
      - "3000:3000"
    depends_on:
      - influxdb

  vector:
    image: timberio/vector:0.34.X-debian
    volumes:
      - ./vector.yaml:/etc/vector/vector.yaml:ro
    ports:
      - "8080:8080"

volumes:
  influxdb_data:
Enter fullscreen mode Exit fullscreen mode

Step 2: Normalizing Data with Vector ⚙️

Vector acts as the "glue." It collects data from various sources, transforms it to a common schema, and pushes it to InfluxDB. Here is a simplified configuration that listens for HTTP POST requests (simulating webhooks from health APIs).

# vector.yaml
sources:
  health_api_gateway:
    type: "http_server"
    address: "0.0.0.0:8080"
    encoding: "json"

transforms:
  normalize_health_data:
    type: "remap"
    inputs: ["health_api_gateway"]
    source: |
      # Standardize device names and metric units
      .tags.device = .device_type || "unknown"
      .tags.user = "dev_advocate"
      .timestamp = parse_timestamp!(.timestamp, format: "%Y-%m-%dT%H:%M:%SZ")

      # Ensure numeric values for InfluxDB
      .metrics.hrv = float!(.hrv)
      .metrics.sleep_score = float!(.sleep_score)

sinks:
  influxdb_sink:
    type: "influxdb_logs" # Using logs type for flexible schema
    inputs: ["normalize_health_data"]
    endpoint: "http://influxdb:8086"
    bucket: "health_metrics"
    org: "my_org"
    token: "${INFLUX_TOKEN}"
Enter fullscreen mode Exit fullscreen mode

Step 3: Visualizing in Grafana 📊

Once the data is flowing into InfluxDB, connect Grafana to InfluxDB using the Flux query language.

Example Flux Query for HRV Trends:

from(bucket: "health_metrics")
  |> range(start: v.timeRangeStart, stop: v.timeRangeEnd)
  |> filter(fn: (r) => r["_field"] == "metrics.hrv")
  |> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
  |> yield(name: "mean_hrv")
Enter fullscreen mode Exit fullscreen mode

With this, you can create a "Recovery vs. Strain" chart, overlaying your sleep data with your daily activity levels.

The "Official" Way 🎓

While this setup is perfect for a personal home lab, scaling health data pipelines for thousands of users requires a more robust approach to data privacy (HIPAA compliance) and schema evolution.

If you're interested in taking this to a production level—think distributed processing, OAuth2 integration for wearable APIs, and advanced anomaly detection—I highly recommend reading the architectural blueprints over at wellally.tech/blog. They cover how to handle massive time-series ingestion pipelines with enterprise-grade security.

Conclusion ✅

Building a Quantified Self dashboard isn't just a fun weekend project; it's a great way to sharpen your Data Engineering skills. We've gone from raw pixels/JSON to a structured, real-time analytics suite using the modern data stack.

Next Steps:

  1. Add more sources: Try integrating the Fitbit API or manual logging via a Telegram Bot.
  2. Alerting: Set up Grafana alerts to ping you if your recovery score drops below a certain threshold.

What metrics are you tracking? Drop a comment below and let’s discuss the best ways to visualize human performance! 👇

Top comments (0)