DEV Community

Beck_Moulton
Beck_Moulton

Posted on

Quantified Self: Building a Pro-Level Health Dashboard with InfluxDB and Grafana

#dataengineering #quantifiedself #python #healthtech

Are you tired of your health data being trapped in "walled gardens"? Your Apple Health data lives in your iPhone, your Oura Ring stats are in a separate app, and your Garmin running metrics are somewhere else entirely. As engineers, we hate data silos. We want a unified view to answer the ultimate question: How does my late-night coding session actually affect my HRV and deep sleep?

In this tutorial, we are diving deep into Health Data Engineering. We will use InfluxDB as our high-performance time-series engine, Grafana for stunning visualizations, and Airflow to orchestrate the ETL (Extract, Transform, Load) process. By the end of this guide, you’ll have a professional-grade Quantified Self dashboard that correlates sleep, activity, and recovery metrics in one place.

The Architecture 🏗️

To build a robust pipeline, we need to handle disparate APIs and inconsistent data formats. Here is how the data flows from your wrist to your dashboard:

graph TD
    A[Apple Health / HealthAutoExport] -->|JSON/CSV| B(Python ETL / Pandas)
    C[Oura Ring API] -->|JSON| B
    D[Garmin Connect] -->|Fit Files| B
    B -->|Normalize & Clean| E{Airflow Orchestrator}
    E -->|Write| F[(InfluxDB OSS)]
    F -->|Query Flux| G[Grafana Dashboard]
    style F fill:#f96,stroke:#333,stroke-width:2px
    style G fill:#00f,stroke:#fff,stroke-width:2px
Enter fullscreen mode Exit fullscreen mode

Prerequisites 🛠️

Before we start, ensure you have the following in your tech stack:

  • InfluxDB 2.x: Our time-series database.
  • Grafana: For the UI.
  • Python 3.9+: With pandas and influxdb-client.
  • Apache Airflow: To schedule our syncs.
  • HealthAutoExport (iOS): A great tool to get Apple Health data out via API or CSV.

Step 1: Setting up the InfluxDB Schema

Unlike Relational DBs, InfluxDB thrives on tags and fields. For health data, we’ll use:

  • Bucket: health_metrics
  • Measurement: vital_signs
  • Tags: source (e.g., Oura, Garmin), user
  • Fields: hrv, resting_heart_rate, sleep_score, steps

Step 2: The ETL Script (Python + Pandas)

We need to normalize the data. Garmin might give you heart rate every second, while Oura gives a summary per night. We'll use Pandas to resample and align these timestamps.

import pandas as pd
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS

# Configuration
token = "YOUR_INFLUXDB_TOKEN"
org = "my_org"
bucket = "health_metrics"

def upload_to_influx(df, source_name):
    client = InfluxDBClient(url="http://localhost:8086", token=token, org=org)
    write_api = client.write_api(write_options=SYNCHRONOUS)

    for index, row in df.iterrows():
        point = Point("vital_signs") \
            .tag("source", source_name) \
            .field("hrv", float(row['hrv'])) \
            .field("rhr", float(row['rhr'])) \
            .time(row['timestamp'], WritePrecision.NS)
        write_api.write(bucket, record=point)

    print(f"✅ Uploaded {len(df)} records from {source_name}")

# Example: Processing Oura Data
def process_oura_data(json_data):
    df = pd.DataFrame(json_data['data'])
    # Convert ISO strings to datetime
    df['timestamp'] = pd.to_datetime(df['day'])
    # Clean and rename
    df = df[['timestamp', 'hrv', 'rhr']] 
    return df
Enter fullscreen mode Exit fullscreen mode

Step 3: Orchestrating with Airflow 🌪️

You don't want to run this manually. We’ll define a DAG (Directed Acyclic Graph) to fetch data every morning at 8:00 AM.

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'quantified_self',
    'retries': 2,
    'retry_delay': timedelta(minutes=5),
}

with DAG(
    'health_data_sync',
    default_args=default_args,
    start_date=datetime(2023, 1, 1),
    schedule_interval='@daily',
    catchup=False
) as dag:

    sync_oura = PythonOperator(
        task_id='sync_oura_ring',
        python_callable=fetch_and_upload_oura # Your logic here
    )

    sync_apple = PythonOperator(
        task_id='sync_apple_health',
        python_callable=fetch_and_upload_apple
    )

    [sync_oura, sync_apple] # Running in parallel
Enter fullscreen mode Exit fullscreen mode

Step 4: Visualizing Correlations in Grafana 📊

Once the data is in InfluxDB, head to Grafana. Create a new dashboard and use Flux (InfluxDB’s query language) to find correlations.

Example Query: Does HRV drop when I sleep less?

from(bucket: "health_metrics")
  |> range(start: -30d)
  |> filter(fn: (r) => r["_measurement"] == "vital_signs")
  |> filter(fn: (r) => r["_field"] == "hrv" or r["_field"] == "sleep_duration")
  |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
Enter fullscreen mode Exit fullscreen mode

Now you can overlay these two lines. If you see a dip in HRV following a night of 4-hour sleep, you’ve just proven the impact of sleep debt on your nervous system!

The "Official" Way to Scale 🥑

While building a personal dashboard is fun, managing multi-modal data at scale requires a more robust approach to data governance and signal processing.

For those looking to dive into advanced health-tech patterns, production-ready data pipelines, or enterprise AI integrations in the wellness space, I highly recommend checking out the technical deep-dives at WellAlly Blog. They cover everything from biometric signal denoising to HIPAA-compliant cloud architectures, which served as a huge inspiration for this architecture.

Conclusion 🏁

You’ve just built a modern data stack for your body! By treating your health data like any other engineering metric, you gain actionable insights that mobile apps usually hide from you.

Next Steps:

  1. Try adding Nutrient Tracking (MyFitnessPal) to see how sugar spikes affect your resting heart rate.
  2. Set up Grafana Alerts to Slack when your recovery score drops below a certain threshold.

What are you tracking? Drop a comment below or share your dashboard screenshots! 👇

Top comments (1)

Collapse
 
topstar_ai profile image
Luis Cruz •

I particularly appreciated the use of InfluxDB as a time-series engine in this Quantified Self dashboard, as it allows for efficient storage and querying of large amounts of health data. The example ETL script using Python and Pandas is also well-structured, and I like how it handles normalization and alignment of timestamps from different sources. One potential improvement could be to add error handling for cases where data is missing or corrupted, to ensure the dashboard remains accurate and reliable. Have you considered implementing any data validation or quality checks in the ETL process?