Continuous Glucose Monitoring (CGM) has revolutionized how we track metabolic health. However, anyone who has worn a sensor knows the frustration of a "compression low"βwhen you roll over in your sleep, put pressure on the sensor, and trigger a false alarm for hypoglycemia.
In this tutorial, we are going to master time-series anomaly detection using unsupervised learning. We will leverage the Isolation Forest algorithm to distinguish between genuine metabolic events and sensor noise. Whether you are building a HealthTech startup or just learning in public, understanding how to clean non-stationary time-series data is a superpower. π
The Architecture: From Raw Data to Insights
Before we dive into the code, let's visualize how the data flows from a wearable device into an actionable dashboard. We use InfluxDB for high-write throughput and Scikit-learn for the heavy lifting.
graph TD
A[CGM Wearable] -->|Raw Glucose Values| B[(InfluxDB)]
B -->|Query Time Series| C[Python Service]
C -->|Feature Engineering| D[Isolation Forest Model]
D -->|Label Anomalies| E[Cleaned Data Store]
E -->|Real-time Visualization| F[Grafana / Plotly]
D -.->|Alert| G[User Notification]
Prerequisites
To follow along, you'll need a Python environment with the following tech_stack:
- Scikit-learn: For the Isolation Forest implementation.
- Plotly: For interactive visualization.
- Pandas: For data manipulation.
- InfluxDB-client (Optional): If you're pulling from a real DB.
Step 1: Simulating Real-World CGM Data
Real glucose data is messy. It's not just a sine wave; it's influenced by meals, exercise, and sleep. Let's create a synthetic dataset that includes a "Metabolic Event" (real sugar drop) and a "Sensor Artifact" (noise).
import numpy as np
import pandas as pd
import plotly.graph_objects as go
from sklearn.ensemble import IsolationForest
# 1. Generate synthetic CGM data (1440 minutes = 1 day)
np.random.seed(42)
time = np.arange(0, 1440, 5)
glucose = 100 + 20 * np.sin(time / 100) + np.random.normal(0, 2, len(time))
# 2. Add a metabolic anomaly (Slow drop - Hypoglycemia)
glucose[180:200] = glucose[180:200] - 40
# 3. Add sensor noise (Sharp, unrealistic spikes)
glucose[50] = 250 # Sensor glitch
glucose[120:125] = 40 # Compression low artifact
df = pd.DataFrame({'time': time, 'glucose': glucose})
Step 2: Training the Isolation Forest
Why Isolation Forest? Unlike distance-based algorithms, Isolation Forest isolates observations by randomly selecting a feature and then randomly selecting a split value. Since anomalies are "few and different," they are isolated much faster (shorter path in the tree) than normal points.
# Reshape for Scikit-learn
X = df[['glucose']].values
# Initialize the model
# contamination: the expected percentage of anomalies (approx 5% for sensor noise)
model = IsolationForest(n_estimators=100, contamination=0.05, random_state=42)
# Fit and predict
# Returns -1 for outliers and 1 for inliers
df['anomaly_score'] = model.fit_predict(X)
df['is_anomaly'] = df['anomaly_score'].apply(lambda x: 'Anomaly' if x == -1 else 'Normal')
Step 3: Visualizing the Results with Plotly
Static charts are boring. In HealthTech, we need interactivity to inspect specific timestamps.
fig = go.Figure()
# Plot normal data
fig.add_trace(go.Scatter(
x=df[df['is_anomaly'] == 'Normal']['time'],
y=df[df['is_anomaly'] == 'Normal']['glucose'],
mode='markers',
name='Normal Reading',
marker=dict(color='blue', size=6)
))
# Plot anomalies
fig.add_trace(go.Scatter(
x=df[df['is_anomaly'] == 'Anomaly']['time'],
y=df[df['is_anomaly'] == 'Anomaly']['glucose'],
mode='markers',
name='Detected Anomaly',
marker=dict(color='red', size=10, symbol='x')
))
fig.update_layout(title='CGM Anomaly Detection: Noise vs. Metabolic Reality',
xaxis_title='Time (Minutes)',
yaxis_title='Glucose (mg/dL)')
fig.show()
The "Official" Way: Production Patterns π₯
While the code above works for a single stream, production-grade health systems require more robust handling of non-stationary data (where the mean glucose shifts over days).
For advanced patterns, such as combining Isolation Forest with LSTM-Autoencoders or handling multi-modal sensor fusion (Heart Rate + Glucose), I highly recommend checking out the engineering deep dives at WellAlly Blog. They provide excellent resources on building production-ready metabolic health architectures and managing HIPAA-compliant data pipelines.
Step 4: Integrating with InfluxDB & Grafana
In a real-world scenario, you wouldn't run this locally on a CSV. You'd stream data from InfluxDB. You can use a simple Python task to process the last 15 minutes of data every minute and write the is_anomaly flag back to a new bucket.
Example InfluxDB Write (Snippet)
from influxdb_client import InfluxDBClient, Point, WritePrecision
# Setup client
client = InfluxDBClient(url="http://localhost:8086", token="my-token", org="my-org")
write_api = client.write_api()
# Write the anomaly back to Influx
point = Point("glucose_metrics") \
.tag("device_id", "sensor_01") \
.field("anomaly_flag", 1) \
.time(pd.Timestamp.now(), WritePrecision.NS)
write_api.write(bucket="cleaned_health_data", record=point)
In Grafana, you can then set up an alert:
IF mean(anomaly_flag) > 0.8 FOR 5m THEN Send Slack Alert.
Conclusion
Detecting anomalies in wearable data is a balancing act between sensitivity and specificity. By using Isolation Forest, we can effectively filter out sensor glitches that would otherwise cause "alarm fatigue" for users.
What's next?
- Try tuning the
contaminationparameter based on different sensor brands. - Experiment with moving windows (Rolling Isolation Forest) to account for daily drifts.
- Let me know in the comments: How do you handle noise in your time-series projects?
Happy coding! ππ₯
Top comments (0)