Litestream is a streaming replication tool for SQLite that continuously backs up your database to S3, Azure Blob, Google Cloud Storage, or SFTP. It's a popular choice for indie developers and small teams who want SQLite in production without risking data loss.
But Litestream replication failures are silent by default. If replication stops, your data is safe for now — until a server failure wipes out hours or days of unbackedup writes. This guide explains how to monitor Litestream alongside your app using Vigilmon.
What Can Go Wrong with Litestream
- The Litestream process crashes or is not started
- S3 bucket credentials expire or IAM permissions change
- Network issues block uploads to the replica destination
- The replica destination reaches a storage quota
- The Litestream sidecar container exits in a Docker/Kubernetes setup
None of these produce an error your users see — they only matter when you need to restore.
Monitoring Strategy for Litestream
Since Litestream doesn't expose an HTTP health endpoint, you have three monitoring approaches:
Approach 1: Health Endpoint in Your Application
The most reliable approach is to add a health endpoint in your own app that checks Litestream's replication status by examining the replica:
// Example: check when the last WAL frame was replicated
async function checkLivestreamHealth(): Promise<boolean> {
try {
// Read the litestream replica stats via its built-in metrics
const response = await fetch('http://localhost:9090/metrics')
if (!response.ok) return false
const text = await response.text()
// Check that replication lag is not too high
const lagMatch = text.match(/litestream_replica_lag_secondss+([d.]+)/)
if (lagMatch) {
const lagSeconds = parseFloat(lagMatch[1])
return lagSeconds < 300 // Alert if lag > 5 minutes
}
return true
} catch {
return false
}
}
Then expose this via an HTTP endpoint:
// app/api/health/route.ts
export async function GET() {
const livestreamOk = await checkLivestreamHealth()
return Response.json(
{ status: livestreamOk ? 'ok' : 'degraded', litestream: livestreamOk },
{ status: livestreamOk ? 200 : 503 }
)
}
Approach 2: Litestream Metrics Endpoint
Litestream exposes Prometheus metrics at a configurable port. Add this to your litestream.yml:
addr: ":9090"
dbs:
- path: /data/db.sqlite
replicas:
- url: s3://your-bucket/db
Now http://localhost:9090/metrics is available. You can monitor this with Vigilmon if your server is externally accessible:
- URL:
http://your-server.com:9090/metrics - Interval: 5 minutes
- Expected status: 200
Note: Only expose the metrics port externally if you have proper firewall rules or authentication in front of it.
Approach 3: S3 Last-Modified Check (Serverless/Script)
For a simple check, verify that your S3 replica was updated recently:
#!/bin/bash
# Check if litestream replica was updated in the last 10 minutes
LAST_MODIFIED=$(aws s3api head-object \n --bucket your-bucket \n --key db \n --query 'LastModified' \n --output text)
LAST_TS=$(date -d "$LAST_MODIFIED" +%s)
NOW=$(date +%s)
DIFF=$((NOW - LAST_TS))
if [ $DIFF -gt 600 ]; then
echo "ALERT: Litestream replication appears stalled ($DIFF seconds since last write)"
exit 1
fi
echo "OK: Last replicated $DIFF seconds ago"
Run this as a cron job and have it POST to a Vigilmon webhook on failure.
Monitoring Your Application with Vigilmon
Regardless of how you monitor Litestream's replication, monitor your application's uptime with Vigilmon:
- Sign in at vigilmon.online
- Add a monitor:
-
URL:
https://your-app.com/api/health(include Litestream check in this endpoint) - Interval: 5 minutes
- Expected status: 200 (returns 503 if Litestream is degraded)
-
URL:
Alerting on Replication Lag
The most important alert for Litestream is replication lag — how far behind the replica is from the primary. Configure your health endpoint to return 503 if lag exceeds your RPO (Recovery Point Objective):
- For most indie apps: lag > 5 minutes is worth alerting
- For financial data: lag > 1 minute should alert
- For low-write apps: lag > 30 minutes may be acceptable
What Vigilmon Monitors for a Litestream Setup
| What | How | Interval |
|---|---|---|
| App uptime | HTTP check on /health
|
1 min |
| Replication health | Health endpoint returns 503 if lagged | 5 min |
| Metrics endpoint | Direct Prometheus endpoint check | 5 min |
| SSL certificate | TLS cert expiry | Daily |
Summary
Litestream is a lightweight replication solution that works silently — which is great until it stops working silently. Adding external monitoring with Vigilmon and a custom health endpoint closes the gap between "replication broken" and "I know about it."
Set up monitoring for your Litestream-powered app at vigilmon.online.
Top comments (0)