DEV Community

Vigilmon
Vigilmon

Posted on • Originally published at vigilmon.online

How to Monitor Your Litestream Replication with Vigilmon

Litestream is a streaming replication tool for SQLite that continuously backs up your database to S3, Azure Blob, Google Cloud Storage, or SFTP. It's a popular choice for indie developers and small teams who want SQLite in production without risking data loss.

But Litestream replication failures are silent by default. If replication stops, your data is safe for now — until a server failure wipes out hours or days of unbackedup writes. This guide explains how to monitor Litestream alongside your app using Vigilmon.

What Can Go Wrong with Litestream

  • The Litestream process crashes or is not started
  • S3 bucket credentials expire or IAM permissions change
  • Network issues block uploads to the replica destination
  • The replica destination reaches a storage quota
  • The Litestream sidecar container exits in a Docker/Kubernetes setup

None of these produce an error your users see — they only matter when you need to restore.

Monitoring Strategy for Litestream

Since Litestream doesn't expose an HTTP health endpoint, you have three monitoring approaches:

Approach 1: Health Endpoint in Your Application

The most reliable approach is to add a health endpoint in your own app that checks Litestream's replication status by examining the replica:

// Example: check when the last WAL frame was replicated
async function checkLivestreamHealth(): Promise<boolean> {
  try {
    // Read the litestream replica stats via its built-in metrics
    const response = await fetch('http://localhost:9090/metrics')
    if (!response.ok) return false
    const text = await response.text()
    // Check that replication lag is not too high
    const lagMatch = text.match(/litestream_replica_lag_secondss+([d.]+)/)
    if (lagMatch) {
      const lagSeconds = parseFloat(lagMatch[1])
      return lagSeconds < 300 // Alert if lag > 5 minutes
    }
    return true
  } catch {
    return false
  }
}
Enter fullscreen mode Exit fullscreen mode

Then expose this via an HTTP endpoint:

// app/api/health/route.ts
export async function GET() {
  const livestreamOk = await checkLivestreamHealth()
  return Response.json(
    { status: livestreamOk ? 'ok' : 'degraded', litestream: livestreamOk },
    { status: livestreamOk ? 200 : 503 }
  )
}
Enter fullscreen mode Exit fullscreen mode

Approach 2: Litestream Metrics Endpoint

Litestream exposes Prometheus metrics at a configurable port. Add this to your litestream.yml:

addr: ":9090"

dbs:
  - path: /data/db.sqlite
    replicas:
      - url: s3://your-bucket/db
Enter fullscreen mode Exit fullscreen mode

Now http://localhost:9090/metrics is available. You can monitor this with Vigilmon if your server is externally accessible:

  • URL: http://your-server.com:9090/metrics
  • Interval: 5 minutes
  • Expected status: 200

Note: Only expose the metrics port externally if you have proper firewall rules or authentication in front of it.

Approach 3: S3 Last-Modified Check (Serverless/Script)

For a simple check, verify that your S3 replica was updated recently:

#!/bin/bash
# Check if litestream replica was updated in the last 10 minutes
LAST_MODIFIED=$(aws s3api head-object \n  --bucket your-bucket \n  --key db \n  --query 'LastModified' \n  --output text)

LAST_TS=$(date -d "$LAST_MODIFIED" +%s)
NOW=$(date +%s)
DIFF=$((NOW - LAST_TS))

if [ $DIFF -gt 600 ]; then
  echo "ALERT: Litestream replication appears stalled ($DIFF seconds since last write)"
  exit 1
fi

echo "OK: Last replicated $DIFF seconds ago"
Enter fullscreen mode Exit fullscreen mode

Run this as a cron job and have it POST to a Vigilmon webhook on failure.

Monitoring Your Application with Vigilmon

Regardless of how you monitor Litestream's replication, monitor your application's uptime with Vigilmon:

  1. Sign in at vigilmon.online
  2. Add a monitor:
    • URL: https://your-app.com/api/health (include Litestream check in this endpoint)
    • Interval: 5 minutes
    • Expected status: 200 (returns 503 if Litestream is degraded)

Alerting on Replication Lag

The most important alert for Litestream is replication lag — how far behind the replica is from the primary. Configure your health endpoint to return 503 if lag exceeds your RPO (Recovery Point Objective):

  • For most indie apps: lag > 5 minutes is worth alerting
  • For financial data: lag > 1 minute should alert
  • For low-write apps: lag > 30 minutes may be acceptable

What Vigilmon Monitors for a Litestream Setup

What How Interval
App uptime HTTP check on /health 1 min
Replication health Health endpoint returns 503 if lagged 5 min
Metrics endpoint Direct Prometheus endpoint check 5 min
SSL certificate TLS cert expiry Daily

Summary

Litestream is a lightweight replication solution that works silently — which is great until it stops working silently. Adding external monitoring with Vigilmon and a custom health endpoint closes the gap between "replication broken" and "I know about it."

Set up monitoring for your Litestream-powered app at vigilmon.online.

Top comments (0)