I Automated My Weekly Server Checklist With 50 Lines of Python (Here's the Script)

Every sysadmin has one: the weekly checklist you do by hand. Disk space, cert expiry, failed services, stale backups. It eats an hour a week and fails exactly when you're on vacation. Here's the 50-line Python script that replaced mine — and the checklist format that makes it work.

The Weekly Ops Checklist (automate these 5 first)

1. Disk space — alert at 80%, not 99%
2. Certificate expiry — the outage that was 100% preventable
3. Failed systemd units — yesterday's crash, today's outage
4. Backup freshness — a backup older than 24h is a rumor
5. Log volume spikes — the early-warning signal nobody watches

The Script

#!/usr/bin/env python3
"""weekly-ops-check.py — the 5 checks, one pass, one email."""
import subprocess, shutil, datetime, smtplib
from email.message import EmailMessage

THRESHOLDS = {"disk_pct": 80, "cert_days": 14, "backup_max_age_h": 24}
failures = []

def check_disk():
    total, used, free = shutil.disk_usage("/")[0:3]
    pct = used / total * 100
    if pct > THRESHOLDS["disk_pct"]:
        failures.append(f"DISK: {pct:.0f}% used (threshold {THRESHOLDS['disk_pct']}%)")

def check_certs(domain):
    out = subprocess.run(
        ["curl", "-vsI", f"https://{domain}", "--max-time", "10"],
        capture_output=True, text=True).stderr
    for line in out.splitlines():
        if "expire date" in line:
            exp = datetime.datetime.strptime(line.split(":")[1].strip()[:14], "%b %e %H:%M:%S")
            days = (exp - datetime.datetime.utcnow()).days
            if days < THRESHOLDS["cert_days"]:
                failures.append(f"CERT {domain}: {days} days left")
            break

def check_failed_units():
    out = subprocess.run(["systemctl", "list-units", "--failed", "--no-legend"],
                         capture_output=True, text=True).stdout.strip()
    if out:
        failures.append(f"FAILED UNITS: {out}")

def check_backup_age(path):
    age_h = (datetime.datetime.now() - datetime.datetime.fromtimestamp(
        __import__('os').path.getmtime(path))).total_seconds() / 3600
    if age_h > THRESHOLDS["backup_max_age_h"]:
        failures.append(f"BACKUP: {age_h:.0f}h old (max {THRESHOLDS['backup_max_age_h']}h)")

check_disk()
check_certs("example.com")
check_failed_units()
check_backup_age("/var/backups/latest.tar.gz")

print("ALL CLEAR" if not failures else "\n".join(failures))

Cron it Monday 8am. One line of output: either ALL CLEAR or exactly what's broken. No dashboard login, no SSH safari.

Why This Pattern Wins

The magic isn't the code — it's that each check has a threshold and a verb. "Disk at 80% → expand or clean" beats "keep an eye on disk." When every check carries its own action, the script output is the runbook.

Start with the 5 checks above. Resist adding more until each existing check has survived a real incident.

The Full Toolkit

This script is one module in our Ops Starter Kit — runbook templates, alert hygiene, incident response checklists, and the monitoring patterns from our 3AM incident playbook.

Copy it, cron it, reclaim your Monday.

What's still on your manual checklist? Comment below — if it's cron-able, someone here has the script.