The cheapest outage in existence costs nothing to trigger and nothing to prevent. And it takes down sites every single day: an expired TLS certificate.
I've watched a six-person team lose a full morning to it. The site wasn't hacked, the server wasn't down, the database was fine. The padlock had simply aged out overnight, and every browser on Earth started waving a red flag at their customers. Conversion rate went to zero in about eleven minutes. Nobody noticed for four hours, because the monitoring checked that the server answered 200 — and it did, proudly, to curl.
Why this keeps happening to good teams
Certificates expire on a fixed date. That's the entire failure mode. So why does it repeat everywhere?
1. Nobody owns the date. The cert was installed once, by a person, maybe two jobs ago. Expiry lives in a browser warning nobody is looking at.
2. Monitoring watches uptime, not validity. GET /healthz -> 200 says nothing about the certificate. A site can be perfectly up and perfectly broken at the same time.
3. Renewal is manual somewhere in the chain. Even with Let's Encrypt, someone has to make sure the renewal job still runs after the server migration, the cron prune, the DNS move. Silent cron deaths are their own epidemic.
4. The fix is invisible when it works. Renewal that succeeds makes no sound. Renewal that fails makes no sound either. Until Tuesday.
The 10-minute fix, in three layers
Layer 1 — see the date coming (3 minutes). One line, run from anywhere:
echo | openssl s_client -connect yourdomain.com:443 -servername yourdomain.com 2>/dev/null | openssl x509 -noout -enddate
Now automate it: check daily, alert at 30 days (calm), 14 days (loud), 7 days (page someone). Don't alert at 1 day — that's not monitoring, that's a countdown.
Layer 2 — make renewal boring (5 minutes). If you're on Let's Encrypt, certbot renew on a timer is the whole job — but verify the renewal the way you'd verify a backup: run certbot renew --dry-run monthly and alert if it fails. A renewal job you've never tested is a theory, same as a backup you've never restored.
Layer 3 — the fallback runbook (2 minutes). When renewal does fail — expired API token, port 80 blocked, DNS misroute — someone at 9AM needs a card that says exactly what to do, in order, without thinking. Ten lines. Write it once while you're calm.
The part that generalizes
SSL is just the loudest example. The same shape of failure exists for: domain registrations, DNS records, API tokens, signed webhooks, insurance documents, liquor licenses, that one vendor contract. Anything with an expiry date and no owner is a future outage with a scheduled date.
The general fix is one list: every date that can hurt you, one owner, three alarm points. An afternoon to build, and it deletes an entire category of 2AM.
I packaged this as ready-to-run scripts and templates — the SSL/expiry watchdog, the 30/14/7 alert thresholds, dry-run evidence checks, and the fallback runbook — in the Automation Starter Pack. The lighter Ops Starter Kit covers the checklist/runbook side if you'd rather keep it manual, and the Ops Mega Bundle is everything (monitoring + backups + handoffs + runbooks) at bundle pricing. One-time purchase, instant download, no subscription.
More war stories from the same front: dev.to/hive80lab.
Set up the expiry check today. It's the cheapest insurance you will ever deploy — and unlike a certificate, it doesn't expire.
Top comments (0)