The 3AM page fires. The last thing you want is a 40-page runbook written by someone who left in 2022. You want one printed page with nine lines on it. Here is the checklist pattern on-call engineers actually tape to their desks — steal it.
1. Is it actually down, or just slow?
One curl against the health endpoint. 5 seconds. Don't open dashboards yet.
2. What changed in the last 2 hours?
Deploys, DNS, cert renewals, cron. 80% of incidents live here.
3. One rollback or one restart — pick ONE.
Never both in the same 10-minute window. You'll never know which one fixed it, and you'll never know which one broke the next thing.
4. Check the boring trio: disk, memory, cert expiry.
Disk-full is still the #1 "mystery outage" in small teams. It's never a mystery.
5. Write one line to the incident channel BEFORE you fix anything.
"What I see: X. What I'm about to do: Y." This one habit saves postmortems.
6. If fix #1 fails, stop and escalate.
Two failed attempts at 3AM is a pattern, not a streak. Escalate.
7. Confirm recovery with the SAME probe that detected the failure.
Different probes lie to each other.
8. Post the timeline while it's fresh.
Three lines: detected / acted / resolved. Future-you writes the postmortem in 10 minutes instead of an hour.
9. Schedule the permanent fix the same morning.
An incident without a follow-up ticket is an incident scheduled for a rematch.
Want the printable version? We packaged this into the Ops Starter Kit — the same desk-tape checklists plus on-call runbook templates for small teams. If you need the full 24/7 monitoring playbook, Agent Ops 24/7 covers escalation ladders, health probes, and the audit cadence from this series.
What's the first line on YOUR 3AM card? Drop it in the comments — the best ones get folded into the next revision.
Top comments (0)