The First 30 Minutes: What a 3-Person Team Does When Prod Breaks
No on-call rotation. No observability budget. Just three people and a pager that lives in someone's head. Here is the runbook that keeps small teams sane when production breaks.
Minute 0-5: Stabilize the room
- One person owns the incident (the Incident Lead). Everyone else executes.
- Say the magic sentence: "We have an incident, ETA updates every 10 minutes." Then actually post updates every 10 minutes, even if the update is "still investigating."
Minute 5-15: Triage with three questions
- Is money moving? (checkout, payments) -> highest priority, always.
- Is data safe? (no corruption, no loss)
- Who is affected? (all users vs one segment)
Minute 15-30: Communicate before you fix
- Post a status note BEFORE you know the cause. Users forgive bugs; they don't forgive silence.
- Rollback beats root-cause at this stage. Restore last known good, diagnose later.
After 30: The 5-line postmortem
What happened / impact / timeline / fix / prevention. If it takes more than 30 minutes to write, you're writing a novel, not a postmortem.
Free kit: the First 30 Minutes incident response kit packs this runbook into templates you can copy today. The full ops library (runbooks, sender engine, SOPs) lives in the Hive80 Ops Kit.
Top comments (0)