An incident starts before anyone is calm enough to think. That's the trap. The first thirty minutes decide whether you run the incident or the incident runs you — and they're also the minutes when your lead engineer is still reading Slack backlog and your on-call person is trying to remember where the status page lives.
This is the runbook shape we landed on after running small-team incident drills. Field guidance, not law. Adapt the timings.
Minute 0–5: declare and freeze
One sentence in the incident channel: what's broken, who's incident lead, where updates land. Naming a lead matters more than being right about the cause — a room with two leads has zero leads. Freeze deploys unless the fix is a rollback. Most "quick patches" during an active incident add a second incident.
Minute 5–10: severity, honestly
Three levels is enough for a small team. SEV1: customers blocked, all hands. SEV2: degraded, one owner + one watcher. SEV3: cosmetic or internal, schedule it. The failure mode to design against is SEV-inflation: everything is a SEV1 because nothing is written down, so nobody sleeps and nobody escalates the real fires.
Minute 10–20: stabilise one customer-visible surface
Pick the narrowest thing that makes users functional again: a status page update, a cached fallback, a feature flag. Do not chase root cause yet. Root cause hunting before stabilisation is how a 30-minute outage becomes a 4-hour one.
Minute 20–30: the comms beat
A short external update beats a perfect post-mortem published tomorrow. Three lines: what happened, what we're doing, when the next update lands. Then actually land the next update — the second silence is what users remember.
After: the 5-line timeline
Within 24h, five timestamps: detected, declared, mitigated, user-comms, resolved. That's the whole timeline doc for most small-team incidents. Longer documents get written and never read. The five lines get reused at the next tabletop exercise.
The drill is the product
The teams that look calm in incidents aren't calmer people — they've run the sequence more times. A 30-minute tabletop drill every two weeks beats a 30-page plan nobody opens. Run the drill with a fake incident, a timer, and one honest observer who writes down where people hesitated.
We keep our checklists and comms templates public (status page copy, severity matrix, first-30-minutes sheet) — take them, edit them, make them yours:
- Free checklist: https://hive80-lab.github.io/ops-notes/first-30-minutes-incident-response.html
- Full IR plan template for small teams: https://hive80-lab.github.io/ops-notes/incident-response-plan-template-small-teams.html
- Severity matrix: https://hive80-lab.github.io/ops-notes/severity-matrix-3-levels.html
If you want the full pack (runbook templates, comms scripts, drills) in one place, it's here: https://hive80lab.gumroad.com/l/ops-starter-kit
Top comments (0)