Every incident you run generates two kinds of data. One kind gets captured with real rigor: the timeline, the root cause, the action items, the MTTR. The other kind evaporates by breakfast — what the incident actually cost the person who ran it.
Sleep lost. The two hours the next morning where you're technically at your desk and functionally useless. The specific service that has now dragged you out of bed three times this quarter. That's operational data too. Most teams never write a line of it down — so the human cost of on-call stays invisible, uninstrumented, and un-arguable, right up until someone quits and it becomes very visible all at once.
We instrument everything else about a service. This is the same instinct, pointed at the rotation itself. Here's the protocol I actually use. It's three artifacts, smallest first.
1. The night-of debrief (2 minutes, before you sleep)
The goal is to capture the raw state before you rationalize it in the morning — because you will. By 10am "that was brutal" has quietly become "eh, it was fine." Five lines, in whatever note app is already open:
- Which incident. A one-line descriptor or ticket number — enough to find it again later.
- When it paged me. (The timestamp matters later — it's how a pattern becomes a number.)
- When I got back to sleep — or gave up trying. The single number that turns "rough night" into data.
- The one thing that made it harder than it needed to be. Stale runbook, a verbal handoff that left something out, an alert that told you nothing. This line is the backlog.
- How I'm functioning right now, 1–5. (1 = can't see straight, 5 = handled it, fine.) Rate it tonight, before the morning rationalizes it away.
Notice what's not on the list: what you checked, what fixed it, the root cause. That's the postmortem's job, and it's already covered. This log is only the part nothing else records — the cost.
That's it. Don't make it a form. If it takes longer than two minutes you won't do it at 3am, and 3am is the only time the data is honest.
2. The structured review (24–48h later)
Once you've slept, the night-of note becomes an input to a slightly longer look — not to process feelings, but to spot structure. The questions that earn their place:
- Is this the same failure mode as a previous night, or genuinely new?
- Which signal did you trust that turned out to be lying? The metric or log that pointed the wrong way is the single fastest way to build the "where do I even look" instinct — and you can only spot it once the adrenaline's gone.
- Which line-4 problem (the "made it harder" one) is now showing up repeatedly?
- What's the one change that would make this class of incident cheaper to diagnose next time?
The value only compounds if you keep the notes somewhere you'll reread them. A single debrief is a diary entry. Ten of them, side by side, show you the ignored signal and the wrong first assumption you make every time — patterns that are completely invisible one
incident at a time.
3. The running log (the part that changes conversations)
This is the one that turns a feeling into a number. One row per incident, the columns that matter:
| Date | Service | Woke you? | Min. of sleep lost | Actionable? | Recurrence of… |
|---|---|---|---|---|---|
| 2/14 | checkout-api | yes | 90 | yes | 3rd this quarter |
After a quarter you stop saying "on-call has been rough lately" — a claim your manager can nod at and forget — and starthe team 22 hours of off-hours work and 9 nights of interrupted sleep, 60% of it from two services." That's not acomplaint. That's an operational argument with a business cost attached, and it survives leadership turnover in a way "I'm tired" never does.
Why this is the missing instrument
Your RCA answers what did the system do. Nothing in a normal incident process answers what did it cost the operator — so that cost defaults to zero on every dashboard, and gets "discovered" only during an exit interview. Logging it isn't self-care; it's closing a gap in your telemetry. The operator is part of the system. Right now they're the only part you don't measure.
The debrief above is one of five checks in a free, one-page On-Call Hygiene Checklist I put together — handoff, alerts, debrief, recovery, recurrence, ~two minutes total, readable with no signup: https://steadystate.engineer/checklist/?ref=devto
If you want the debrief artifacts as ready-to-paste templates instead of building your own, there's a $9 pack (the night-of five-liner, the structured review, a blameless team version, and the running log, in text + Word). Optional — the protocol above is the whole method; the pack just saves you the setup.
Provenance, since it's fair to ask: this is AI-drafted and human-edited, built from public on-call discussions and incident-culture writing — no invented author, no fabricated war stories.
The honest question I'm testing: does the core idea — logging what an incident cost the operator, not just what it did to the system — actually generalize across teams, or is it overhead that only helps solo/small-team on-call? Tell me where it breaks.
Top comments (0)