The Handover Between Two Teams Is Where Incidents Go to Die

The incident lasted nine hours. About forty minutes of that was diagnosis and roughly fifteen minutes was the fix. The rest was the ticket sitting in a queue while two teams each waited for the other to do something.

I have reread that timeline several times because it is such a clean example of a failure mode nobody owns. The service desk raised it correctly. Application support looked at it, concluded the fault was in the network path, and reassigned it. The network team looked at their side, found nothing wrong, wrote "no issues observed on our infrastructure" and sent it back. That round trip happened three times over an afternoon. Nobody was negligent. Every person involved did the thing their role told them to do, closed their part, and moved on to the next item. The customer, meanwhile, was down the entire time, and no single person in the chain ever held the whole problem.

The structural cause is that we assign tickets rather than assign ownership. A ticket can be moved. Ownership cannot be moved by clicking a button, which is exactly why nobody moves it. So the incident becomes a parcel being passed between departments, and the parcel has no advocate.

The change that fixed this for us was unglamorous. Any incident that changes hands more than once gets an incident lead attached, and the lead does not change when the ticket does. That person does not need to know how to fix it. Their job is to keep both teams in the same conversation, to insist on a joint look rather than sequential looks, and to be the one embarrassed by the elapsed time. We also stopped accepting "not us" as a resolution. If your team's investigation clears your layer, you say what you actually checked and what that rules out, so the next team starts from evidence instead of starting over.

Mean time to repair is usually reported as a technical number. In my experience most of it is organisational. The minutes we lose are rarely spent thinking about the fault. They are spent waiting for somebody to decide the fault is theirs.

– Serguey Shinder