The most reliable system I ever helped run was held together by three people who were slowly falling apart. On paper the on-call rotation looked healthy, a name for every week, a schedule that covered the calendar. In practice the pages did not distribute evenly, because the hard problems always landed on whoever actually understood the system, and that was the same three engineers regardless of whose week it officially was. They got called at night on their weeks and consulted at night on everyone else's. The rotation was a fiction, and it was quietly grinding down exactly the people we could least afford to lose.
Nobody planned this. It happened the way most burnout happens, one reasonable exception at a time. A junior on call hits something scary and pings a senior, because that is the sensible thing to do at 3am. The senior helps, because that is the decent thing to do. Repeat that a hundred times and you have built an informal always-on rotation for your most senior people on top of the formal one, and it appears nowhere on any schedule, so nobody is accounting for its cost.
The damage does not show up in the metrics you usually watch. Uptime stayed excellent, which was precisely the problem, because good numbers hid a team that was fraying. What you see instead is subtler and slower. The best engineer starts taking longer to reply. Their pull requests get terser. One day they mention, very casually, that they have been talking to a recruiter, and you realize the reliability you were so proud of was being paid for out of a budget you never tracked, the finite patience of a few exhausted humans.
Now I treat on-call load as a first-class operational signal, not an HR afterthought. Who is actually getting paged, at what hours, and is it the same names every time regardless of the roster. If the answer concentrates on a few people, the fix is not a pep talk. It is spreading the knowledge that makes them indispensable, so the pages can genuinely land on whoever is on call.
A rotation that always ends up waking the same three people is not a rotation. It is a slow way of losing them.
– Serguey Shinder
Top comments (0)