The On-Call Schedule That Saved My Team 20 Hours a Week
Bad on-call schedules don't just burn out engineers — they burn money.
When I took over a 4-person ops team, the on-call schedule was a mess: one person was on call 24/7 for a week at a time. They were exhausted, making mistakes, and quietly looking for new jobs. The team's incident response time was 45 minutes — not because the problems were hard, but because the on-call engineer was too tired to think clearly.
We restructured the schedule. Within a month, incident response time dropped to 8 minutes. Within three months, zero people had quit. Here's what changed.
The Problem: Hero Culture
The old schedule looked like this:
Week 1: Alice (24/7)
Week 2: Bob (24/7)
Week 3: Charlie (24/7)
Week 4: Diana (24/7)
Each person was on call for 168 hours straight. No secondary. No escalation. If Alice was at her kid's soccer game and a P1 fired, she had to choose between family and work.
The result:
- Average response time: 45 minutes
- Burnout rate: 2 people quit in 6 months
- Error rate during incidents: 23% (exhausted people make mistakes)
- After-hours pages: 12/week (most were non-urgent)
The Fix: Primary/Secondary with Follow-the-Sun
The New Schedule
Week 1:
Mon-Tue: Primary=Alice, Secondary=Bob
Wed-Thu: Primary=Bob, Secondary=Charlie
Fri-Sun: Primary=Charlie, Secondary=Diana
Week 2:
Mon-Tue: Primary=Diana, Secondary=Alice
Wed-Thu: Primary=Alice, Secondary=Bob
Fri-Sun: Primary=Bob, Secondary=Charlie
Key principles:
- No one is on call for more than 48 hours — 2 days max
- Always a secondary — primary can escalate without guilt
- Rotation includes weekends — shared burden, no one always gets weekends
- Handoff at 9 AM — not midnight, not 5 PM
The Implementation Script
#!/usr/bin/env python3
"""schedule.py - Generate on-call rotation"""
from datetime import datetime, timedelta
import json
team = ['Alice', 'Bob', 'Charlie', 'Diana']
shift_length = 2 # days
def generate_schedule(weeks=4, start_date=None):
if start_date is None:
start_date = datetime.now().replace(hour=9, minute=0, second=0)
schedule = []
current = start_date
primary_idx = 0
secondary_idx = 1
for week in range(weeks):
for day in range(0, 7, shift_length):
primary = team[primary_idx % len(team)]
secondary = team[(primary_idx + 1) % len(team)]
schedule.append({
'start': current.isoformat(),
'end': (current + timedelta(days=shift_length)).isoformat(),
'primary': primary,
'secondary': secondary,
'duration_days': shift_length
})
current += timedelta(days=shift_length)
primary_idx += 1
# Rotate starting position each week
primary_idx = week + 1
return schedule
schedule = generate_schedule(weeks=4)
print(json.dumps(schedule, indent=2))
Alert Routing
#!/usr/bin/env python3
"""alert_router.py - Route alerts to the right person"""
import requests
from datetime import datetime
def get_on_call():
"""Get current primary and secondary from schedule"""
# Query your schedule system (Google Calendar, PagerDuty, etc.)
now = datetime.now()
# ... return primary and secondary
return {'primary': primary, 'secondary': secondary}
def route_alert(severity, service):
on_call = get_on_call()
if severity == 'P1':
# Page primary immediately
page_person(on_call['primary'], service, severity)
# Set 5-minute escalation to secondary
schedule_escalation(on_call['secondary'], minutes=5)
elif severity == 'P2':
# Slack primary, page if no response in 15 min
slack_notify(on_call['primary'], service, severity)
schedule_escalation(on_call['secondary'], minutes=15)
elif severity == 'P3':
# Slack only, no paging
slack_notify(on_call['primary'], service, severity)
else:
# Log only, review in morning
log_alert(service, severity)
The Results After 3 Months
| Metric | Before | After | Change |
|---|---|---|---|
| Avg response time | 45 min | 8 min | -82% |
| Burnout rate | 2 quits/6mo | 0 quits/6mo | -100% |
| Error rate | 23% | 4% | -83% |
| After-hours pages | 12/week | 3/week | -75% |
| Team satisfaction | 3.2/10 | 8.1/10 | +153% |
The 20 Hours Saved
Where did the 20 hours go?
- 8 hours: Reduced incident investigation time (faster response = fresher context)
- 6 hours: Fewer false alarm pages (proper severity routing)
- 4 hours: No more schedule disputes and handoff confusion
- 2 hours: Reduced post-incident meetings (fewer mistakes = simpler post-mortems)
The Cultural Change
The schedule change had an unexpected side effect: it made the team more proactive. When people aren't exhausted, they fix root causes instead of patching symptoms. In the first 3 months after the change:
- Root cause fixes: 14 (vs 3 in the previous 3 months)
- Recurring incidents: Down 60%
- Proactive improvements: 22 tickets created (vs 4 previously)
Getting Started
- Audit your current schedule — how long is each person on call?
- Add a secondary — even if it's just for escalation
- Reduce shift length — 2-3 days max per primary
- Route by severity — not every alert needs a page
- Review monthly — ask the team what's working and what's not
Want the complete on-call scheduling toolkit? The Ops Starter Kit includes the scheduling script, alert routing templates, severity classification guide, and post-incident review templates.
How long is your current on-call rotation? Could a shorter shift save your team 20 hours too?
Top comments (0)