We fixed it by rebooting. Then it broke worse.

Our biggest outage taught us that 'reboot it' is not a fix — it's evidence destruction with a keyboard shortcut.

The story. A 12-person company's file sync service started throwing latency spikes every afternoon at 2pm. The junior admin rebooted the box. It came back clean. Three days later the spikes returned — but now with a corrupted write-ahead log, because the reboot had interrupted the sync engine mid-transaction. One afternoon of slowness became two days of downtime.

What the post-mortem taught us (the hard way):

1. A reboot erases the crime scene. Memory dumps, open file handles, connection tables — gone. Snapshot before you restart: jstack, netstat, dmesg | tail, a top screenshot. Two minutes of evidence beats two days of guessing.

2. The fix that works once is a hypothesis, not a fix. It returning after reboot doesn't mean reboot solved it — it means the state that caused the problem was reset. Time-to-recurrence is the metric: 3 days = you never found the cause.

3. Afternoon-only slowness = a schedule, not a load problem. Tie the symptom to a clock: backups at 1:30, report cron at 14:00, vendor API quota reset at 13:00 UTC. Calendar correlation finds more root causes than any profiler.

4. Corruption is the tax you pay for interrupting writes. Any service with a WAL/journal should be drained gracefully (systemctl stop and wait, not kill -9) — even when everyone's screaming.

5. Write the recurrence clause. Every incident ticket gets one line: "If this returns within N days, escalate to X and do NOT reboot." It converts the next 2am panic into a procedure.

The unglamorous truth: most "restart fixes" are debt with interest. The spike would have been a 10-minute config fix (a mis-sized connection pool, as it turned out). Instead it cost 2 days.

We turn incidents like this into the checklists and runbooks we publish — if you run lean infrastructure, grab them here:

hive80lab.gumroad.com — ops checklists & free First-30-Minutes incident kit