The latest articles on DEV Community by Fabio Ghidini (@fabio_ghidini_fb1c1cced68). https://dev.to/fabio_ghidini_fb1c1cced68 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1548726%2Fbbcf7ddc-bccc-4470-aa74-21d14ae0dfbd.png https://dev.to/fabio_ghidini_fb1c1cced68 en Fabio Ghidini Wed, 24 Jun 2026 08:38:29 +0000 https://dev.to/fabio_ghidini_fb1c1cced68/a-single-probe-saying-down-shouldnt-wake-you-at-3am-1p1f https://dev.to/fabio_ghidini_fb1c1cced68/a-single-probe-saying-down-shouldnt-wake-you-at-3am-1p1f <p>Most uptime monitors work the same way: one probe somewhere checks your site, and if that probe can't reach it, you get paged. I ran tools like that for years. Maybe half the "down" alerts I got at 3am were the probe's own network having a bad minute — not my site.</p> <p>A check from a single location is one machine's opinion at one moment. A local routing hiccup, a flaky peering link, a momentary DNS blip on the probe's side: from one vantage point they all look exactly like "your site is down."</p> <p>So when I built <a href="https://www.sonarops.it" rel="noopener noreferrer">SonarOps</a>, I made the confirmation cascade across regions. Checks run every 60 seconds. When one probe sees an outage, a second probe in another region (EU and USA) re-checks before any alert fires. Second probe also can't reach you? It's real, you get paged. It can? The first probe just had a bad moment, and you stay asleep.</p> <p>It isn't a vote or a quorum. It's a cascade: one probe raises a hand, a second one somewhere else confirms before I bother you.</p> <p>The tradeoff is honest: that step costs a second or two of detection delay. I'll take it. I'd rather hear about a real outage two seconds later than get woken for a phantom one that fixes itself before I've opened the laptop.</p> <p>Three things I learned building this:</p> <ul> <li> <strong>Retrying on the same probe isn't enough.</strong> If the probe's local network is the problem, a retry from the same place just confirms its bad minute.</li> <li> <strong>Geography beats count.</strong> Two probes in one datacenter aren't independent. Two in different regions are.</li> <li> <strong>Most false pages are network, not server.</strong> Once I cross-checked across regions, the 3am noise basically went away.</li> </ul> <p>Full version — how cross-region monitoring works and where single-probe checks break: <a href="https://www.sonarops.it/blog/multi-location-monitoring" rel="noopener noreferrer">Multi-location monitoring guide</a>.</p> <p>And if you just want to play with the numbers, I keep a few free calculators (no signup) — uptime SLA, downtime cost, error budget: <a href="https://www.sonarops.it/tools" rel="noopener noreferrer">sonarops.it/tools</a>.</p> devops sre monitoring webdev