DEV Community

Hive80-lab
Hive80-lab

Posted on

A Customer Told Me My Site Was Down All Night. It Cost Me $948.

Last Tuesday a customer emailed us at 9am: "your site was down all night, I'm taking my business elsewhere." The site had gone down at 1:14am. It came back at 6:40am. Nobody noticed until the email.

Five hours of downtime. One lost customer. A $79/month customer — $948/year — gone because nobody was watching between 1am and 6am.

If you run a small SaaS or sell anything online, downtime isn't an ops problem. It's a revenue leak with a timestamp.

The math you can't ignore

Say your product makes $3,000/month. Your service is up 99.0% instead of 99.9%. That's 7.2 extra hours down per month. At any reasonable traffic-to-revenue ratio, that's real money leaving — and worse, it leaves quietly: churned users rarely file a ticket, they just stop showing up.

Rule 1: Monitor from outside, not inside

"Inside" monitoring (localhost checks) tells you the server can talk to itself. Useless. Your customers are outside. An external check every 60 seconds from a different network is the minimum honest setup.

Rule 2: Downtime needs a dollar value attached

When your alert says "HTTPS check failed," you feel annoyed. When it says "HTTPS check failed — this is the third 2am failure this month, each costs ~$40 in churn," you feel motivated. Track minutes-down per month next to revenue. One spreadsheet column.

Rule 3: The fix must be automatable before 2am ever comes

Most 2am pages have the same 3 fixes: restart the service, clear the disk, failover the DNS. If your fix requires you specifically, you don't have an ops setup — you have a single point of failure with your name on it. Write the fix as a script, even an ugly one.

Rule 4: Verification, or the "fix" is a rumor

Every automated fix must end with an external check that confirms recovery — not "the script exited 0." We got burned by a restart script that succeeded while the app was still returning 500s. Now: fix runs, external check re-fires, and only a green check closes the incident.

What we ship now

After that Tuesday, we rebuilt our night coverage around one idea: a machine should notice before the customer does — every minute, from outside, with a fix attached.

We packaged the whole thing — external uptime checks, restart scripts, verification steps, and the dollar-per-downtime tracker — as Nightwatch Ops: automated uptime & self-healing monitoring kit. Buy once, deploy tonight, no subscription.

The full catalog of ops/automation kits is at the Hive80Lab store.

What's your worst "found out from a customer" story? Comments open — let's compare scars.


P.S. 99.9% uptime sounds technical. It means "your customers can sleep, and so can you."

Running a stack like this? Our Ops Starter Kit ships the runbooks, checklists, and monitoring patterns behind these posts — grab it and cut your next incident short.

Top comments (0)