DEV Community

Hive80-lab
Hive80-lab

Posted on Originally published at hive80-lab.github.io

The backup we 'tested' for three years had never restored once

The backup we'd been "testing" for three years had never restored once.

The audit question was innocent: "when did we last run a live restore?" Not check the backup dashboard says healthy — an actual restore, onto different hardware, timed. The honest answer was never. Three years of green checkmarks in the backup console, and not one byte of it had ever come back out. We changed that in one morning, and turned the exercise into a half-day annual security review that now runs every year, whether anyone feels nervous or not.

Here's what the morning covers, in the order that earns its keep.

Prove the recoveries, not the backups

  • Run one live restore — a file, a mailbox, a database — onto a different machine. Time it. That number is your real recovery time, not the brochure number.
  • Have one person "lose" their phone and recover every critical account. This is where MFA and passkey rollouts quietly break: the front door got stronger while the spare key got lost in a drawer.
  • Verify the break-glass admin account actually works, and that exactly one person knows where its backup factor physically lives.

Rotate the things that rotate

Secrets age like milk. Once a year: every API key and webhook token (unused ones get killed, not rotated — a dead key is the only safe key), shared passwords and router Wi-Fi, and every recovery-code printout older than your last two hires.

Audit who can get in — and who still could

  • Every admin account maps to a current, employed human. Exceptions get disabled the same day.
  • Demote from admin anyone who doesn't need it. Every demoted account is an account a phisher can't use to ruin your quarter.
  • Re-screen critical vendors with the same 12 questions each year, and confirm the vendors you fired are still gone. Ghosts come back through integrations, not invoices.
  • MFA everywhere it's supported, passkeys where allowed, SMS-only recovery killed wherever a stronger path exists.

Test the humans before the phishers do

Send one internal fake-invoice email. Not to punish — to measure. The number that matters isn't who clicked; it's whether anyone flagged it and whether anyone forwarded it to finance before paying. Then walk the first hour of your phishing response on paper: who gets called, in what order, from a list that still has current phone numbers on it.

Check the machinery nobody looks at

Router firmware, the NAS in the cupboard, certificate and domain expiry dates (an expired cert looks identical to a hack from the customer's side), log retention that matches what you'd claim in an audit, and every phone number in the incident plan.

Then write one page

The review ends with a single page: tested (with dates), fixed (what changed), accepted (risks you're consciously keeping, each with an owner and a revisit date). "We accept the shared printer password until March" written down is a decision. The same fact unwritten is a future surprise with better timing than your insurance renewal.

The rule we landed on: a security review you can't hand to someone else didn't happen. The page becomes what you attach to the cyber insurance application, what you give the next hire, and what you wish you'd had at 2AM.

The full one-morning checklist — restore test, rotation pass, access audit, phish drill, machinery, and the one-page record — is on our ops notes site: Annual Security Review Checklist for Small Business.

If you want the incident-response muscle around it, the Ops Starter Kit gives you the fillable plan, severity matrix, and comms templates in about 20 minutes — and the free First 30 Minutes one-pager is the quick-start version. Launch week: 30% off any paid kit with code HIVE-LAUNCH30.

Be honest: when did you last restore from backup — not check the dashboard, but actually pull data back out? We're now the kind of team that can answer with a date. It took one morning to become that team.

Top comments (0)