Most small teams copied their recovery numbers from an enterprise template: RTO 4 hours, RPO 15 minutes. Then the disk dies and the real numbers turn out to be RPO 22 hours (the nightly dump ran at 02:00) and RTO 19 hours (nobody had ever restored it).
The declared numbers were not a plan — they were a mood. RTO and RPO are useful exactly when treated as prices: RPO is how much work you can afford to lose, RTO is how long your customers can afford to wait, and every 10× improvement in either costs real money every month.
You can read the full note on the HIVE80lab ops-notes site — here is the working version.
What the two numbers actually mean
- RPO — how much work disappears. The gap between the last restorable copy and the moment of failure. RPO 24h with a nightly dump means a full day of customer data evaporates, and support spends the next week re-keying it with them.
- RTO — how long until customers can work again. Not until the server boots. A VM that's up in 10 minutes but serves errors for six more hours has an RTO of six hours. The finish line is the service catalog promise — "accepts payments" — being kept again.
- The third number: measured RTO. The only RTO that exists is the one you've timed on a real restore. Declared minus measured isn't a rounding error — it's the risk.
Set them per tier, not per company
One pair of numbers for the whole company is wrong by construction: the checkout database and the internal admin tool have wildly different value per hour. Anchor the numbers to the same tiers that decide paging:
- Tier 1 (customers feel it in minutes): RPO in minutes — streaming replication or managed point-in-time recovery. RTO in tens of minutes — a failover that has been rehearsed, with the promotion steps on one page.
- Tier 2 (workaround exists): RPO up to 24h — nightly dump with a tested restore. RTO up to a business day.
- Tier 3 (internal): RPO a week, RTO best effort, said out loud. Pretending otherwise spends tier-1 money on tier-3 comfort.
The test for each number is affordability: price the next 10× (nightly dump ≈ free; hourly shipping ≈ small; continuous PITR ≈ real money; warm standby ≈ capacity plus rehearsal). A number you can't afford to implement isn't an objective — it's a wish with a footer.
The five traps
Enterprise numbers on small-team plumbing. "RPO 15 minutes" written above a nightly cron job is fiction with a footer. Measure the current gap first, declare second, buy third.
RTO measured to "server up." The customer's clock doesn't stop at boot. Index rebuilds, queue replay, and "where did my Tuesday go?" all count.
Backups never restored. A backup is a hope until a drill runs it end to end — the same law as the untested rollback script. The first drill almost always finds the dump is fine and the path around it (credentials, disk space, ordering, network route) is not.
One number company-wide. The DR PDF says 4 hours; the checkout queue disagrees. Declare per service, per tier.
Silent decay. The RTO was honest when the database was 20 GB. Two years later the restore takes nine hours and overlaps the business day. RTO/RPO rot as data grows — the drill is a calendar event, not a memory.
Worked example
An eight-person B2B expense tool declares "RTO 4h, RPO 1h" in its security questionnaire. The plumbing: nightly dump at 02:00, never restored, same availability zone as the primary. A Thursday-evening disk failure costs 17 hours of customer data and 22 hours of recovery — mostly a permissions puzzle and a 14-hour dump restore onto a fresh host.
The redo takes one afternoon and $85/month: restore the existing dump into a scratch host and measure (3h40m, mostly transfer); move the tier-1 database to managed point-in-time recovery (RPO 5 min, provider-backed); write and rehearse a read-replica promotion runbook (41 minutes end to end, drill-timed). New per-tier declarations: checkout RPO 5 min / RTO 1h; reporting RPO 24h / RTO 8h; internal tools best effort.
Six weeks later a provider outage hits: 40 minutes of replica lag, one status-page update, zero lost data. The counter-example is the competitor whose DR PDF promised RPO 15 minutes while the "replication" was an rsync cron job to a server in the same rack — their real outage lost three days, and the PDF lives on as a sales artifact.
From the HIVE80lab kit
- Score your incident response free — 12 questions, tiered fix-first plan
- The First 30 Minutes — free incident quick-start
- Ops Starter Kit — incident response for small teams — $14
- Ops Starter Kit Vol. 2 — advanced incident response & communications — $27
- Ops Mega Bundle — all 5 kits in one download — $49
Related: the disaster recovery plan template for small teams is the document these numbers anchor, the backup restore drill checklist produces the measured RTO, and the restore test template makes the drill a record instead of a memory.
Top comments (0)