A nightly pg_dump exits zero. That proves a backup command ran—not that the system is recoverable.
A useful restore-verification contract defines:
- RPO and RTO
- covered databases, roles, extensions, and keys
- an isolated clean restore
- schema, integrity, permissions, and representative query checks
- backup age and restore-verification age
- artifact checksum and recovery-point evidence
- ownership for failed drills
Test corrupt archives, missing keys, version mismatch, full disks, expired credentials, and interrupted restores. Then verify application and MCP authorization with the same roles and approved operations used in production.
A database that starts with the wrong grants or stale schema is not recovered.
Full guide: PostgreSQL for AI workloads needs a restore-verification contract
Top comments (1)
Good addition to the checklist: weight the restore contract by where the database sits in your dependency graph. I once sat through a restore of the DB backing our login service. The restore itself "succeeded" and took only a few minutes, but during those minutes nobody could authenticate, so effectively every customer was down, including on parts of the product that had nothing to do with that database. RTO on paper looked acceptable; RTO as experienced by customers was a full outage.
So I'd add two things to the drill: measure restore time against production-sized data, not a small dev dump, and write down what is unavailable while it runs. For anything in the critical auth path that number may tell you dump/restore isn't your recovery mechanism at all — you want a standby or PITR you can fail over to, with the dump as the last resort.