Introduction
PostgreSQL 17 introduced native incremental backup support, a major leap forward in database backup strategy. Rather than duplicating the entire dataset every time, incremental backup captures only the data blocks that have changed since the last backup (full or incremental). This drastically reduces backup time, storage consumption, and system overhead. Prior to PostgreSQL 17, achieving this required third-party tools such as pgBackRest or Barman, which added configuration and maintenance overhead. With native support now built into PostgreSQL, the process has become significantly more streamlined.
What Is Incremental Backup?
An incremental backup records only the changes made since the previous backup — whether that was a full backup or an earlier incremental one. Compared to full backups that copy all data regardless of what has changed, incremental backups are leaner, faster, and more storage-efficient.
Key Features in PostgreSQL 17
Native Integration - Incremental backup is now part of PostgreSQL's core, removing the need for external tools for this functionality.
Storage Efficiency - Only modified data pages are backed up, keeping storage usage minimal.
Faster Backups and Recovery - Since less data is processed each time, backup creation is quicker and recovery is streamlined by applying only the required changes on top of the full backup.
How It Works: Step-by-Step
Step 1 - Enable WAL Summarization In the postgresql.conf file, enable the summarize_wal parameter by setting it to on. This activates the WAL summarizer process, which tracks which data blocks have been modified. It can be enabled on either a primary or a standby server. It is set to off by default.
Step 2 - Take a Full Backup Use pg_basebackup to create the initial full backup. This serves as the foundation for all subsequent incremental backups.
Step 3 - Take the First Incremental Backup After inserting or modifying data, run pg_basebackup again with the --incremental flag, pointing to the backup_manifest file from the full backup. This tells PostgreSQL what the baseline was and allows it to capture only the changes since then.
Step 4 - Take Additional Incremental Backups After further data changes, take another incremental backup — this time referencing the backup_manifest from the first incremental backup. Each incremental backup chains to the previous one using its manifest file.
Restoring the Backups
Restoration is handled by pg_combinebackup, a new utility introduced in PostgreSQL 17. It merges the full backup and all incremental backups into a single, usable backup directory. The backups must be provided in chronological order — starting from the full backup, followed by each incremental in sequence. After combining, you adjust the port in the restored directory's postgresql.conf and start the database server using that data directory. Upon verification, all records from the full backup and every incremental backup are present and intact.
What Is pg_combinebackup?
pg_combinebackup is the companion utility that reconstructs a complete, restorable backup from the chain of incremental backups. It automates the merging process and validates the backup chain for consistency, eliminating the need for manual intervention during restoration.
Advantages of Incremental Backup
Cost Savings - Reduced storage usage means lower costs, whether on cloud or on-premises infrastructure.
Improved Performance - Less data transfer reduces system load, making it particularly valuable during peak operational hours.
Scalability - Well-suited for large databases or environments with frequent data changes where full backups would be impractical.
Limitations to Be Aware Of
summarize_wal must be enabled for this feature to work.
Incremental backups only function with pg_basebackup and cannot be taken from a standby server, they must be run on the primary instance.
Restoration depends on a complete, unbroken backup chain. If any backup in the chain is missing, recovery fails.
Backups operate at the cluster level, with no support for per-table backups.
Proper retention of WAL and summary files is required for the feature to function correctly.
Conclusion
Native incremental backup in PostgreSQL 17 addresses two longstanding pain points, storage waste and slow backup windows, while laying a stronger foundation for disaster recovery. The combination of pg_basebackup (with the --incremental flag) and pg_combinebackup makes the entire backup-and-restore workflow cleaner and more efficient, especially for large-scale, high-transaction environments.
Top comments (0)