PostgreSQL Streaming WAL Archiver and a backup tool in Go (pgrwl)

A production-grade, cloud-native PostgreSQL WAL archiving system designed for:

• streaming WAL to S3 with compression, encryption, and retention
• Kubernetes-native PostgreSQL backup workflows
• zero data loss and reliable Point-in-Time Recovery (PITR)

---

Project

https://github.com/pgrwl/pgrwl

---

Features

• WAL receiver (replication protocol)
• Continuous WAL streaming
• Backup to S3 (MinIO, AWS, etc.)
• Backup to SFTP (backup server)
• WAL compression (gzip, zstd-ready)
• WAL encryption (AES-GCM)
• WAL retention management
• WAL monitoring and observability
• Kubernetes & container ready
• Helm chart support
• YAML / JSON / ENV config
• Lightweight single binary
• Structured logging
• Integration tests (containerized)
• Unit tests
• Backup automation (streaming basebackup)
• Continuous backup for PostgreSQL

---

Key Capabilities

Streaming WAL

• Uses PostgreSQL replication protocol
• Supports synchronous WAL streaming
• Enables zero data loss setups

Storage Backends

• S3-compatible storage
• SFTP backup servers

Compression + Encryption

Pipeline based on filename:

000000010000000000000001.gz.aes

Flow:

• compress -> encrypt -> upload
• download -> decrypt -> decompress

---

Architecture

PostgreSQL
   | (replication protocol)
WAL Receiver
   |
Local FS (fsync)
   |
Uploader (S3 / SFTP)
   |
Retention manager
   |
HTTP server (restore_command)

---

Continuous Backup

• real-time WAL streaming
• safe off-site storage
• full PITR support
• near-zero RPO

---

Kubernetes Ready

• run as StatefulSet
• works with StatefulSets / CNPG / Virtual Machines
• deploy via Helm
• GitOps-friendly

---

Configuration Example

main:
  listen_port: 7070
  directory: wals
receiver:
  slot: pgrwl_v5
log:
  level: trace
  format: text
  add_source: true

---

Testing

• integration tests with real PostgreSQL containers
• end-to-end WAL validation
• unit-tested components

---

Why pgrwl?

• simple deployment (single binary)
• production-grade reliability
• cloud-native design
• built for Kubernetes and containers
• secure and efficient WAL handling

---

Contribute

• Star the repo (https://github.com/pgrwl/pgrwl)
• Open issues (https://github.com/pgrwl/pgrwl/issues)
• Suggest improvements
• Submit PRs (https://github.com/pgrwl/pgrwl/blob/master/CONTRIBUTING.md)

---

Summary

pgrwl is a lightweight, powerful, production-ready WAL archiving solution that brings:

• streaming
• security
• automation
• observability

to PostgreSQL backups.