When managing database backups, key pain points on entry-level servers are memory spikes (Out of Memory crashes) and running out of disk space due to temporary dump files.
To validate the efficiency of LunarDump (a zero-disk streaming backup solution), I conducted a benchmark & memory profiling using Memray across a multi-cloud setup based in Jakarta.
🖥️ Test Environment Setup:
- Server A (Database): Tencent Cloud (Jakarta) — Ubuntu 24.04, 2 vCPU, 2 GB RAM, 40 GB SSD
- Server B (LunarDump Engine): Cloudeka (Jakarta) — Ubuntu 24.04, 1 vCPU, 1 GB RAM, 20 GB SSD
- Destina tion Storage: Google Cloud Storage (GCS) Bucket
đź“Š Benchmark Highlights:
- Total Data Processed: ~8 GB (MySQL Database Stream)
- Execution Duration: 1 Hour 1 Minute (Continuous Cross-Cloud Streaming)
- Peak Memory Usage (Heap): 119.1 MB (~1.4% of total dataset size)
- Resident Set Size (RSS): Stable at ~180 MB
- Local Disk Footprint: 0 Bytes (Data streamed from Server A, encrypted with AES-256-GCM in Server B's RAM, and uploaded directly to GCS on-the-fly)
- Server B CPU Load: Averaged 4% – 8%
đź’ˇ Key Takeaways:
Through Flamegraph visualizations and Resident Set Size Over Time analysis, LunarDump demonstrated Zero Memory Leak. The flat memory footprint confirms that garbage collection and chunk-based buffer recycling function flawlessly—keeping resource consumption predictable even on a low-spec 1 vCPU/ 1 GB RAM node.
Github : lunardump
Top comments (2)
The bounded memory result is useful, but for backups the stronger benchmark is recoverability under interruption. Zero local disk removes one failure mode and also removes a retry checkpoint, so I would kill the process and network at random offsets, resume or restart, then restore the resulting object into an isolated MySQL instance.
The restore gate should verify more than “the dump imports”: source snapshot consistency (including the binlog/GTID coordinate), authenticated chunk ordering, unique GCM nonce use, an end-to-end digest, row counts and sampled data, schema objects/routines, and explicit cleanup of incomplete multipart uploads. Running the source under concurrent writes would show whether the stream represents one coherent database state rather than an hour-long mixture.
That would turn the 119 MB result from a good transport benchmark into evidence for an operational backup contract: bounded resources, known RPO, restart behavior, and a restore you have actually proved.
Thanks for the thoughtful feedback! You raised several important points, especially around reliability during network interruptions and recovery validation.
I completely agree that streaming backups need to prove they're reliable under real-world conditions, not just ideal scenarios. The restore validation criteria you mentioned, including GTID/binlog consistency and GCM nonce handling, are definitely areas I want to improve and test further.
I've added these to the project's roadmap. Thanks again for taking the time to share such detailed insights!