Comparing Compression Methods on Linux: gzip, bzip2, xz, and zstd

Introduction

A recent test was conducted to compare the performance of four popular Linux compression methods: gzip, bzip2, xz, and zstd. The test involved compressing a 4 GB SQL dump file using the standard compression strength, and then with their strongest compression levels. This article presents the results of these tests, including both compression times and compression ratios, and provides a brief description of each compression method.

Compression Method Descriptions

1. gzip: GNU zip is a widely-used compression tool based on the DEFLATE algorithm. It is designed to be fast and suitable for a broad range of use cases.

2. bzip2: bzip2 is a block-sorting file compressor that utilizes block-sorting and Huffman coding techniques to achieve higher compression ratios than gzip, albeit at the cost of slower compression times.

3. xz: xz is a compression tool that uses the LZMA (Lempel-Ziv-Markov chain-Algorithm) for high compression ratios. It is designed to offer better compression ratios than gzip and bzip2, while still maintaining reasonable compression and decompression speeds.

4. zstd: Zstandard, developed by Facebook, is a modern compression algorithm offering high compression levels and fast compression and decompression speeds. It is becoming increasingly popular due to its balance of speed and compression effectiveness.

Results

Compression Ratios

The graph shows the resulting compressed file size in MB. Smaller values are better. The original file size was 4 GB.

Compression Times

The graph shows the compression time in seconds. Smaller values are better.

Compression Commands Used in the Test

In this article, we tested four popular compression methods on an SQL dump file: gzip, bzip2, xz, and zstd. We used the following specific commands for each compression method at their standard compression level and their strongest compression level:

1. gzip:

• Standard Compression: gzip -k -c database.sql > database.sql.gz
• Strongest Compression: gzip -9 -k -c database.sql > database.v2.sql.gz

1. bzip2:

• Standard Compression: bzip2 -k -c database.sql > database.sql.bz2
• Strongest Compression: bzip2 -9 -k -c database.sql > database.v2.sql.bz2

1. xz:

• Standard Compression: xz -k -c database.sql > database.sql.xz
• Strongest Compression: xz -9e -c database.sql > database.v2.sql.xz

1. zstd:

• Standard Compression: zstd -k -c database.sql > database.sql.zst
• Strongest Compression: zstd -19 -k -c database.sql > database.v2.sql.zst

Conclusion

Based on the test results for compressing SQL dump files, zstd offers the best balance between compression time and compression ratio among the four methods. Although xz provides the smallest output file size, it takes a considerably longer time to compress, particularly at its strongest setting. In contrast, zstd performs significantly faster, while still achieving a competitive compression ratio. Therefore, zstd is recommended for users seeking a balance of speed and compression effectiveness when compressing SQL dump files.