Zstd vs Gzip on a Small VPS: The CPU Win, Not the Size Win

I was poking at the response headers on this blog last Tuesday, for no good reason, and noticed every page was going out as Content-Encoding: gzip. Which is fine. Gzip is thirty-odd years old and it works. But I'd just read the Cloudflare post about squeezing petabytes out of their cache with Zstandard, and it nagged at me that a company with that much traffic had decided zstd was worth the engineering time while I was still shipping the default from 1992.

So I spent an evening on it. Turned zstd on, measured, turned it off, measured again, and then turned it back on. This is the post about what I found, with the numbers, and where gzip still wins. If you run a VPS with nginx or Caddy in front of a web app, it's about a twenty-minute change, and you should probably make it. Just not for the reason you might think.

What zstd is, in one paragraph

Zstandard is a lossless compression algorithm Yann Collet wrote at Facebook and open sourced in 2016. The reference implementation is BSD licensed and it's been an IETF standard since RFC 8878. The pitch is speed at a given compression ratio: it aims to compress about as small as zlib at the same level while running several times faster, and it has 22 levels so you can trade one for the other. It also has a dictionary mode that helps a lot on small, similar payloads, which I'm going to ignore for this post because HTTP responses don't get to use it yet.

For the web specifically, the interesting bit is that Content-Encoding: zstd is now a thing browsers accept. Chrome shipped it in 2024, Firefox followed, and caniuse has the current table. Safari has been the holdout, so you can't drop gzip. You add zstd next to it and let content negotiation sort it out.

What Cloudflare measured, and what they didn't

The Cloudflare post is about their cache, not about serving to browsers, and that distinction matters. They were compressing assets at rest on disk and between their data centres, then decoding before sending to the client. Different problem. But the numbers are still the best public data I've seen on zstd against real web traffic.

Their eligible text assets (HTML, JSON, CSS, JavaScript, at least 4 KiB, no existing encoding) compressed about 2.8 times at zstd level 3. Encode cost was around 4.3 nanoseconds per byte, decode around 1.6. In their earlier browser compression testing, which they link from the post, zstd came out 42% faster than Brotli at nearly the same output size, and 11.3% smaller than gzip at a comparable speed.

Two things they were careful to say. First, the 2.8x came from a deliberately compressible test corpus and they don't claim it as a fleet-wide constant. Second, they tried restricting compression to hot content only and it didn't help, because decode happens on every serve regardless. Compressing everything eligible above 4 KiB was the simpler policy and it won. I like that finding because it's the opposite of what I'd have guessed.

What they didn't measure, because it wasn't their problem, is what happens at the origin when a small server compresses on the fly for every request. That's my problem, and probably yours.

My before: nginx and gzip level 6

This blog runs on a small Hetzner box, which I've written about before. The nginx config had the gzip block everyone copies:

# Before
gzip on;
gzip_comp_level 6;
gzip_min_length 1024;
gzip_types text/plain text/css application/json application/javascript
           text/xml application/xml image/svg+xml;
gzip_vary on;

I grabbed the rendered HTML of ten posts, the main CSS bundle, and the JSON output of the REST API endpoint that a client project hits, and ran the CLI tools on them so I'd have a baseline that didn't depend on nginx internals:

for f in samples/*; do
  printf "%-28s %8d  gzip6:%8d  zstd3:%8d  zstd9:%8d\n" \
    "$(basename "$f")" "$(stat -c%s "$f")" \
    "$(gzip -6 -c "$f" | wc -c)" \
    "$(zstd -3 -c "$f" | wc -c)" \
    "$(zstd -9 -c "$f" | wc -c)"
done

On the HTML, zstd level 3 landed within a percent or two of gzip 6, sometimes slightly larger. Level 9 was consistently about 8 to 12% smaller than gzip. On the CSS bundle, similar. On the JSON, zstd did noticeably better at every level, which tracks with Cloudflare's observation that repetitive structured text is where it shines.

So on size alone, at the levels you'd actually run on a small server, zstd is a modest win. Not nothing. Not the 3x the headline numbers suggest, either, because the 3x is against uncompressed, and you weren't serving uncompressed.

Where zstd actually wins: CPU

Then I timed it. Same loop, but with time around a hundred iterations of each. I'm not going to publish my exact milliseconds because they're from a shared vCPU and they moved by 15% between runs, but the shape was stable across five attempts.

Zstd level 3 compressed the sample set in roughly a third of the time gzip 6 took. Zstd level 9, the one that beat gzip on size, was still faster than gzip 6. Decompression, which the browser does and I don't pay for, was faster too, but that's the client's problem.

That's the actual argument for zstd on an origin server: the same or smaller files for a fraction of the CPU. On a box where nginx, PHP-FPM, and Postgres are all sharing two cores, CPU spent compressing is CPU not spent doing the thing the request was for. I'd been running gzip at level 6 for years, and I could have had the same bytes on the wire for less than half the compute.

My after: Caddy with zstd first

Nginx doesn't ship a zstd module. There's a well maintained third party one, tokers/zstd-nginx-module, and if you're on nginx it works, but it means building nginx yourself or trusting someone's package, and I already had reasons to be looking at Caddy after native ACME changed my default. Caddy has zstd built in. The whole config change is one line:

# After
abrarqasim.com {
    encode zstd gzip
    reverse_proxy 127.0.0.1:9000
}

Order matters. Caddy tries encodings in the order listed and picks the first one the client's Accept-Encoding allows, so zstd gzip means Chrome and Firefox get zstd and Safari gets gzip. The encode directive docs cover the knobs, and there are two I changed from default:

encode {
    zstd
    gzip 6
    minimum_length 1024
    match {
        header Content-Type text/*
        header Content-Type application/json*
        header Content-Type application/javascript*
        header Content-Type image/svg+xml*
    }
}

The minimum_length is the same idea as Cloudflare's 4 KiB floor, just lower because my pages are smaller than theirs. Below about a kilobyte the compression framing overhead eats the savings. The match block stops Caddy from trying to compress images and fonts that are already compressed, which was 63% of Cloudflare's bytes and a not-dissimilar share of mine.

Caddy's zstd defaults to a fast level and doesn't expose the numeric level in the Caddyfile the way gzip does, so I couldn't directly reproduce my "level 9" CLI numbers in production. In practice the on-the-wire sizes I see are about even with the old gzip 6 output and the CPU graph on the box dropped visibly during a traffic spike from a post that got shared. That's the trade Cloudflare described, at a scale of one small server instead of petabytes.

Where I'd still leave gzip alone

Three cases.

If your traffic is mostly Safari or mostly older clients, zstd buys you nothing, because they'll negotiate down to gzip anyway. Check your analytics before you bother.

If you're behind a CDN that compresses at the edge, your origin's choice barely matters. Cloudflare in particular will do its own thing between the edge and the browser. Set it up on the origin anyway for the CDN-to-origin hop, but don't expect a visible change for users.

If you pre-compress static assets at build time, which you should for anything served from disk, the CPU argument mostly disappears because you pay once. There, Brotli at level 11 will usually beat zstd on pure size, and size is all that matters when compression is free. Zstd's advantage is at the dynamic edge, where you're compressing on every request and can't afford to be slow about it.

Something to do this week

Run the bash loop above against a handful of your own responses. It takes two minutes and you'll know whether zstd is a size win, a CPU win, or neither for your specific content. Then, if you're on Caddy, add encode zstd gzip and watch your CPU graph for a week. If you're on nginx, the decision is whether a third party module is worth the build step; for me it wasn't, but I'd already decided to move.

I've been doing this kind of measure-first infrastructure work on client servers for a while now, and most of what I've learned about small-box performance is on my site if you want more of it.

---

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.