DEV Community: Rozoom News

Plex Is Being Exploited. The Fix Shipped Eleven Weeks Earlier.

Rozoom News Editor — Tue, 27 Oct 2026 07:20:28 +0000

A media server is the most forgivable thing to expose to the internet. It is also, this month, the most expensive.

On 20 October 2026 CISA added four Plex Media Server vulnerabilities to its Known Exploited Vulnerabilities catalogue at once. Chained, they let an attacker with no credentials obtain an admin token and execute code as the Plex process. The researcher who found them scores the chain 9.8.

The patched build, 1.43.3.10861, was released on 4 August. Everything that has happened since is about the gap between a fix existing and a fix being installed, and that gap is the only part of this story you control.

The timeline, because it is the argument

Date	Event
27 July 2026	Researcher Zach Main reports the chain to Plex
3-4 August 2026	Plex Media Server 1.43.3.10861 ships with the fix
Early September 2026	Shadowserver counts over 36,000 internet-exposed servers still unpatched
16 October 2026	Full technical write-up published
17 October 2026	First in-the-wild exploitation observed, against honeypots, dropping a cryptominer
19 October 2026	Public proof-of-concept exploit on GitHub
20 October 2026	CISA adds all four CVEs to the KEV catalogue
23 October 2026	Federal remediation deadline for the two worst of them

Eleven weeks of warning. Thirty-six thousand servers that did not use them.

What the chain actually is

Four issues, each unremarkable alone, which is exactly why they sat unnoticed:

CVE-2026-96652 (CVSS 5.3), server-side request forgery: Plex can be made to send POST requests to arbitrary destinations, including its own internal APIs.
CVE-2026-96654 (CVSS 6.9), URL injection: attacker-controlled parameters reach other plugins' functions.
CVE-2026-96655 (CVSS 5.3), another SSRF, this one in the transcoder endpoint.
CVE-2026-96656 (CVSS 8.6), arbitrary file write in the transcoder settings: a shared library can be planted where Plex will load it.

Read the last one again, because it is the pattern worth carrying to every other service you run: a process that writes files, and later loads code from the directory it writes to, is one file-write bug away from code execution. Codecs, plugins, themes, extensions, uploads with an include path. The individual scores are medium. The chain is 9.8.

Exploitation needs network access to TCP port 32400. Plex's Remote Access feature, or a manual port forward, is what turns a living-room service into an internet-facing one.

Do these three things first

1. Check the full build number, not the marketing version.

# Debian or Ubuntu package install
dpkg -s plexmediaserver | grep '^Version'
# official Docker image
docker exec plex dpkg -s plexmediaserver | grep '^Version'
# any install, from the box itself: the identity endpoint answers without a token
curl -s http://127.0.0.1:32400/identity

Anything below 1.43.3.10861 is affected. An early 1.43.3 build is still affected: the last five digits are the ones that matter.

2. Update, then confirm the running process is the new one. A container that was never re-pulled, or a package held back by a pin, will happily report a new version in the UI while the old binary is still serving.

docker compose pull plex && docker compose up -d plex
docker exec plex dpkg -s plexmediaserver | grep '^Version'

3. If the server was internet-reachable on an old build, assume it was reached. Plex's own advisory says to treat such servers as potentially compromised, and gives indicators: unfamiliar .so or .dll files in the Codecs directory, and outbound connections to proxystream.st. Check both before you call it done:

# unexpected shared libraries in the codec directory
find ~/Library/Application\ Support/Plex\ Media\ Server/Codecs \
     /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Codecs \
     -name '*.so' -o -name '*.dll' 2>/dev/null | sort

# anything the box is talking to that you cannot name
ss -tn state established | awk 'NR>1{print $4}' | sort | uniq -c | sort -rn | head -20

A cryptominer announces itself in CPU. Proxyware does not: it sells your address as residential bandwidth and stays quiet. Absence of load is not absence of compromise.

The checklist that would have saved you anyway

None of the following requires knowing about a CVE in advance. That is the entire point: these controls pay off during the weeks when you have not patched yet, which is every service you run, always.

Do not port-forward a service to the internet to use it yourself. Remote Access and a forwarded port exist to serve people who are not on your network. If the only remote user is you, a WireGuard tunnel or a mesh VPN removes the attack surface completely rather than defending it. Exposure is a decision, and most of the time nobody ever makes it deliberately.

Default-deny inbound, with the open ports written down. On a box that runs several things, the firewall is the only place where "what is reachable" is answerable in one screen:

ufw default deny incoming
ufw allow 443/tcp
ufw status numbered

Read the output top to bottom and ask what matches first. On a legacy box we audited this month, a careful three-subnet allowlist sat four lines below a blanket ACCEPT for the same port from 0.0.0.0/0, which meant it had never restricted anything. The full audit is here.

Put HTTP services behind one reverse proxy, never directly on the internet. One TLS configuration, one access log, one place to add authentication. The pattern we run on this very site, in full, is in Caddy and Docker in production. The short version for a service that needs no public access at all:

media.example.com {
    @notlocal not remote_ip 10.0.0.0/8 100.64.0.0/10 192.168.0.0/16
    respond @notlocal 403
    reverse_proxy 127.0.0.1:32400
}

Bind the service to localhost and let the proxy be the only door. A container published as -p 32400:32400 is on every interface the host owns, including the public one; -p 127.0.0.1:32400:32400 is not. This single character difference is the most common accidental exposure we find.

Treat outbound traffic as something you are allowed to care about. Both observed payloads here, the miner and the proxyware, are useless without egress. A home or small-office box has no business opening arbitrary outbound connections, and a baseline you can diff is worth more than any IOC list published after the fact.

Know how you would rebuild, and have tested it. After this class of compromise the honest advice is a clean install, which is only survivable if your data is somewhere else and restoring is something you have done before, not something you plan to learn under pressure. Where that somewhere else should be, with real prices on real data, is in our backup storage comparison.

Subscribe to the release notes of anything you expose. Eleven weeks is not an unlucky interval, it is an unread mailing list. For a self-hosted service the vendor has no way to reach you, and no obligation to try.

What we do not know

We have not seen the inside of a compromised host ourselves, and we are not going to pretend otherwise. The indicators above are Plex's, the exposure counts are Shadowserver's, the CVSS numbers are the researcher's, and the exploitation dates come from the vendors and CISA. Our contribution is the part we can stand behind: what the pattern means for the rest of your stack, and the configuration we actually run.

FAQ

Which Plex version fixes Plex2Shell?
1.43.3.10861 and later, released on 4 August 2026. Builds before that are affected, including earlier 1.43.3 builds.

Is my Plex server at risk if it is not exposed to the internet?
The chain needs network access to TCP port 32400. A server reachable only from your LAN is still exploitable from that LAN, so a compromised laptop or an untrusted guest device is enough. It is a smaller risk, not a theoretical one.

How do I know if my Plex server was already compromised?
Plex's advisory points at two indicators: unfamiliar .so or .dll files in the Codecs directory, and outbound connections to proxystream.st. Observed payloads have been cryptominers and proxy malware. If the server was internet-facing on an old build, the safe assumption is compromise.

Does updating remove an existing compromise?
No. Patching closes the door; it does not remove anything already inside. After a confirmed compromise the only clean path is a rebuild from known-good media plus a restore of data you can verify.

Should I just turn off Plex Remote Access?
If the only person streaming from outside your home is you, yes, and reach it over a VPN instead. Remote Access is the feature that puts port 32400 on the internet, and the 36,000 exposed servers counted in September were mostly not deliberate decisions.

What's next

If this prompted a look at the rest of your stack, the one-hour audit for a server you inherited is the checklist we use on an unknown box, and the VPS comparison covers where to put a service you would rather not run at home at all.

Originally published at news.rozoom.app.

Best VPS for Self-Hosting in 2026: Hetzner vs DigitalOcean vs Vultr

Rozoom News Editor — Tue, 27 Oct 2026 06:20:01 +0000

We queried the Hetzner API twice in one day while writing this. At midday, the cheap plan every comparison recommends was unavailable in all three European locations. Five hours later, the same plan was orderable again in two of them. Nobody who copies a price out of another comparison can tell you that, because it is not in the price.

So this one is built differently. The performance numbers come from a box we pay for and benchmarked ourselves. The prices come from the vendors' own APIs, read on the day of writing, with the plan ID next to every figure so you can check them yourself.

TL;DR

Hetzner wins on price for self-hosting: CX23 (x86, 2 vCPU / 4 GB / 40 GB) at €5.49/mo, about 4x cheaper than the equivalent Droplet, with 5x the included traffic.
Check availability before you plan around it. CX23 is not orderable everywhere, and whether it is orderable at all changes during the day. The always-available fallback is CAX11 (Arm, €5.99/mo); the x86 fallback is CPX22 at €19.49/mo, which is only about 20% under DigitalOcean.
DigitalOcean ($24/mo) buys you the best API and docs, per-second billing since January 2026, and 4 TB of transfer.
Vultr ($20/mo) buys you reach: 31 locations, plus a genuinely free 512 MB tier most comparisons claim doesn't exist.

Live prices, read today

The 2 vCPU / 4 GB tier is where most self-hosters land: a few containers, a database, a reverse proxy.

	Plan	vCPU / RAM / disk	Price	Included traffic	Over-usage
Hetzner (x86)	CX23	2 / 4 GB / 40 GB	€5.49/mo	20 TB	€1.00/TB
Hetzner (Arm)	CAX11	2 / 4 GB / 40 GB	€5.99/mo	20 TB	€1.00/TB
Hetzner (x86)	CPX22	2 / 4 GB / 80 GB	€19.49/mo	20 TB	€1.00/TB
DigitalOcean	Basic	2 / 4 GB / 80 GB	$24.00/mo	4 TB	~$0.01/GB
Vultr	vc2-2c-4gb	2 / 4 GB / 80 GB	$20.00/mo	3 TB	~$0.01/GB

Hetzner bills a public IPv4 separately at €0.50/mo. IPv6 is free. Add it to the Hetzner rows if you need a v4 address, which you probably do.

Prices change. Every number above was read from the vendors' own pricing APIs on the date of this post, not copied from another comparison.

The availability trap nobody mentions

Search for a VPS recommendation and you will be told to buy the cheapest Hetzner CX box. The price is right. Whether you can buy it is a different question, and the answer changes.

Here are our two reads of the Hetzner API, the same endpoint, the same day:

Midday. CX23 (2 vCPU / 4 GB / 40 GB, €5.49) listed a price for Falkenstein, Nuremberg and Helsinki, and an empty availability list. Price visible, server not orderable anywhere in Europe.
Five hours later. The same plan came back as orderable in Nuremberg and Helsinki. Falkenstein still not.

Neither read was wrong. Capacity moves. What matters is that the number everyone quotes is not the number that tells you whether you can deploy:

The prices[] array still lists locations you cannot order in, at their old prices. It is not an availability signal.
The signal is server_types[].locations[].available, per location, and it is live.
If you order anyway, the API fails with invalid_input: unsupported location for server type, which reads like you passed a bad region rather than like the plan is gone.
GET /v1/datacenters was removed in June 2026 and returns HTTP 410, so there is no second place to look.

Practical advice: check availability for your target location immediately before you order, try a second European location before giving up, and keep a fallback plan in mind. Today that fallback is CAX11, which has been orderable in all three locations every time we looked.

What actually matters when you self-host

1. Egress is the silent budget killer

Nextcloud, photo sync, backups: traffic adds up faster than compute. This is where the three split hardest.

Hetzner includes 20 TB on every Cloud plan, then bills €1.00/TB, in 100 MB blocks. Not free beyond the cap - but ten times cheaper than the other two, and the cap is high enough that most self-hosters never reach it.
DigitalOcean includes 0.5-6 TB depending on plan (4 TB at the tier above), then about $0.01/GB - roughly $10/TB.
Vultr includes 0.5-5 TB (3 TB at this tier), then about $0.01/GB.

Hetzner notifies you at 75% and 100% of the included traffic. Those are notifications, not a cap: usage keeps billing either way.

2. vCPU is marketing, not performance

All three sell "vCPU", but a shared Hetzner core and a dedicated DigitalOcean core are not the same unit. The only way to know is to measure - which we did, for the one box we own.

3. Snapshots and backups are not free anywhere

Hetzner: snapshots at €0.0143/GB per month on the compressed size; backups are a flat 20% of the server price and give you 7 slots.
DigitalOcean: snapshots $0.06/GB/month; backups 20% (weekly) or 30% (daily) of Droplet cost.
Vultr: snapshots billed per GB of stored data.

Nobody in this bracket gives you free snapshot storage. Budget it.

The benchmark: a real Hetzner box, measured

We took a Hetzner Cloud server we run in production - 2 vCPU shared x86 (Intel Xeon Skylake), 4 GB RAM, 40 GB SSD, Nuremberg - and ran standard, reproducible tests. These numbers are ours.

CPU (OpenSSL speed, single thread):

Test	Result
AES-256-CBC (16 KB blocks)	~630 MB/s
SHA-256 (16 KB blocks)	~255 MB/s

Disk (40 GB SSD):

Test	Result
Sequential write (fdatasync)	468 MB/s
Sequential read, cold	1.1 GB/s
Sequential read, warm (cached)	6.0 GB/s

Network (single stream):

Test	Result
Download from OVH mirror	175 MB/s
Download from Ubuntu mirror	82 MB/s

The box sits on a 1 Gbit/s port with IPv6. For a shared-vCPU plan at this price the disk is the quiet star: 468 MB/s sustained write is plenty for a database and a media stack on the same host.

Three honest caveats. This is a CX-line x86 box, the same 2 vCPU / 4 GB / 40 GB spec as the CX23 we recommend, but not the Arm CAX11. We do not rent from DigitalOcean or Vultr, so their figures here are vendor-published, not measured. And a single-thread OpenSSL run on a shared core is a floor, not a guarantee: noisy neighbours are the trade-off of the shared tier.

How the prices actually compare

Against DigitalOcean's $24 for 2 vCPU / 4 GB:

CX23 (x86) at €5.49 - roughly 4x cheaper, with 5x the included traffic. Half the disk (40 GB vs 80 GB), and subject to the availability caveat above.
CAX11 (Arm) at €5.99 - the same gap, always orderable so far, if your stack runs on arm64. Most upstream container images ship arm64 builds now; check the ones you depend on.
CPX22 (x86) at €19.49 - about 20% cheaper, with 5x the traffic and the same 80 GB disk. This is your number if CX23 is unavailable and you need x86.

That last line is the one missing from every comparison we read. The Hetzner price advantage is 4x when the cheap line is available and roughly a fifth off when it is not, and which of those you get depends on the hour you sign up. Add €0.50/mo for IPv4 to any of them.

When to pick which

Pick Hetzner if:

You're in or near Europe, or your users are.
You run Nextcloud, a media stack, or backups - anything traffic-heavy.
You want the most compute per euro, and you can check plan availability before you commit.

Pick DigitalOcean if:

You want the smoothest API and the best docs.
You need per-second billing for short-lived workloads.
You want one-click apps instead of configuring a stack by hand.

Pick Vultr if:

You need a region the others don't cover - 31 locations against Hetzner's five.
You want to test on the free tier before paying.
You run bursty workloads and spin instances up and down often.

Our recommendation

For European self-hosting, Hetzner Cloud is the pick, and the benchmark above shows the cheap shared tier is not slow. Order CX23 at €5.49/mo if it is available in your location; take CAX11 at €5.99/mo if it is not and your stack runs on Arm; fall back to CPX22 at €19.49/mo if you need x86 either way. Check availability first, in that order, and the decision takes a minute.

If your users are in the US or Asia, DigitalOcean and Vultr are both solid - just model the egress before you sign up.

FAQ

Is Hetzner good for self-hosting?
Yes, and it is still the price leader in Europe. Two caveats: the identity verification step at signup, and plan availability that changes without notice - check your target location before ordering.

Why can't I order the cheap Hetzner plan everyone recommends?
Because CX-line availability varies by location and over time, while the price stays visible on comparison sites and in the API's own price list. Availability is a separate field and it flips: we saw CX23 unavailable across all of Europe and orderable in two locations five hours later. Try another European location, then fall back to CAX11.

Does DigitalOcean charge for egress?
Yes. Beyond the included transfer (4 TB at the 4 GB tier) it's about $0.01/GB, roughly $10 per TB. Model this before you sign up if you serve media or move backups.

Does Vultr have a free tier?
Yes - a 1 vCPU / 512 MB / 10 GB instance in a small number of locations, with no included bandwidth. There is also a $2.50/mo plan, but it is IPv6-only and limited to two locations. The cheapest generally useful plan is $5/mo.

Which VPS has the best price-to-performance?
At 2 vCPU / 4 GB, Hetzner leads on both price and included traffic by a wide margin when its cheap line (CX23, CAX11) is available. If you are pushed onto CPX22, the gap narrows to roughly 20%, which is close enough that DigitalOcean's tooling or Vultr's regions can be the better buy.

Can I self-host on a $6 VPS?
Yes, for a reverse proxy, a couple of containers and a small database. The moment you add media or heavy backups, the thing that breaks your budget is egress, not CPU.

What's next

This is the first in a series. Coming up: running Nextcloud AIO on Hetzner and Caddy + Docker in production - both built on the same box we benchmarked here.

Originally published at news.rozoom.app.

Deploy From GitHub Actions Without Giving CI a Shell on Your Server

Rozoom News Editor — Tue, 27 Oct 2026 06:19:25 +0000

Every "deploy a static site with GitHub Actions" tutorial ends the same way: generate an SSH key, paste the private half into repository secrets, rsync the build to your server. It works on the first try, which is the problem. What you just created is an interactive shell account on your server, held by a system that runs code from pull requests.

This post is the version with the guard rails: a key that can do exactly one thing, a host key you pinned yourself, and three checks that stop a broken build from replacing a working site. It is the pipeline that ships this blog, and it is maybe fifteen extra minutes of work.

The server side assumes you have a box. If you don't, we compared the cheap tiers in Best VPS for self-hosting in 2026.

What the naive deploy key actually grants

An entry in ~/.ssh/authorized_keys with no options is a login. Anything holding that private key can:

open an interactive shell as that user,
read every file that user can read, including other projects on the same box and any .env lying around,
sudo if the user has it,
forward ports and use your server as a jump host into whatever it can reach.

None of that is needed to copy a directory of HTML. And the private key lives in a CI system where a workflow change, a compromised third-party action, or a misconfigured pull_request_target trigger can read it.

We ran exactly this setup for months without thinking about it, which is the honest reason this post exists.

Lock the key to one command

OpenSSH lets you attach options to each key in authorized_keys. The one that matters is command=, which replaces whatever the client asked to run with what you specify:

command="/usr/bin/rrsync -wo /home/deploy/site",restrict ssh-ed25519 AAAA... github-actions

Piece by piece:

rrsync ships with rsync (/usr/bin/rrsync on Debian and Ubuntu; it may be a gzipped script under /usr/share/doc/rsync/scripts/ on older releases). It is a wrapper that accepts only rsync's server-side invocation and refuses everything else.
-wo means write-only: the key can upload into the directory and cannot download from it. If your deploy only pushes, take the stricter option.
The path argument is a jail. Paths outside it are rejected, including anything clever with ...
restrict disables port forwarding, agent forwarding, X11 and PTY allocation in one word. On any OpenSSH from the last several years, prefer it over listing no-port-forwarding,no-agent-forwarding,... by hand.

Test it the direct way, from your laptop with that private key:

ssh -i deploy_key deploy@your-server 'id'      # must fail
rsync -rltz -e "ssh -i deploy_key" ./public/ deploy@your-server:/   # must succeed

The first command failing is the whole point. If it prints a user id, the options are not being applied - check that the whole thing is on one line and that the line starts with command=, not with the key type.

Note the destination path in the rsync call: it is relative to the jail, so / means the directory you locked the key into.

Pin the host key, don't disable the check

The other half of the tutorials' shortcut is StrictHostKeyChecking=no, which turns off the check that the server you are shipping to is the server you think it is. Pin it instead. On the server:

ssh-keyscan -t ed25519 your-server.example

Put that line into a repository secret and write it to known_hosts in the job. Now a changed host key fails the deploy loudly instead of silently shipping your site to whoever answers on that address.

The workflow

name: Build and deploy

on:
  push:
    branches: [main]
  workflow_dispatch:

concurrency:
  group: deploy
  cancel-in-progress: false

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with:
          submodules: recursive

      - name: Build
        run: your-build-command

      - name: Guard against an empty build
        run: |
          set -euo pipefail
          test -s public/index.html
          test -s public/sitemap.xml

      - name: Deploy over rsync
        env:
          DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
          DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
          DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
        run: |
          set -euo pipefail
          mkdir -p ~/.ssh
          printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy_key
          chmod 600 ~/.ssh/deploy_key
          printf '%s\n' "$DEPLOY_KNOWN_HOSTS" > ~/.ssh/known_hosts
          rsync -rltz --delete --omit-dir-times \
            -e "ssh -i ~/.ssh/deploy_key -o IdentitiesOnly=yes -o UserKnownHostsFile=~/.ssh/known_hosts" \
            public/ "$DEPLOY_USER@$DEPLOY_HOST:/"

      - name: Smoke test
        run: |
          set -euo pipefail
          for url in https://example.com/ https://example.com/sitemap.xml; do
            code=$(curl -s -o /dev/null -w '%{http_code}' --retry 3 --retry-delay 5 "$url")
            echo "$url -> $code"
            test "$code" = "200"
          done

The parts that are not decoration:

concurrency without cancel-in-progress. Two deploys running at once against the same directory with --delete is how you serve a half-copied site. Queueing them costs nothing.

The empty-build guard. --delete is what makes rsync a deploy rather than an upload, and it is also what turns a build that silently produced nothing into a deleted website. Three test -s lines are cheaper than that outage. Assert on files that must exist and on at least one that only appears when the build really ran.

IdentitiesOnly=yes. Without it, ssh may offer other keys from the agent first and get refused before it reaches yours.

--omit-dir-times. With a write-only jail, rsync's attempts to set directory timestamps can fail noisily. This keeps the output clean without weakening anything.

The smoke test runs against the public URL, not against the server over ssh. It is the only step that proves a visitor can see the result: it catches a dead container, an expired certificate and a proxy pointing at the wrong directory, none of which rsync knows about.

What this does and does not protect you from

It does protect you from the realistic failure: a leaked CI secret turning into shell access on your server. The key can write one directory and nothing else, cannot read back, cannot forward, cannot open a terminal.

It does not protect you from someone who can change the workflow file, because they can replace the content you ship. If the site is the thing you care about, protect the branch: require review, and keep the deploy on push to the default branch only - never on pull_request_target, which runs with your secrets available to a fork's code.

It also does not protect against the directory itself. Whatever is served from that path can be overwritten by the key. Keep the deploy target separate from config, data volumes and anything else the server holds.

The retrofit, in order

If you already have an unrestricted deploy key, fixing it takes a few minutes:

Confirm rrsync exists on the server: ls -l /usr/bin/rrsync.
Edit authorized_keys and add command="/usr/bin/rrsync -wo /path/to/site",restrict in front of the existing key line.
From your laptop with that key, check that ssh ... 'id' fails and a test rsync still succeeds.
Re-run the workflow with workflow_dispatch and watch the smoke test pass.
Only then delete the backup of the old line.

Rotating the key at the same time is a reasonable habit, especially if it was ever printed in a log.

What's next

The server side of the same setup: Caddy and Docker in production - automatic HTTPS, several sites on one box, and the cache headers that decide whether your readers see the page you just published.

Originally published at news.rozoom.app.

The One-Hour Audit for a Server You Inherited

Rozoom News Editor — Tue, 27 Oct 2026 06:18:49 +0000

Somebody hands you the root password to a machine that has been running for years. It serves real sites, nobody remembers how it was set up, and the person who did set it up is gone.

The temptation is to do the big thing first: rebuild it, migrate it, containerise it. Do the small thing first. One hour, five questions, and you will know whether the machine is an inconvenience or an emergency.

This is the audit we ran on exactly such a box last week, with what it actually turned up. Every finding below is real, and not one of them was visible from the outside.

1. What is listening, and what does the firewall think?

These two answers are always compared, never read separately. The first is the truth about the machine; the second is what somebody once intended.

ss -tlnp                     # every listening TCP socket, with the process
iptables -L INPUT -n --line-numbers

Our finding, and it is the one to steal from this article: the firewall had a careful allowlist for the admin panel port, three trusted subnets, written by somebody who knew what they were doing. Four lines above it sat a blanket ACCEPT for the same port from 0.0.0.0/0, added later.

iptables evaluates in order and stops at the first match. The allowlist had never, from the moment that line was added, allowed anything less than everybody. The admin login had been open to the internet for years behind a rule that read, to a casual glance, like a restriction.

Read your rules top to bottom, with line numbers, and ask what matches first. A rule list is a program, not a set.

2. Who is actually using the thing you are about to close?

The audit's single most useful habit. Before closing any path, read its access log and count real users.

# top talkers against one path, last 10k lines
grep -h 'phpmyadmin' /var/log/nginx/*access* | awk '{print $1}' \
  | sort | uniq -c | sort -rn | head

Two of our findings came straight out of this:

A phpMyAdmin from 2017, reachable from the internet, on a web server whose PHP had no upstream patches left. The log showed 201 requests from one address and 38 from another, both scanners walking a wordlist. Not one human session in the window.
proftpd running, in plaintext, no TLS. xferlog was empty since June. The only connections were anonymous login attempts from a botnet.

Both were closed in minutes, with no risk of breaking anyone's workflow, because the logs proved there was no workflow. Had the logs shown daily sessions from one office IP, the answer would have been an allowlist and a conversation rather than a shutdown.

The rule: never close a service on the belief that nobody uses it. Close it on evidence that nobody uses it. Those feel the same right up until the invoicing department calls.

3. Do not lock yourself out

Everything in this section is firewall work on a machine you reach only through that firewall. Two precautions, both cheap.

Arm an auto-revert before you commit anything. A timer that restores the known-good ruleset in ten minutes, cancelled by hand once you have confirmed you can still get in:

iptables-save > /root/iptables.good
echo 'iptables-restore < /root/iptables.good' | at now + 10 minutes

Apply your change, open a brand new SSH session from somewhere else, and only then cancel the job with atrm. Testing on the session you already have proves nothing: an established connection survives rules that block new ones.

Then make it survive a reboot, which is a separate step and the one people forget:

service iptables save          # or: netfilter-persistent save
systemctl is-enabled iptables

Ours rebooted on a weekly schedule we did not know about. The hardening was still there afterwards, because the rules had been saved, and we knew it was still there because we checked after the reboot rather than assuming.

4. What renews itself, and what only appears to?

The quiet killer. On this box, TLS renewal was a job inside a control panel whose licence had expired two months earlier. The renewal cron died with the licence. Four sites had already been failing their TLS handshake for five days before the audit found it, and seven more were scheduled to expire within three weeks.

# every scheduled job on the box, in one pass
crontab -l; ls -l /etc/cron.*/ /etc/cron.d/; systemctl list-timers --all

Read all of it, including the commented lines, because they tell you what used to run. And look for duplicates: we disabled one renewal cron and only on a second pass found a second one, same job, different hour, four lines further down.

Taking renewal back is a short job even on an end-of-life distro - we wrote it up in acme.sh on a server where certbot will not install.

5. What is installed that nothing uses?

Legacy boxes accumulate runtimes. Ours had eleven separate PHP builds, going back to 5.2. The interesting question is not what is installed but what is wired up:

grep -rhoE 'php[0-9]+' /etc/nginx/conf.d/*.conf /etc/httpd/conf.d/*.conf \
  | sort | uniq -c | sort -rn

Eight sites were on 7.4, three on 8.0, and every build older than that was dead weight. That single command changed the migration plan: a machine that looks like it needs a hosting platform supporting PHP 5.2 actually needs 7.4 and 8.x, which is to say anything modern.

An audit is not only about what to close. It is about discovering that the scary part of the job does not exist.

What an hour bought

Four findings on one machine: an allowlist that allowed everyone, a nine-year-old admin tool exposed to scanners, an FTP daemon serving nobody, and certificate renewal that had been dead for two months with four sites already down.

None of it required a rebuild. None of it required a maintenance window. Three of the four were fixed the same hour, and the fourth - the distro itself being past end of life - is the one honest finding you cannot patch, which is exactly why it belongs in the migration plan rather than in today's work.

If the audit sends you shopping for a replacement box, we compared the options with live prices in Best VPS for self-hosting in 2026, and wrote up four years on one provider in our Hetzner Cloud review.

Originally published at news.rozoom.app.

Hetzner Cloud Review 2026: Four Years, Three Servers, and Every Bill Explained

Rozoom News Editor — Tue, 27 Oct 2026 06:18:13 +0000

The oldest server in our account was created on 29 September 2022 and is still running. It has never been migrated, rebuilt or restored. Two more boxes joined it since, the newest three weeks ago. That is the entire basis of this review: three machines, four years, one invoice a month, and a list of things that went wrong that is almost entirely our own fault.

Most Hetzner reviews are a pricing table with adjectives. This one is the bill, itemised, plus the parts of the bill that are easy to miss, and the honest account of what we broke.

What we actually run

Box	Type	Created	Job
1	CX32, 4 vCPU / 8 GB	Sep 2022	Legacy multi-site host, eleven small sites, mail
2	CX22, 2 vCPU / 4 GB	Nov 2023	Three containerised storefronts plus three CI runners
3	CX23, 2 vCPU / 4 GB	Oct 2026	Static blog, analytics, a relay service

No load balancers, no managed anything, no Kubernetes. Three Linux boxes, Docker, and a reverse proxy. Four years of uptime that we did not have to think about is the headline finding, and it is worth saying clearly before the complaints start: in four years, not one of our outages was caused by Hetzner.

The bill, every line

These figures come from the Hetzner pricing API, read the day this was published, in EUR excluding VAT.

Line item	Price
CX23, 2 vCPU / 4 GB / 40 GB	€5.49/mo
CAX11, Arm, 2 vCPU / 4 GB / 40 GB	€5.99/mo
CPX22, 2 vCPU / 4 GB / 80 GB	€19.49/mo
Primary IPv4 address	€0.50/mo
IPv6	free
Included traffic, EU locations	20 TB/mo
Traffic over the allowance	€1.00/TB
Automated backups	+20% of the server price, 7 slots
Snapshots and custom images	€0.0143/GB/mo
Block storage volumes	€0.0572/GB/mo
Floating IP	€3.00/mo

Two of those lines deserve a paragraph each, because they are the two that quietly take your money.

The floating IP line

A floating IP costs €3.00 a month, six times the price of the primary IPv4 on the server, and it is billed whether or not anything is attached to it. We found two of them on our account left over from a migration that had finished months earlier, routing nothing. Deleting them cut €6 a month, which on a fleet whose cheapest box is €5.49 is a whole extra server.

Go and look at your floating IPs now. If one is unattached, it is not a spare, it is a subscription.

The traffic line is genuinely generous

20 TB per month included, per server, in the European locations, and €1.00 per TB beyond it. For comparison, the equivalent allowance at the big US clouds is measured in hundreds of gigabytes and the overage is priced per gigabyte, typically around €0.09. At a terabyte of egress a month, the difference is between a rounding error and a bill that doubles.

This is the single strongest reason to run anything bandwidth-shaped - media, backups, a package mirror, an analytics endpoint - on Hetzner rather than on a hyperscaler. We looked at the alternatives in Best VPS for self-hosting in 2026.

What went wrong in four years

Every incident below was ours. We are listing them because they are the ones you will hit too, and none of them appear in a feature comparison.

1. The images ship with no swap

A CI runner on box 2 went offline and stayed offline for ten days. The symptom read like an unreachable machine. The cause was a build that allocated more than the 4 GB available, the kernel OOM-killer taking the runner process, and systemd not retrying. The box had been up for 85 days with zero swap, because the stock image does not configure any.

A 4 GB swapfile in /etc/fstab fixed it permanently, and the same build then completed:

fallocate -l 4G /swapfile && chmod 600 /swapfile
mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

If you run builds on a 4 GB box, do this on day one. Treat "service has been down for days" on a Hetzner box as a possible OOM before you treat it as a network problem.

2. Docker build cache ate two thirds of the disk

On a 38 GB disk, 24 GB was Docker build cache. Nothing warns you; the box simply starts failing writes. docker system prune -af --volumes on a schedule, or at minimum a disk alert, is not optional on a 40 GB plan.

3. fail2ban banned our own bridge

A host started answering 503 to everything. fail2ban had matched traffic coming from the Docker bridge, which is to say from our own reverse proxy, and banned the gateway address. Whitelist your bridge networks in ignoreip before you deploy fail2ban next to Docker, not after.

4. The plan you budgeted for may not be orderable

The CX line's availability flips through the day, per location. We read the API at midday and the €5.49 plan was unavailable across all three European locations; five hours later it was orderable in two of them. The prices[] array lists locations you cannot order in, which makes a price look like an offer when it is only a record. The live signal is server_types[].locations[].available, and GET /v1/datacenters has returned HTTP 410 since June 2026, so there is no second source to check.

What Hetzner still does not sell

As of this month's changelog, and this is the honest reason not to choose them:

No managed database. No equivalent of RDS or managed Postgres. You run the database, you handle the failover, you own the backups.
No managed Kubernetes. There is an official cloud controller manager and a CSI driver, which make a self-managed cluster pleasant, but the control plane is yours to operate.
Thin managed everything else. Object storage and storage boxes exist; load balancers, networks and firewalls exist and work. Beyond that the catalogue is deliberately short, and 2026 added features, not products.
Fewer regions. Three European locations plus a handful elsewhere. If your users are in Asia or South America, latency is a real argument against.
Support is a ticket, not a phone call. In four years we have opened almost none, which is its own kind of answer, but do not expect an account manager.

Backups: the checkbox we leave off, and why

Hetzner's automated backups cost 20% of the server price and keep 7 slots. On a €5.49 box that is €1.10 a month, which is cheap enough that the default advice should be to turn it on.

We leave it off, and the reasoning is worth copying only if the second half applies to you: snapshots of a whole server restore a whole server, which is the wrong unit when what you need back is one database and one config directory. Our backups go to S3-compatible object storage on a schedule, versioned, and they get restore-tested. If you are not going to build that, turn the checkbox on today. An untested €1.10 backup beats a disciplined plan you never wrote.

Who should use Hetzner, and who should not

Use it if you run your own stack, your traffic is bandwidth-heavy, your users are in Europe, and you would rather spend an hour on apt than €200 a month on managed services. The price-to-performance is not a marginal win, it is a factor of three or four, and the hardware has been boring in the best sense.

Do not use it if you need a managed database with a support contract behind it, a control plane somebody else gets paged for, or a region in Asia. The money you save will go straight back out as your own time, and if your time is the scarce resource, that trade is bad.

For us, across four years and three machines, the verdict is simple: the infrastructure has never been the problem. Everything on the incident list above we did to ourselves, and every one of those mistakes would have cost more to make somewhere else.

Originally published at news.rozoom.app.

acme.sh on a Server Where certbot Will Not Install

Rozoom News Editor — Tue, 27 Oct 2026 06:17:37 +0000

Four sites on one legacy box went dark. Not slow, not 500 - dark, as in the TLS handshake failed before a request existed. They had been down for five days before anyone noticed, because nothing was watching the one thing that mattered.

The cause was mundane: certificate renewal lived inside a control panel, the panel's licence expired, and its daily renewal cron died with it. Seven more certificates on the same machine were scheduled to expire over the following three weeks.

The distro was years past end of life, so certbot would not install cleanly and the vendor packages were long gone. This is the recipe that fixed it in under thirty minutes without touching a single vhost file, and the monitoring that makes sure it cannot happen quietly again.

Why acme.sh and not certbot

certbot is a Python application with a dependency tree. On a current distro that is fine. On an end-of-life one it is the whole problem: the system Python is too old for modern cryptography wheels, pip wants a compiler you do not want on a production box, and the snap route needs a daemon the distro never shipped.

acme.sh is a shell script. It needs bash and either curl or wget. That is the entire dependency list. It talks to the ACME API over HTTPS via curl, so even an ancient OpenSSL is not in the signing path, and bash 4.2 is new enough.

There is also a pleasant irony worth knowing if you are escaping a panel: several commercial panels use acme.sh under the hood. You are not replacing the mechanism, you are taking it out from under something that stopped running.

Install, and fix the default CA

curl https://get.acme.sh | sh -s email=you@yourdomain.dev
source ~/.bashrc

Then immediately do the step most guides skip. Since version 3, acme.sh defaults to ZeroSSL, not Let's Encrypt, and ZeroSSL requires account registration with EAB credentials. If you wanted Let's Encrypt, say so once:

acme.sh --set-default-ca --server letsencrypt
acme.sh --register-account -m you@yourdomain.dev

Skip this and your first issue attempt fails with a registration error that looks like a network problem.

Issue without touching the vhosts

The goal is zero changes to the web server configuration, because on a box like this the configs are generated by something you no longer control and a hand edit will be overwritten.

Two things make that possible.

One: use webroot validation into the directory the server already serves. Check first whether a path is already wired up - panels usually leave one behind:

curl -s -o /dev/null -w '%{http_code}\n' \
  http://yourdomain.dev/.well-known/acme-challenge/test

Anything other than a connection error means the path is reachable and you only need to find which directory on disk it maps to. Then:

acme.sh --issue -d yourdomain.dev -d www.yourdomain.dev \
  -w /var/www/letsencrypt

Two: write the certificate to the filenames the server already expects. Do not point the config at acme.sh's directory; point acme.sh at the config's filenames. --install-cert does exactly that, and the reload hook makes it live:

acme.sh --install-cert -d yourdomain.dev \
  --fullchain-file /etc/ssl/sites/yourdomain.dev.crt \
  --key-file      /etc/ssl/sites/yourdomain.dev.key \
  --reloadcmd     "nginx -s reload && apachectl graceful"

The content of the files changes, the paths never do, and nothing in the web server configuration is edited.

Expired certificates do not block renewal

The natural worry: every site redirects HTTP to HTTPS, and HTTPS is broken, so how does validation reach the challenge file?

It works. Let's Encrypt follows the redirect to HTTPS and does not validate the certificate during the HTTP-01 challenge. A long-expired certificate, a self-signed one, a hostname mismatch - none of it stops issuance. You do not need to unwind the redirects first, which is good, because on a generated config you could not anyway.

Automate it, then check for a rival

acme.sh --install-cronjob
crontab -l | grep acme

That second line is the important one, and it is the mistake we made. The panel's renewal cron was disabled - and a second panel cron doing the same job at a different hour was sitting four lines further down, missed on the first pass. Two renewal mechanisms racing over the same files is a problem that surfaces weeks later as a certificate that is valid on disk and stale in the server's memory.

Read the whole crontab. Comment out, do not delete, so you can see what used to be there:

crontab -l > /root/crontab.bak-$(date +%Y%m%d)

acme.sh renews at 60 days by default, so a failed run has 30 days of retries before anything is visible to users. That is a generous margin and exactly why a silent failure can run for months.

Monitor the certificate that is served, not the file on disk

This is the part that would have caught the original outage, and almost nobody sets it up.

Checking the file is not enough: it tells you that renewal wrote something, not that the running process picked it up. A reload that silently failed leaves a fresh file on disk and an expired certificate in memory. Check the handshake:

#!/usr/bin/env bash
# days until the served certificate expires, per domain
for d in "$@"; do
  end=$(echo | openssl s_client -servername "$d" -connect "$d":443 2>/dev/null \
        | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2)
  if [ -z "$end" ]; then echo "CRITICAL $d: no TLS handshake"; continue; fi
  days=$(( ( $(date -d "$end" +%s) - $(date +%s) ) / 86400 ))
  [ "$days" -lt 25 ] && echo "WARN $d: $days days left" || echo "OK $d: $days days"
done

Run it daily across every hostname the box serves and alert under 25 days. Two properties make it the right check: it fails loudly when the handshake fails at all, which is the real outage, and because it reads what the server hands a client, a reload that did not happen cannot hide behind a renewed file.

Add a tail of the renewal log to the same job and you have the whole story in one alert:

tail -n 5 /var/log/acme-renew.log

The result

Eleven certificates reissued in one pass, all valid for the next three months, four dead sites back up, renewal owned by a 200 KB shell script with a cron entry we can read. No vhost file was modified; only the bytes inside the certificate files changed.

The wider lesson is not about ACME clients. It is that the expiry of a thing is not the failure; the silence is. The renewal had been broken for roughly two months before the first site went down, and the only reason anyone found out was a site going dark in front of users. A twenty-line handshake check would have raised it the first week.

If you are putting a new box together rather than rescuing an old one, Caddy does all of this with no configuration at all - we wrote that stack up in Caddy and Docker in production. This recipe is for the machine you inherited, the one where the config is generated, the distro is frozen, and the only acceptable change is the smallest one.

Originally published at news.rozoom.app.

Best Privacy-First Email for Developers in 2026: Proton vs Infomaniak

Rozoom News Editor — Tue, 27 Oct 2026 06:17:01 +0000

Every side project ends up needing two different things from email, and most guides only talk about one of them.

The first is a human mailbox: you@yourdomain.dev, read on a laptop and a phone, kept away from an advertising company. The second is an outbound SMTP endpoint: the thing your app authenticates against at three in the morning to send a password reset. These are not the same job, the same traffic pattern, or the same failure mode, and the moment you treat them as one you get a project whose signups silently stop working.

This is a comparison of the two privacy-first providers that are genuinely worth running a domain on in 2026, with the prices as they are billed today, the DNS records you will actually paste, and an honest line about where each one stops.

The short version

	Proton	Infomaniak
Entry paid tier	Mail Plus, $3.99/user/mo billed yearly ($4.99 monthly)	kSuite Standard, CHF 1.76/user/mo (about €1.91)
Business entry	Mail Essentials, $6.99/user/mo yearly, 15 GB, 3 domains	Mail Service, CHF 2.29/address/mo (about €2.48)
Mailbox storage	15 GB on Mail Essentials, 1 TB on Workspace Standard	Unlimited on the mail service
Plain IMAP/SMTP for a desktop client	No. Needs the Bridge app, desktop only	Yes, standard IMAP and SMTP
SMTP for your application	Yes, submission tokens, paid plans, custom-domain addresses only	Yes, ordinary SMTP auth
End-to-end encryption between users	Yes, by default	No, encryption at rest and in transit
Bundled cloud storage	500 GB on Proton Unlimited at $9.99/mo yearly	50 GB kDrive per user on Standard, 3 TB on Business

Rates checked 25 October 2026; the CHF conversion uses 1 CHF = €1.0838 from 23 October 2026. Proton publishes USD on the plan pages and converts at checkout.

What "privacy-first" actually buys you

Be precise about the threat you are paying to remove, because the two providers remove different ones.

Proton encrypts mail end to end between Proton users, and at rest with zero-access encryption for everything else. The practical consequence is the one people forget: the provider cannot read your inbox, and neither can anything you plug into it. There is no plain IMAP, because there is nothing on the server to hand over in plaintext. That is the feature. It is also the limitation.

Infomaniak does not do end-to-end encryption. It does ordinary, well-run, European-hosted mail with encryption in transit and at rest, no ad profiling, and a company that sells hosting rather than attention. For most side projects that is the real alternative to a free Gmail account, and it speaks every protocol your tooling already knows.

If your mail must be unreadable to the provider, that decision is already made for you. If what you want is to stop being the product, Infomaniak is cheaper, faster to set up, and far less likely to fight your tools.

Developer reality check: the Bridge

This is the single thing that decides the choice for most people writing code, so it goes before the price table in your head.

Proton has no IMAP server you can connect to. To use Thunderbird, Apple Mail, mutt, neomutt, aerc, or anything else that speaks the protocol, you run Proton Bridge: a local daemon that holds your keys, exposes 127.0.0.1 IMAP and SMTP, and decrypts on your machine. It works well and it has a headless mode. But it is a process that has to be running, on a machine that has to be unlocked, which means:

Mail on a headless server through Proton means running the Bridge on that server and trusting it with the keys.
A phone uses the Proton app, not your preferred client. There is no way around this.
If the Bridge is not running, your client is simply offline.

Infomaniak gives you mail.infomaniak.com on 993 and 465, and your client connects. That is the whole paragraph.

Sending from your application

Both providers can send mail for an app. Neither should be your transactional mail provider. Those statements are compatible, and the distinction is where most people get burned.

Proton SMTP submission

Paid plans can generate SMTP submission tokens for custom-domain addresses. The settings:

host:     smtp.protonmail.ch
port:     587
security: STARTTLS
user:     noreply@yourdomain.dev
password: <the generated token>

Three details that are not obvious from the setup screen:

The token is shown once, at creation. Put it in your secret store the same minute.
Tokens only pair with custom-domain addresses, never with your @proton.me one.
Generate one token per client and per address. Revoking then costs you one integration instead of all of them.

A msmtp config, which is all a cron job or a small container needs:

account        proton
host           smtp.protonmail.ch
port           587
tls            on
tls_starttls   on
auth           on
from           noreply@yourdomain.dev
user           noreply@yourdomain.dev
passwordeval   "cat /run/secrets/smtp_token"

account default : proton

The limit nobody documents

Proton's published limits are 50 emails an hour and 150 a day on free accounts, and for paid accounts "it depends on the reputation of your account", with up to 100 recipients on a single message. There is no published paid ceiling.

Read that twice before you point a signup flow at it. An undocumented, reputation-shaped limit is fine for a contact form, a backup alert, or a nightly report. It is not a foundation for a product that sends onboarding mail in bursts, because the first time you hit the limit you will learn about it from your users rather than from a dashboard.

The rule worth keeping: your mailbox provider sends mail to you and as you. A transactional provider with a real API, webhooks and a bounce dashboard sends mail to your users. Pay one bill for each. The second bill is usually zero at side-project volume.

The DNS records, in full

Identical in shape for both providers, and the step people skip at the cost of a week in spam folders. Replace the selector and host values with the ones your provider shows.

; MX
yourdomain.dev.          3600  IN  MX   10 mail.provider.example.
; SPF - exactly one TXT record starting with v=spf1
yourdomain.dev.          3600  IN  TXT  "v=spf1 include:_spf.provider.example -all"
; DKIM - copy the public key your provider generates
sel._domainkey.yourdomain.dev. 3600 IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBg..."
; DMARC - start at none, read the reports, then tighten
_dmarc.yourdomain.dev.   3600  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.dev; fo=1"

Four things that cause nearly every delivery problem we have had to debug:

Two SPF records. Only one TXT record may begin with v=spf1. A second one, usually left behind by an old provider, invalidates both. Merge them.
-all before you are ready. Start with ~all, confirm in the DMARC reports that everything legitimate passes, then harden to -all.
p=reject on day one. You will reject your own mail. p=none for two weeks is not timidity, it is the only way to find the sender you forgot about.
A missing PTR when you send from your own box. Not relevant with either provider here, but it is the reason self-hosted SMTP on a VPS goes to spam. If you are tempted to run your own mail server, read Best VPS for self-hosting in 2026 first and then do it for the mailbox, not for sending.

Cost at the sizes that matter

One person, one domain. Proton Mail Plus at $3.99/month yearly gives one user, custom domain support and SMTP tokens. Infomaniak's standalone mail service is CHF 2.29 per address per month with unlimited mail storage, or CHF 1.76 per user on kSuite Standard, which bundles two addresses and 50 GB of cloud storage. Infomaniak wins on price; Proton wins if you want the mailbox encrypted against its own operator.

A small team of five. Proton Mail Essentials is $6.99 per user per month on a yearly term, so about $420 a year, with 15 GB per user and up to 3 domains. Infomaniak kSuite Business is CHF 7.33 per user (about €7.94), halved for the first year, and brings 3 TB of cloud storage per user, which is the line item that actually changes the comparison if you were about to pay for storage anyway.

You already pay for a VPN or cloud storage. Proton Unlimited at $9.99/month yearly folds mail, 500 GB, VPN and a password manager into one bill. Against four separate subscriptions it is the cheaper answer, and that bundle, not the mail client, is the honest reason most developers end up on Proton.

What we would pick

For a private mailbox on your own domain where the provider must not be able to read it, take Proton, accept the Bridge, and use the SMTP token only for low-volume machine mail.

For everything else - a team that lives in standard clients, a project that wants cheap mailboxes and storage in one place, a mail setup that should take fifteen minutes and never be thought about again - take Infomaniak.

And whichever you choose, send your product's mail through a transactional provider. The €2 mailbox and the 10,000-message sending tier are different purchases, and conflating them is the most expensive mistake in this entire article.

Next in this series: the full mail setup on a custom domain, from DNS to a working client, with the DMARC reports read out loud.

Originally published at news.rozoom.app.

Best Backup Storage for Self-Hosters in 2026: the price nobody shows you

Rozoom News Editor — Tue, 27 Oct 2026 06:16:25 +0000

Every comparison of backup storage quotes a price per terabyte. That number is real, but it is not the number that decides your bill. Most self-hosters do not back up a terabyte. They back up a database, a config directory and a photo folder - a few tens of gigabytes - and what they actually pay is set by each provider's minimum, not by the per-terabyte rate.

We tested this on our own data. We run two separate object-storage backups: an ERP database and a hosting panel. Here is what they are, read out of the buckets the day we wrote this.

What we actually back up

Backup	Objects	Size
ERP database	2,256	13 GB
Hosting panel	384	30 GB
Total	2,640	43 GB

That is the realistic self-hoster profile: a handful of services, a few tens of gigabytes, all of it worth keeping, none of it huge. Now price those same 43 GB everywhere.

The same 43 GB, six providers

	Headline price	Egress	Minimum	S3 API	Our 43 GB
Backblaze B2	$6.95 / TB	free up to 3x stored, then $0.01/GB	none	yes	≈ $0.30/mo
AWS S3 Standard	$24.50 / TB (eu-central-1)	$0.09/GB after 100 GB/mo free	none	yes	≈ $1.05/mo
Hetzner Storage Box BX11	€3.20 flat, 1 TB	unlimited	1 TB block	no	€3.20/mo
Infomaniak backup storage	CHF 4.18 / TB + CHF 1.84 per device	not published	1 TB block	yes	≈ CHF 6.02/mo
Hetzner Object Storage	€6.49 flat, includes 1 TB + 1 TB egress	1 TB included	1 TB block	yes	€6.49/mo
Wasabi	$7.99 / TB	$0	1 TB + 90 days	yes	$7.99/mo

Prices read from each vendor's own pricing page on the day of writing; European prices exclude VAT. Sorted by what our 43 GB costs, the per-terabyte ranking turns upside down: the "expensive" AWS row lands second-cheapest, and the cheap-per-terabyte rows cost an order of magnitude more than the pay-as-you-go leader.

The reason is the minimum, not the rate

Four of the six providers sell storage in blocks. You buy a terabyte whether you use 43 GB or 950 GB:

Wasabi states a minimum monthly charge tied to 1 TB per account, plus a 90-day minimum storage duration. Delete a file after a week and you still pay for it for 90 days.
Hetzner Object Storage charges a €6.49 base price that includes 1 TB of storage and 1 TB of egress. Below a terabyte you are paying for headroom.
Hetzner Storage Box is a fixed 1 TB box for €3.20.
Infomaniak sells backup capacity in 1 TB steps at CHF 4.18, plus CHF 1.84 per device you back up.

Only Backblaze B2 and AWS S3 bill purely for what you store. At 43 GB that difference is the whole story.

The flat prices are not a trap at the right size

None of this makes the block-priced providers bad. It makes them wrong at our size:

Store 900 GB and Hetzner Storage Box at €3.20 is the cheapest terabyte you can buy anywhere, with snapshots included.
Store several terabytes and restore often, and Wasabi's $0 egress starts paying for itself.
Hetzner Object Storage bundles 1 TB of egress into the base price, which for a 1 TB dataset means a full restore every month at no extra cost.

The decision is not "which provider is cheapest" but "where does my data size sit relative to their minimum".

The one that is not S3

Hetzner Storage Box is the cheapest row in the table and the one most likely to waste your afternoon. It is reachable over SFTP, rsync, SCP, SMB and WebDAV, and it has no S3 API. Hetzner sells S3 separately as Object Storage, at twice the price.

If your backup tool speaks S3 - and restic, rclone, duplicity, pgBackRest and most modern tools default to it - a Storage Box needs a translation layer or a different tool. Decide on the tool first, then on the storage.

The default that quietly costs the most

S3 Standard is where every tutorial starts, and at 43 GB it is genuinely fine: about a dollar a month, no minimum, and a 43 GB restore fits inside the 100 GB of free monthly data transfer AWS grants across its services. That allowance is the part most comparisons get wrong in both directions - AWS egress is not automatically expensive at small scale.

Where it turns is scale. At 500 GB you are paying $12 a month for storage that costs $3.50 on B2, and a full restore is 500 GB at $0.09/GB once past the free 100 GB: about $36, on the worst possible day. Per-request charges on PUT and LIST apply on top, which matters when your backup tool writes thousands of small objects - ours writes 2,640 of them per run.

Egress is the bill you pay on your worst day

Storage price is what you pay every month. Egress is what you pay the day you actually need the backup, which is exactly when you are in no position to shop around.

Backblaze B2: free egress up to 3x your average monthly storage, then $0.01/GB. At 43 GB that is ~129 GB free per month - three full restores, free.
Wasabi: $0 egress, always. Its real selling point.
Hetzner Object Storage: 1 TB of egress in the base price, then billed per TB.
Hetzner Storage Box: unlimited traffic, no egress charge by design.
AWS S3: 100 GB/month free across services, then $0.09/GB. Small restores free, large restores expensive.

Model one full restore before you commit, not after.

The recommendation, by size

Under ~500 GB, tool speaks S3 → Backblaze B2. No minimum, $6.95/TB, free egress up to 3x stored. Our 43 GB costs about 30 cents a month. This is the default for most self-hosters.

Around 1 TB, tool speaks rsync/SFTP → Hetzner Storage Box. €3.20 a month is the cheapest terabyte on this list, as long as you never need the S3 API.

Around 1 TB, tool speaks S3, you want one European provider for storage and mail → Infomaniak or Hetzner Object Storage. Both are block-priced, both are S3-compatible, both are in Europe; Infomaniak bills per device on top of capacity, Hetzner bundles a terabyte of egress.

Several TB with frequent restores → Wasabi. The 1 TB minimum stops being a penalty once you actually store that much, and $0 egress works in your favour.

Already inside AWS → S3 Standard with lifecycle rules. Move old versions to a cheaper class and set a retention policy, or you pay the highest storage rate here for the least forgiving restore path.

Whichever row you pick: make sure the backup exists and that you have restored from it at least once. The cheapest storage in the world costs nothing if you never pull from it, and the moment you do is the moment the pricing table becomes real.

FAQ

What is the cheapest object storage for small backups?
Backblaze B2, because it has no monthly minimum. Our 43 GB costs about $0.30/month. The flat-price competitors - Wasabi, Hetzner Object Storage and Storage Box, Infomaniak - charge for a full terabyte whether you use it or not.

Is Wasabi really cheaper than AWS S3?
Per terabyte yes, and it has no egress or API fees. But it carries a minimum monthly charge tied to 1 TB and a 90-day minimum storage duration, so below a terabyte you pay for storage you do not use. At 43 GB, Wasabi costs roughly eight times what AWS S3 costs.

Is Hetzner Storage Box S3-compatible?
No. It speaks SFTP, rsync, SCP, SMB and WebDAV. Hetzner's S3-compatible product is Object Storage, a separate service with a €6.49 base price that includes 1 TB of storage and 1 TB of egress.

Does Backblaze B2 charge for egress?
Free up to 3x your average monthly storage, then $0.01/GB. At a few tens of gigabytes that is effectively unlimited for restores.

How much does it cost to restore 43 GB?
Nothing, on every provider here: B2 within its 3x allowance, Wasabi and Storage Box by design, Hetzner Object Storage within its included terabyte, and AWS within the 100 GB of free monthly transfer. Egress only starts to hurt at hundreds of gigabytes - which is exactly when nobody checks.

Which backup storage has no egress fees at all?
Wasabi ($0 egress, always) and Hetzner Storage Box (unlimited traffic). The others include an allowance and charge beyond it.

What's next

Backups are only as good as the restore you have practised. The companion pieces: self-hosted analytics on the same box, and the VPS comparison that hosts the whole stack.

Originally published at news.rozoom.app.

Self-Hosted Analytics in 20 Minutes, and the Traffic It Will Never Show You

Rozoom News Editor — Tue, 27 Oct 2026 06:15:49 +0000

Eleven days ago we put a privacy-first analytics server next to a site that was already running. No new machine, no managed service, no cookie banner. It took about twenty minutes, it now uses 340 MB of RAM, and its database has grown to 9.5 MB.

It also recorded 96 pageviews in those eleven days, while the web server in front of it logged 2016 successful page responses.

Both numbers are real, both are from the same box, and the gap between them is the most useful thing in this article. Start with the install, because the install is the easy part.

Why self-host this at all

Three reasons, in the order they actually matter.

You already pay for the box. Hosted analytics is priced per month and scales with pageviews, so the bill grows exactly when the site starts working. A container next to an existing site costs whatever 340 MB of RAM costs you, which on a €5.49 server is a rounding error. We sized the rest of the fleet in our Hetzner review and in the VPS comparison.

No cookie banner. Umami does not set cookies and does not build cross-site profiles. That is the difference between a site that opens with content and a site that opens with a consent dialog, and it is worth more than any metric the dialog is protecting.

The data stays in your Postgres. You can query it with SQL, back it up with everything else, and delete it when you want to. No export quota, no retention tier.

The honest counter-argument: you now operate a database and an app that nobody else patches for you. If you will not run docker compose pull once a month, use the hosted version and stop reading.

The compose file, for a box that already runs something

This is the part most guides get wrong by assuming an empty machine. The realistic case is a server with a reverse proxy and a site on it already, and the goal is to add two containers without touching what works.

services:
  umami-db:
    image: postgres:17-alpine
    container_name: umami-db
    restart: unless-stopped
    env_file: [.env.umami]
    volumes:
      - umami_db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U umami"]
      interval: 10s
      timeout: 5s
      retries: 10

  umami:
    image: ghcr.io/umami-software/umami:postgresql-latest
    container_name: umami
    restart: unless-stopped
    env_file: [.env.umami]
    depends_on:
      umami-db:
        condition: service_healthy

volumes:
  umami_db:

Two details are doing real work here.

Neither container publishes a port. They talk to the reverse proxy over the compose network and are unreachable from the internet except through it. If you add ports: ["3000:3000"] out of habit, you have just put an admin login on a public port with a default password on it, and the firewall is the only thing between you and a bad evening.

The healthcheck is not decoration. Umami runs its migrations on first boot. Start it against a Postgres that is still initialising and the first run fails on a connection error, leaving you to debug a problem that fixes itself on restart. condition: service_healthy removes the race.

The .env.umami file next to it:

POSTGRES_DB=umami
POSTGRES_USER=umami
POSTGRES_PASSWORD=<long random string>
DATABASE_URL=postgresql://umami:<same string>@umami-db:5432/umami
APP_SECRET=<another long random string>

DATABASE_URL is the only variable the application truly requires. APP_SECRET signs the authentication tokens, and the documentation is explicit that every installation should have its own. Generate both the same way and never reuse them:

openssl rand -hex 32

Keep that file at chmod 600. It holds the database password in two places.

Wiring it to Caddy

If you already run Caddy the way we described in the Caddy and Docker setup, the whole integration is five lines:

stats.yourdomain.dev {
    encode zstd gzip

    header {
        X-Content-Type-Options nosniff
        Referrer-Policy no-referrer
        -Server
    }

    reverse_proxy umami:3000
}

Caddy resolves umami over the compose network and provisions the certificate on first request. There is no TLS configuration to write, which is the entire reason to use it on a small box.

Point the DNS record at the server before you reload, or the certificate request fails and Caddy backs off for a while.

The step you must not postpone

Umami creates a default account on first boot: username admin, password umami. This is documented, which means every scanner on the internet knows it too.

curl -s -o /dev/null -w '%{http_code}\n' \
  -X POST https://stats.yourdomain.dev/api/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"username":"admin","password":"umami"}'

401 is the answer you want. Anything else means the change did not take, and you have a public dashboard with a published password on it.

Then add your site in the UI and drop the snippet it gives you into your HTML head:

<script defer src="https://stats.yourdomain.dev/script.js"
        data-website-id="your-website-id"></script>

The website id is not a secret. It identifies which site a pageview belongs to, nothing more.

What eleven days of real numbers look like

Here is the whole cost of running this, measured on a live box rather than estimated:

Metric	Value
Umami container memory	302 MB
Postgres container memory	37 MB
Database size, 96 pageviews	9.5 MB
CPU, both containers, steady state	0.00%
Uptime since install, restarts	11 days, zero

The 9.5 MB is almost entirely empty schema. Postgres does not grow linearly with a handful of events, so do not read a per-pageview cost into that number. The memory figure is the one that matters for capacity planning: budget roughly 350 MB and you will not be surprised.

Now the uncomfortable part

Same box, same eleven days, two sources of truth that disagree violently.

The access log recorded 2016 successful HTML responses. Umami recorded 96 pageviews across 26 sessions. If you take those at face value, analytics saw 4.8% of your traffic and you should panic.

Do not panic. Take the log apart instead.

Filtering user agents containing bot, spider, crawl, curl, python, wget or headless removes 1143 of those 2016 requests. More than half the traffic to a new site is openly declared automation, and that is before counting the ones that lie.

Of the 873 that survive, 196 are GoogleOther - a Google crawler whose user agent contains no occurrence of the string bot at all. If your log analysis greps for "bot", you have been silently counting a crawler as a reader. That one is worth remembering the next time a dashboard built on access logs tells you something flattering.

The remainder still contains unlabelled scrapers. A heavy concentration of one exact ancient Safari build, for instance, is a fingerprint, not an audience.

So the real comparison is 96 recorded pageviews against a few hundred plausibly-human requests, and the gap has three sources, in descending order of size:

Automation that does not run JavaScript. It appears in the log and never in analytics. This is most of the gap, and it is analytics being right.
Blocked trackers. Some filter lists block by hostname pattern, and a host called stats. is a pattern. Self-hosting does not grant immunity, it only removes the well-known third-party domain. The project documents proxying the script under your own domain and moving the collection endpoint with COLLECT_API_ENDPOINT if you need to close that gap.
Prefetch and no-JS readers. Small, real, unfixable.

The conclusion is not "logs are honest and analytics lies". It is that they measure different populations: the log counts requests, analytics counts browsers that execute script. A new site's log is mostly robots, which is why reading raw logs makes a young site feel popular and is the single easiest way to lie to yourself about traction.

If you want one number to steer by in the first months, use sessions from analytics and treat the log as a debugging tool.

Consent, briefly and carefully

Cookieless analytics avoids the consent requirement that attaches to storing or reading information on the visitor's device, which is what makes the banner mandatory for conventional tracking. It does not make the processing invisible: an IP address still reaches your server, as it does for every request to every site.

Practical consequences, not legal advice: say in your privacy page what you collect, keep it aggregate, and set IGNORE_IP for your own addresses so your dashboard is not mostly you. If your situation is regulated, ask someone qualified rather than a blog.

Maintenance, which is the whole deal you signed

docker compose pull umami umami-db
docker compose up -d umami umami-db

That is the monthly job. Schema migrations run automatically on container start, which is convenient and also exactly why the volume matters: umami_db is the only thing you cannot recreate from this file. Back it up with the rest of your data and test the restore once, or you are keeping an untested copy of 9.5 MB and a false sense of safety.

After the first week, do one more thing: open the dashboard and compare it against what you know to be true. If you visited the site yourself twenty times and the dashboard shows two, something between the script tag and the collector is being blocked, and better to learn that now than in three months, when you are making decisions from the number.

Originally published at news.rozoom.app.

Where You Can Actually Run a Blockchain Node in 2026

Rozoom News Editor — Tue, 27 Oct 2026 06:15:13 +0000

Search for how to run an Ethereum or Bitcoin node and you get a hundred tutorials that start with "rent a VPS". Not one of them checks whether that VPS provider allows it. Then the server disappears, usually with no warning, and the tutorial's comment section fills up with people who lost a synced chain.

The confusion has a simple root: mining and running a node are different activities, and hosting contracts are mostly written as if only mining exists. This post is what the contracts actually say, quoted from the providers' own legal pages, plus what a node really costs in disk, RAM and bandwidth.

Mining, staking, and just running a node

Three different things get lumped together:

Mining burns CPU or GPU cycles continuously to win blocks. It saturates hardware, and it is why hosts care: it makes shared infrastructure slow for everyone and it attracts fraud.
Running a full node downloads and validates the chain. It is mostly disk and bandwidth, with modest CPU. It earns nothing.
Staking or validating runs a full node plus signing duties, with capital locked up. Also not CPU-heavy.

A provider that bans "mining" has written a rule about the first one. Whether that rule covers the other two is usually not stated anywhere, and that gap is where people lose servers.

What the contracts actually say

Read on the providers' own legal pages at the time of writing:

Provider	Mining	Nodes mentioned?	Exact wording
Hetzner	prohibited	no	"The operation of applications for mining cryptocurrencies remains prohibited." Elsewhere: "mining, farming and plotting of cryptocurrencies."
Netcup	prohibited	no	"Operation of mining-services such as 'Bitcoin', 'Ethereum', 'OneCoin' or 'Monero'."
DigitalOcean	prohibited without permission	no	"Mining any cryptocurrency, including but not limited to BitCoin or any similar currency, without explicit written permission."
Scaleway	prohibited	no	"not to engage in any harmful practices, such as but not limited to: cryptocurrency mining, Black Hat SEO, spamming..."
Contabo	not mentioned in the terms	sells node hosting	no crypto clause in the terms and conditions
OVHcloud	not mentioned in the terms we read	sells node hosting	no crypto clause in the dedicated-server conditions

Two things stand out.

Not one of these contracts uses the word "node". Every ban is written about mining. If you run a Bitcoin full node on a provider whose rule says "mining", you are in a grey area that a support agent resolves, not a lawyer.

DigitalOcean is the only one that leaves a door open in writing. "Without explicit written permission" means there is a process: ask, get an answer in writing, keep it. If you want a node on a mainstream cloud and you want to sleep at night, that sentence is the most useful one in this table.

Hetzner is the clear no

Hetzner's terms ban mining outright, and in 2022 the company went further than the written text: a spokesperson stated that even running a single node violates its terms, and over a thousand Solana validators went offline when the enforcement landed. The written policy still only says "mining", but the enforcement record says nodes. Treat Hetzner as off-limits for anything chain-related, however cheap the server is.

That is a genuine loss, because for everything that is not a node, Hetzner is usually the price-performance answer. We say so in our own VPS comparison and we still mean it. Just not for this.

Contabo and OVHcloud sell it as a product

The flip side is the part the tutorials miss. Two of the providers in the table do not merely tolerate nodes, they advertise them.

Contabo has a page titled Cryptocurrency Nodes at Contabo. OVHcloud runs a blockchain infrastructure line whose copy reads "Run your node, chain, or layer 2 exactly how you want to" and "A network that keeps your blockchain nodes synched". When a provider markets the exact workload you are deploying, your terms-of-service risk is close to zero: you are the customer they built the page for.

What we could not verify

Vultr's use policy returns an access error to automated requests, so we could not read it first-hand and have left it out of the table rather than quote a figure we did not see. Latitude.sh's terms defer to a third-party master agreement we did not read. If you are choosing between those two, open the pages yourself before you deploy.

What a node actually needs

From each project's own documentation, not from forum folklore:

	CPU	RAM	Disk	Notes
Bitcoin Core	1 GHz, any modern core	512 MB minimum	750 GB	740 GB one-time initial download
Ethereum (Geth + consensus client)	quad core	16 GB	2 TB SSD	grows ~14 GB/week; archive node over 12 TB
Solana validator	12 cores / 24 threads	256 GB ECC	1 TB accounts + 1 TB ledger + 500 GB snapshots, separate NVMe each	RPC node: 16 cores, 512 GB

The Ethereum line is the one most people get wrong. A 2 TB disk is not a comfort margin, it is the requirement, and the chain grows about 14 GB a week underneath you. A node that fits today stops fitting in a year.

The Solana line is not a self-hosting project at all. 256 GB of ECC memory and three separate NVMe drives is a dedicated machine, and Solana's own docs add a line worth quoting to anyone planning it on a cheap VPS: running in the cloud "requires significantly greater operational expertise to achieve stability and performance".

The bandwidth bill nobody mentions

This is the number missing from every node tutorial. A Bitcoin Core node with default settings uploads 5 GB a day, about 150 GB a month, serving blocks to other peers. It is not an accident, it is the job.

What that costs depends entirely on how your host bills traffic:

Unmetered or generous included traffic (most European VPS providers): free.
Metered cloud egress: 150 GB a month at $0.09/GB is about $13.50 a month in bandwidth alone, which is more than the server on most plans.

So the hosting decision for a node is not "which VPS has the best CPU". It is "which VPS does not meter the 150 GB a month my node will push out by design". You can cap it (Bitcoin Core takes a bandwidth limit), but then you are a worse peer and a slower node.

Does any of it pay?

Worth saying plainly, because the search results around this topic are full of promises:

A plain full node earns nothing. Bitcoin, Ethereum, anything. It is infrastructure you run for sovereignty, privacy or an application you are building, not income.
Ethereum staking needs 32 ETH locked up. At the time of writing that is roughly 74,000 USD of capital for a yield in the low single digits, before the server bill. The yield is denominated in an asset whose price moves more in a week than the yield pays in a year.
Solana validating costs money before it earns any: the docs state that voting "can cost up to 1.1 SOL per day" in vote transactions. At today's price that is about 110 USD a day, roughly 40,000 USD a year, paid regardless of what you earn, and you need a large delegated stake to cover it.

Run a node because you want your own view of the chain. Do not run one as a side income: the economics only work with capital or scale you do not have if you are reading a self-hosting blog.

Before you deploy, do this

Open the provider's acceptable use policy and search it for "mining". Thirty seconds. If the word is there and "node" is not, you are in the grey area.
If it is a grey area and the server matters, ask support in writing. On DigitalOcean this is explicitly the documented path. Keep the reply.
Check how egress is billed before you check CPU benchmarks. 150 GB a month is the baseline for Bitcoin.
Size the disk for next year, not for today. Ethereum adds roughly 14 GB a week.
Prefer a provider that advertises the workload. A marketing page about node hosting is a stronger guarantee than silence in the terms.

FAQ

Can I run a crypto node on Hetzner?
No. Hetzner's terms prohibit mining applications, and in 2022 the company stated that even a single node violates its terms, removing over a thousand Solana validators in the process. The written text still says "mining", but the enforcement record is clear.

Is running a node the same as mining?
No. Mining burns CPU or GPU continuously to win blocks. A full node downloads and validates the chain: mostly disk and bandwidth, with modest CPU, and it earns nothing. Most hosting contracts only ban the first and say nothing about the second.

Which hosting providers allow blockchain nodes?
Contabo and OVHcloud both market node hosting as a product, which is the strongest signal you can get. DigitalOcean allows crypto workloads with explicit written permission. Hetzner, Netcup and Scaleway prohibit mining in their terms.

How much disk does an Ethereum node need?
2 TB SSD for a full node running Geth plus a consensus client, per Geth's own documentation, and the database grows about 14 GB per week. An archive node needs over 12 TB.

How much bandwidth does a Bitcoin node use?
About 5 GB per day of upload, 150 GB per month, with default settings, plus a one-time 740 GB download for the initial sync. On a metered cloud that traffic can cost more than the server.

Can I make money running a node?
Not from a plain full node: it earns nothing. Ethereum staking needs 32 ETH of capital. Solana validating costs up to 1.1 SOL per day in vote fees before it earns anything. Nodes are infrastructure, not income.

What's next

If you are picking a server for something that is not a node, our VPS comparison has live pricing from three vendors, and the backup storage comparison covers where the chain snapshots should go.

Originally published at news.rozoom.app.

Caddy and Docker in Production: the Whole Config Behind This Site

Rozoom News Editor — Tue, 27 Oct 2026 06:14:55 +0000

This site runs on one small VPS. On the same box: a static blog, a self-hosted analytics stack, two internal APIs and a password-protected app. One Caddy container terminates TLS for all of them, renews certificates without being asked, and has needed exactly zero manual attention since it was set up.

This post is that setup, config file included, plus the two mistakes we made along the way. If you want the hardware side first, we compared the options in Best VPS for self-hosting in 2026 - everything below runs comfortably on the cheapest tier there.

Why Caddy instead of nginx

nginx is excellent and we still run it as the static file server inside this stack. But for the edge, Caddy wins on one thing that matters at this scale: TLS is not a project. No certbot, no renewal cron, no --dry-run rehearsal at 2 a.m. You write a hostname, Caddy obtains the certificate on first request and renews it in the background.

The trade-off is real and worth knowing: fewer tuning knobs, a smaller ecosystem of copy-paste configs, and a config language that is its own thing rather than nginx syntax you already know. For a single-box setup, that trade is strongly in your favour.

The Compose file

Five services, one network, no orchestration:

services:
  blog:
    image: nginx:alpine
    container_name: blog
    restart: unless-stopped
    volumes:
      - ./site:/usr/share/nginx/html:ro

  analytics-db:
    image: postgres:17-alpine
    container_name: analytics-db
    restart: unless-stopped
    env_file: [.env.analytics]
    volumes:
      - analytics_db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U analytics"]
      interval: 10s
      timeout: 5s
      retries: 10

  analytics:
    image: ghcr.io/umami-software/umami:postgresql-latest
    container_name: analytics
    restart: unless-stopped
    env_file: [.env.analytics]
    depends_on:
      analytics-db:
        condition: service_healthy

  caddy:
    image: caddy:2
    container_name: caddy
    restart: unless-stopped
    ports: ["80:80", "443:443"]
    volumes:
      - ./caddy:/etc/caddy:ro
      - caddy_data:/data
      - caddy_config:/config

volumes:
  analytics_db:
  caddy_data:
  caddy_config:

Four details in there are load-bearing:

Only Caddy publishes ports. Nothing else binds to the host. The blog and the analytics app are reachable on the Compose network by container name and nowhere else. This is the single biggest security win of the layout: you cannot accidentally expose a database to the internet if it has no published port.

caddy_data is a named volume, not a bind mount. It holds the certificates and the ACME account key. Lose it and you re-issue everything on next start, which is survivable but counts against rate limits if you do it repeatedly while debugging.

The database has a healthcheck and the app waits on it. Without condition: service_healthy, the app container starts first, fails to connect, and restarts in a loop until Postgres is up. It converges either way, but the logs are a mess and a first deploy looks broken when it is not.

restart: unless-stopped, not always. The difference shows up when you deliberately stop a container to debug it and then reboot the host: always brings it back and overrides your intent.

The Caddyfile

One block per hostname. This is the real structure, with our specifics swapped for placeholders:

blog.example.com {
    encode zstd gzip

    header {
        X-Content-Type-Options nosniff
        Referrer-Policy no-referrer-when-downgrade
        -Server
    }

    # fingerprinted assets can be cached hard, html stays revalidated
    @immutable path /assets/* /favicon*.png /apple-touch-icon.png /og-default.png
    header @immutable Cache-Control "public, max-age=31536000, immutable"
    header /favicon.ico Cache-Control "public, max-age=604800"

    reverse_proxy blog:80
}

stats.example.com {
    encode zstd gzip
    header -Server
    reverse_proxy analytics:3000
}

app.example.com {
    basic_auth {
        admin $2a$14$...bcrypt hash here...
    }
    encode zstd gzip
    reverse_proxy app:3000
}

That is the entire TLS configuration: there isn't any. Each hostname gets a certificate on first request, renewed automatically, with HTTP redirected to HTTPS for free.

Three things worth copying:

-Server removes the server header. Minor, but free. Caddy announces itself by default.

The cache split matters more than it looks. Hashed asset filenames can be cached for a year because a content change produces a new filename. HTML must not be, or visitors keep reading a stale page after you publish. Getting this backwards is the classic "I deployed but nothing changed" bug, and it is invisible to you because your own browser revalidates on reload.

basic_auth takes a bcrypt hash, not a password. Generate it with docker exec caddy caddy hash-password. Never put the plaintext in the file.

Redirects as a config concern

Our affiliate links never point at a partner directly. Every one goes through a redirect defined in the edge config:

    redir /go/vendor-a* https://www.vendor-a.example/ 302
    redir /go/vendor-b* https://www.vendor-b.example/ 302
    redir /go/* /affiliate-disclosure/ 302

Two reasons, and the second is the one nobody thinks of until it bites:

Partner URLs change. Tracking parameters get reissued, programmes move domains. With this layout you edit one line and reload Caddy; without it you rewrite every post that ever linked there.
The catch-all protects you. /go/* falling through to the disclosure page means a typo or a retired programme produces a real page, not a 404 in the middle of a monetised article.

Keep the trailing * on each pattern so query strings survive the hop, and keep 302 rather than 301: a permanent redirect to a partner URL you will eventually change is cached in browsers you do not control.

Two mistakes we actually made

Bind-mounting a single file pinned a stale inode. The Caddyfile was mounted as ./Caddyfile:/etc/caddy/Caddyfile. Edits on the host stopped reaching the container: Docker bind-mounts a file by inode, and most editors write a new file and rename it over the old one, which leaves the container holding the deleted original. The site served an old config for days with nothing in the logs to say so. Fix: mount the directory, not the file, as in the Compose file above.

An unrestricted deploy key, still in use. The blog's own deploy key is locked to a single rsync target, but an older CI key for another app on the same box was created with no restrictions at all: whatever holds it gets an interactive shell, not write access to one directory. It has worked perfectly for months, which is exactly why nobody looked at it. Audit every line in authorized_keys, not just the one you set up most recently - the next post is about getting that right.

Operating it

The whole maintenance surface is three commands:

docker compose up -d          # apply changes
docker exec caddy caddy reload --config /etc/caddy/Caddyfile   # config only, no downtime
docker compose logs -f caddy  # watch the ACME dance on a new hostname

Add a new site by adding a service and a hostname block, then reload. First request to the new name takes a second or two while the certificate is issued; after that it is indistinguishable from the rest.

One caveat on certificates: if your DNS sits behind a proxy that terminates TLS itself, the ACME challenge Caddy prefers can fail in ways that look like a Caddy bug. Point the record straight at the box while you are setting up, then put the proxy back if you want it.

What's next

The companion piece: deploying to this box from GitHub Actions without giving CI a shell - the restricted-rsync setup we should have used from day one.

Originally published at news.rozoom.app.