DEV Community: Pavel

Let's Encrypt Wildcard Certs in Kubernetes: cert-manager + DNS-01 (and When We Skipped It)

Pavel — Tue, 05 May 2026 16:52:23 +0000

If you run Kubernetes and want a wildcard TLS cert from Let's Encrypt — say *.example.com — you need a DNS-01 challenge. HTTP-01 cannot prove control over a wildcard. That single fact rules out the easy path most tutorials show.

This post is what we actually run at Hostim.dev for our shared *.region.hostim.dev wildcard. We use cert-manager for per-app certs and a plain certbot Ansible playbook for the wildcard. Two different tools for two different jobs. We will explain why, then show the code for both.

Why two tools for one cluster?

You can do everything with cert-manager. It supports DNS-01 with a long list of providers. So why are we running a second tool?

Three reasons:

Our DNS provider (Namecheap) does not have a stable cert-manager webhook. There are community webhooks, but they break on upgrades. Maintaining one for a single cert is more work than running certbot once a quarter.
The wildcard cert covers our shared ingress, not user apps. It rotates rarely, lives in one namespace, and is read by every ingress as a TLS secret. cert-manager is built for the opposite case: many short-lived certs per Ingress.
A failed cert-manager renewal at 3 a.m. is hard to debug. A failed Ansible run on our laptop is a stack trace we can read.

For per-app domains (my-app.user.tld with cert-manager + HTTP-01), the controller-driven model wins. For the one shared wildcard, the manual model wins. Use the right tool.

Path A: cert-manager + HTTP-01 (per-app domains)

This is the standard path. Most apps want a cert for one or two hostnames. HTTP-01 is the simplest challenge: cert-manager spins up a temporary pod, the ACME server hits http://app.example.com/.well-known/acme-challenge/..., the pod responds, the cert is issued.

1. Install cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml

Wait for the three pods (cert-manager, cert-manager-webhook, cert-manager-cainjector) to be ready.

2. Create a ClusterIssuer

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: you@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account
    solvers:
      - http01:
          ingress:
            class: nginx

Apply it. cert-manager will register an ACME account on first use.

3. Annotate your Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
    - hosts: ["app.example.com"]
      secretName: app-example-com-tls
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app
                port:
                  number: 80

That is it. cert-manager sees the annotation, requests the cert, solves the HTTP-01 challenge, writes the cert into the app-example-com-tls secret. Renewal is automatic.

This works for any number of distinct hostnames. We do this exact thing for every user app on hostim.dev.

Path B: certbot + DNS-01 (the wildcard)

For *.region.hostim.dev, HTTP-01 cannot work — the ACME server cannot resolve every possible subdomain. We need DNS-01: prove control over the parent domain by adding a TXT record.

You can do this with cert-manager and a DNS-01 webhook for your provider. We chose not to. Here is the Ansible playbook we run instead.

The flow

Ansible writes two scripts: an auth hook (creates the TXT record) and a cleanup hook (deletes it).
certbot --manual --preferred-challenges dns runs the auth hook, waits for DNS to propagate, lets ACME verify, then runs the cleanup hook.
The resulting fullchain.pem and privkey.pem get loaded into a Kubernetes Secret of type kubernetes.io/tls.
Every ingress in the shared namespace references that secret.

The playbook (trimmed)

- name: Issue and upload wildcard TLS certificate
  hosts: localhost
  vars:
    sld: "example"
    tld: "com"
    region: "eu-center"
    wildcard_domain: "*.{{ region }}.{{ sld }}.{{ tld }}"
    local_tmp: "/tmp/wildcard-{{ region }}"
    k8s_namespace: "ingress-nginx"
    k8s_secret_name: "wildcard-{{ region }}-tls"

  tasks:
    - name: Create certbot auth hook (creates the TXT record)
      copy:
        dest: "/tmp/certbot-auth-{{ region }}.sh"
        mode: "0755"
        content: |
          #!/bin/bash
          set -e
          namecheap-cli setone \
            --sld {{ sld }} --tld {{ tld }} \
            --type TXT --name "_acme-challenge.{{ region }}" \
            --address "${CERTBOT_VALIDATION}" --ttl 60
          # Wait for DNS to propagate
          for i in {1..30}; do
            val=$(dig TXT _acme-challenge.{{ region }}.{{ sld }}.{{ tld }} @1.1.1.1 +short | tr -d '"')
            [[ "$val" == "${CERTBOT_VALIDATION}" ]] && break
            sleep 10
          done
          sleep 30  # belt and suspenders

    - name: Issue wildcard certificate
      command: >
        certbot certonly --manual --preferred-challenges dns
        --manual-auth-hook /tmp/certbot-auth-{{ region }}.sh
        --manual-cleanup-hook /tmp/certbot-cleanup-{{ region }}.sh
        --agree-tos -m you@example.com
        --server https://acme-v02.api.letsencrypt.org/directory
        -d "{{ wildcard_domain }}"
        --work-dir {{ local_tmp }} --config-dir {{ local_tmp }}
        --logs-dir {{ local_tmp }} --non-interactive

    - name: Create or update TLS Secret
      kubernetes.core.k8s:
        state: present
        namespace: "{{ k8s_namespace }}"
        definition:
          apiVersion: v1
          kind: Secret
          metadata:
            name: "{{ k8s_secret_name }}"
          type: kubernetes.io/tls
          data:
            tls.crt: "{{ lookup('file', local_tmp + '/live/.../fullchain.pem') | b64encode }}"
            tls.key: "{{ lookup('file', local_tmp + '/live/.../privkey.pem') | b64encode }}"

Reference the secret in your Ingress

spec:
  tls:
    - hosts: ["*.region.example.com"]
      secretName: wildcard-region-tls

When does it run?

We run the playbook every 60 days. Let's Encrypt certs are valid for 90 days, so 60 leaves a 30-day buffer. A simple cron on a bastion host is enough — we do not even need to automate this. The cost of a manual run twice a quarter is lower than the cost of debugging a webhook.

"Unable to locate package 'appengine'" — a real gotcha we hit

If you copy this playbook and your certbot is from your distro's package manager, you may hit:

ImportError: cannot import name 'appengine' from 'urllib3.contrib'

This is a Python env collision. System certbot (often 1.21) wants old urllib3; you have a newer one in ~/.local/lib/python3.10/site-packages. The newer version dropped appengine.

Quick fix — add PYTHONNOUSERSITE: "1" to the certbot task's environment:

- name: Issue wildcard certificate
  environment:
    PYTHONNOUSERSITE: "1"
  command: >
    certbot certonly --manual ...

Long-term fix — install certbot via snap or pipx so it has its own Python env.

Should you do it this way?

Probably not. If your DNS provider has a stable cert-manager webhook (Cloudflare, Route53, DigitalOcean, Google Cloud DNS), use cert-manager for both per-app and wildcard certs. It is simpler and renews automatically.

The hybrid model only makes sense when:

Your DNS provider has no first-party or stable cert-manager support
You have one wildcard, not many
You would rather audit a 30-line shell script than a webhook deployment

For us those three are all true. For most teams, only the first might be — and even then, switching DNS provider is often easier than maintaining a webhook.

TL;DR

Per-app domains → cert-manager + HTTP-01 + ClusterIssuer. One annotation per Ingress, automatic renewals.
Wildcards → DNS-01 is mandatory. Use cert-manager with your DNS provider's webhook if it exists. Otherwise, a 60-day Ansible run with certbot --manual and a TLS Secret.
Two tools is fine. Don't force one model onto two different problems.

Want to skip TLS entirely?

Hostim.dev does this for you. Bring a Docker image or a git repo, get a cert and a domain.

Should Small Teams Even Bother with Kubernetes?

Pavel — Sun, 26 Apr 2026 10:43:15 +0000

Most small teams hit the same question at some point: should we move to Kubernetes? The honest answer for the majority of them is no, but that answer alone is not very helpful. So here is the longer version, with real prices and a clear line where the answer flips to yes.

What Kubernetes gives you

Underneath the marketing, Kubernetes is four practical things:

Declarative scheduling – you describe the desired state and a controller keeps the cluster in that state
Self-healing – crashed pods restart, dead nodes are drained, replicas come back automatically
Bin-packing – many workloads share the same nodes with CPU and memory limits
A standard API – Deployments, Services, Ingress, Jobs, Secrets, all the same on any cluster

These are real benefits. The catch is that you pay for them in money, time, or both.

What it actually costs

A realistic small-team setup looks like this: 3 services (API, worker, frontend), one Postgres, one Redis, around 50GB of storage. Here is what the same workload costs in three common shapes, all in eu-central-1 / Frankfurt.

Managed Kubernetes (AWS EKS)

EKS control plane, 0.10 USD per hour: ~67 €/mo
3× t3.medium nodes (2 vCPU / 4GB): ~92 €/mo
RDS Postgres db.t3.small, Single-AZ: ~27 €/mo
ElastiCache Redis cache.t3.micro: ~13 €/mo
ALB base plus LCU: ~15 €/mo
50GB EBS gp3 plus ~200GB egress: ~14 €/mo

Total: around 228 €/mo, before backups, observability, or any of your time.

GKE used to give you the first cluster for free. That is gone now: control plane is 0.10 USD per hour, and you get a 74.40 USD monthly billing-account credit that offsets one zonal cluster. Regional clusters pay full price.

Single Hetzner box + Docker Compose

AX42 dedicated (8-core Ryzen 7 PRO, 64GB DDR5, NVMe): from 57 €/mo (April 2026 pricing)
Postgres, Redis, app – all containers on the same machine, isolated by Compose
nginx and Let's Encrypt for HTTPS
Storage Box BX11 (1TB) for backups: ~4 €/mo

Total: around 61 €/mo. That box has enough headroom to run several more projects beside the main one.

Self-hosted Kubernetes (k3s on Hetzner Cloud)

3× CCX13 (2 dedicated vCPU / 8GB / 80GB SSD): ~48 €/mo
You run the control plane, etcd, ingress controller, cert-manager, backups and monitoring yourself

Compute is around 48 €/mo, but the real cost is the hours you put into the cluster every week.

The ops cost nobody prices in

Hosting is the cheap part. Kubernetes adds work that simply does not exist with Compose:

Cluster upgrades. A new minor lands every four months. If you skip a few, the upgrade path becomes painful.
Ingress and cert-manager. Works fine until cert-manager hits a CRD migration or your ingress controller deprecates an annotation you depend on.
CNI debugging. A misbehaving Calico or Cilium pod can take half a day to track down.
RBAC and ServiceAccounts. Required even for trivial things like letting one pod read one secret.
PVCs and storage classes. A reboot at the wrong moment can leave a volume stuck in Terminating and you reading the controller logs.
etcd. Quiet most of the time, then your cluster is suddenly read-only at 2am and you are restoring from a snapshot.

Realistic estimate: 2 to 5 hours a week of cluster maintenance for a self-hosted setup. Managed clusters cost less time but more money, as the table above shows. For a 3-person team, 2-5 hours a week is 5-12% of one engineer's time spent on infrastructure that does not ship features.

When Kubernetes is the right call

There are real cases where the cost is justified:

You run more than 20 services that need consistent deploys, secrets and networking
Multi-region or multi-tenant with hard isolation per customer
Compliance work (SOC 2, HIPAA) where audited RBAC and NetworkPolicies save weeks of paperwork
Your team already knows Kubernetes well and Compose would slow them down
Bursty workloads that genuinely benefit from horizontal autoscaling on shared nodes
You are building a platform where the Kubernetes API itself is the product (operators, CRDs)

If two or more of those apply, Kubernetes earns its keep. If none do, you are paying for capabilities you will not use.

When it is not

Most small teams have a workload that looks like this:

1 to 5 services
One Postgres, maybe a Redis
A single region
Fewer than 5 deploys a day

This fits comfortably on one Hetzner box with Docker Compose, or on a PaaS. No Kubernetes needed, much less money spent, and far less time on ops.

The "we will need it eventually" argument is mostly survivorship bias. Most projects never reach the scale where Kubernetes is actually the cheapest option, and migrating later is easier than people claim. A docker-compose.yml maps almost line-for-line to Deployments and Services when the day comes.

Quick decision table

Situation	Use
Solo dev, 1-3 services	Docker Compose on a VPS
Small team, up to ~10 services, one region	PaaS or Compose + Ansible
Multi-tenant SaaS with isolation needs	Kubernetes (managed)
Compliance-heavy, audited infrastructure	Kubernetes (managed)
Building a platform or operator	Kubernetes
"Everyone else uses it"	Not a real reason

The middle ground

A PaaS exists exactly for this gap. You get the useful parts of Kubernetes – self-healing, declarative deploys, automatic HTTPS, isolated namespaces – without running the cluster yourself.

Hostim.dev runs Kubernetes underneath, on bare metal in Germany, so you do not have to. You paste a docker-compose.yml and get a deployed app with HTTPS, Postgres, Redis, volumes, metrics and logs.

The same stack priced on Hostim:

3× shared App (2 vCPU / 2GB): 13.50 €
Postgres (10GB): 10 €
Redis (2.5GB): 5 €
50GB volume: 10 €

Total: 38.50 €/mo, with HTTPS, metrics, logs and backups included.

If you actually need Kubernetes, run Kubernetes. If you are reaching for it because it is the default answer, a PaaS or a single Hetzner box will probably serve you better, for less money and less weekend work.

👉 Try Hostim.dev

Which Database Should You Self-Host? SQLite vs MySQL vs PostgreSQL vs Redis

Pavel — Wed, 08 Apr 2026 16:38:44 +0000

When you're deploying your own app, the database choice matters more than most people think. It affects performance, ops complexity, backups, and how much memory your server needs.

There are four options you'll run into most often: SQLite, MySQL, PostgreSQL, and Redis. They're not all the same kind of database – and that's the point. Here's when each one makes sense.

SQLite – the zero-ops embedded database

Best for: small apps, prototypes, CLIs, single-user tools, edge deployments
Strengths: no server process, single file, zero config, instant setup
Weaknesses: no concurrent writes, no replication, hard to scale past one instance

SQLite is not a server – it's a library that reads and writes a single file. That makes it perfect for apps where simplicity matters more than scale. If your app has one process writing to the database and modest traffic, SQLite will outperform anything else because there's no network round-trip. The moment you need concurrent writes or multiple app replicas, you've outgrown it.

MySQL – the reliable workhorse

Best for: web apps, CMS platforms, CRUD-heavy workloads, WordPress/Laravel stacks
Strengths: fast reads, mature replication, huge ecosystem, low memory footprint
Weaknesses: weaker JSON support, less strict by default, fewer advanced types

MySQL powers a massive chunk of the internet. It's battle-tested, well-documented, and runs well even on small VPS instances. If you're running a standard web app with mostly reads and simple queries, MySQL will serve you well without hogging resources. Just be aware that its default configs are more lenient than PostgreSQL – silent truncations and implicit type casts can bite you.

PostgreSQL – the feature-rich powerhouse

Best for: complex queries, data integrity, JSON workloads, GIS, analytics
Strengths: advanced types (JSONB, arrays, hstore), strong standards compliance, extensions ecosystem
Weaknesses: higher memory usage, more tuning needed, steeper learning curve for ops

PostgreSQL is the database you pick when correctness and flexibility matter. It handles complex joins, window functions, CTEs, and full-text search natively. The extension ecosystem (PostGIS, pg_cron, pgvector) makes it a Swiss army knife. The trade-off: it's hungrier on resources and rewards careful tuning of shared_buffers, work_mem, and connection pooling.

Redis – the in-memory speed layer

Best for: caching, sessions, rate limiting, queues, pub/sub, leaderboards
Strengths: sub-millisecond reads, rich data structures (lists, sets, sorted sets, streams), built-in TTL
Weaknesses: data must fit in RAM, persistence is optional and lossy, not a primary data store

Redis isn't a replacement for a relational database – it's a complement. Use it for things that need to be fast and can tolerate occasional data loss: session tokens, cache layers, job queues. Redis Streams can even replace simple message brokers. Just don't store your source of truth here – if the server restarts between RDB snapshots, recent writes are gone.

Quick comparison

Feature	SQLite	MySQL	PostgreSQL	Redis
Type	Embedded	Relational server	Relational server	In-memory store
Ease of setup	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐
Concurrent writes	❌ Single-writer	✅ Good	✅ Excellent	✅ Very fast
Complex queries	Basic	Good	Excellent	N/A (key-value)
Memory usage	Minimal	Low–moderate	Moderate–high	High (all data in RAM)
Replication	None built-in	Mature	Mature	Built-in
Best self-host size	Single instance	Small–large	Medium–large	Any (as cache layer)
Persistence	Always (file)	Always (disk)	Always (disk)	Optional (RDB/AOF)

So which one should you choose?

Building a prototype or CLI tool? → SQLite
Running a standard web app? → MySQL
Need complex queries, JSONB, or extensions? → PostgreSQL
Need a fast cache, session store, or queue? → Redis

Most real-world apps end up using two: a relational database (MySQL or PostgreSQL) for your data, and Redis for caching and sessions. That's not overkill – it's the right tool for each job.

Self-hosting these databases

Running databases on a VPS means managing backups, updates, and disk space yourself. It's doable, but it's one more thing to maintain.

On Hostim.dev, MySQL, PostgreSQL, and Redis are built in – provisioned alongside your app with metrics and no extra config. Paste a docker-compose.yml and your database is ready.

Bastion Host & GitHub Actions on Hostim.dev

Pavel — Thu, 08 Jan 2026 19:00:09 +0000

I haven't posted updates for a while, but several core features landed on Hostim.dev recently.

Instead of shipping from a fixed roadmap, I'm following support-driven (customer-driven) development: features move to the top of the queue once users actively need them.

Over the past month, this resulted in three practical additions around Docker CI/CD, GitHub Actions deployment, and secure bastion host access.

GitHub Actions deploy for Docker apps

Hostim.dev now supports GitHub Actions deployments out of the box.

You can trigger a deploy directly from GitHub Actions using a simple API call. This works well for common Docker CI/CD setups:

Build and deploy on merge to main
Restart an app after pushing a new Docker image
Manual deploys via workflow_dispatch

There's no OAuth and no hidden logic. Your workflow controls everything — branches, conditions, environments. Hostim only executes the requested action.

This is especially useful if you already run CI/CD with Docker and just want a clean deployment target.

Bastion host for secure shell access to containers

Each project now includes a built-in SSH bastion host.

If you're unfamiliar: a bastion host is a hardened entry point used to access private infrastructure without exposing services to the public internet.

On Hostim.dev, the bastion host allows you to open a shell into running apps:

shell my-app

This answers common questions like:

What is a bastion host used for?
How do I securely SSH into containers?
How can I debug a production Docker app without public access?

Typical use cases:

Debugging production issues
Running database migrations
Inspecting environment variables
Accessing internal services safely

The bastion host is private, key-based, and isolated per project.

Custom commands for Docker apps

Apps can now override the container command.

This enables common Docker patterns such as:

One image, multiple roles (web + worker)
Background jobs using the same Docker image
CI/CD pipelines that reuse images across environments

This pairs naturally with Docker CI/CD pipelines, where images are built once and reused consistently.

Why this approach

Many platforms ship features based on assumptions.

Instead, these changes came directly from:

"How do I deploy with GitHub Actions?"
"How do I get shell access without exposing ports?"
"How do I run workers with the same image?"

Support questions shape the roadmap.

What's next

More items are planned, but user feedback decides the order.

If something feels missing, it's probably already on the list.

👉 https://hostim.dev

A Better Umami Dashboard with Grafana

Pavel — Thu, 20 Nov 2025 17:24:38 +0000

Umami is great. Lightweight, privacy-friendly, no cookies, no tracking drama.
We use it ourselves on Hostim.dev, and we ship a one-click Umami template for anyone who wants simple, privacy-focused analytics.

But once you start relying on analytics to make actual decisions, you hit the limits pretty quickly.

Why the default Umami dashboard wasn't enough

Umami intentionally keeps things minimal, but some gaps become obvious:

No clear view of when visitors peak during the day
Hard to isolate bots from real traffic
No moving averages or trend smoothing
No grouped referrers (e.g. "search", "LLM", "other")
Limited visibility into relationships between sessions and custom events

None of this is criticism — Umami is intentionally simple.
But sometimes you want more resolution.

So I took the quickest path:

Deploy Grafana → connect it to Umami's PostgreSQL → build a custom dashboard.

This took maybe ten minutes and unlocked:

Daily heatmap showing real traffic peaks
7-day moving averages for referrers
Qualified sessions (≥2 pageviews) to filter out most bots
Selectable custom events
Raw stats for the selected period

Suddenly Umami became "actionable" instead of just "nice".

How to connect Grafana to Umami's PostgreSQL

Inside Grafana:

Configuration → Data sources → Add data source → PostgreSQL

Fill in the credentials from your Umami database and save.

You can now import our dashboard:

👉 Grafana.com Dashboard

Try it yourself

If you prefer to self-host on your own VPS, here is a complete Docker Compose stack for Umami, PostgreSQL, and Grafana.

Full Docker Compose stack

services:
  postgres:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_USER: umami
      POSTGRES_PASSWORD: umami_pass
      POSTGRES_DB: umami
    volumes:
      - postgres_data:/var/lib/postgresql/data

  umami:
    image: ghcr.io/umami-software/umami:postgres-latest
    restart: always
    depends_on:
      - postgres
    environment:
      DATABASE_URL: postgres://umami:umami_pass@postgres:5432/umami
      DATABASE_TYPE: postgresql
      APP_SECRET: "replace_this_with_a_random_secret"
    ports:
      - "127.0.0.1:3000:3000"

  grafana:
    image: grafana/grafana:latest
    restart: always
    depends_on:
      - postgres
    environment: GF_SERVER_DOMAIN=grafana.example.com
      GF_SERVER_ROOT_URL=https://grafana.example.com
    ports:
      - "127.0.0.1:3001:3000"
    volumes:
      - grafana_data:/var/lib/grafana

volumes:
  postgres_data:
  grafana_data:

Start everything:

docker compose up -d

Then:

Umami → http://localhost:3000
Grafana → http://localhost:3001

In Grafana, configure a PostgreSQL data source:

Host: postgres
Port: 5432
User: umami
Password: umami_pass
Database: umami

Import the dashboard using the ID from Grafana.com.

Don't want to run a server?

If you don't want to manage Docker, OS maintenance, or networking, you can deploy the same stack on Hostim.dev by simply pasting the Compose file above when creating a new project.

Choose Paste Docker Compose
Use the YAML from this section

Hostim.dev will handle HTTPS, internal networking, logs, metrics, and persistence for all three services.

👉 Try Hostim.dev — deploy the full stack without touching SSH

Already running Umami on Hostim.dev?

If you deployed Umami using the one-click template, you don't need a new project. Just:

Create a separate Grafana App in the same project
Use the grafana/grafana:latest image
Add the existing Umami PostgreSQL as a Grafana data source
Import the dashboard JSON

Both apps run on the same private project network, so they can communicate without exposing ports or adjusting firewall rules.

MetalLB on Hetzner Dedicated with vSwitch

Pavel — Tue, 28 Oct 2025 17:00:41 +0000

When running Kubernetes on Hetzner Dedicated, there is no cloud load balancer. But you can provide public LoadBalancer IPs by attaching a routed IP range to a vSwitch and letting MetalLB announce addresses over L2.

Our setup:

Calico (VXLAN + WireGuard)
kube-proxy IPVS with strictARP
ingress-nginx for ingress traffic

1. Assign a public subnet to your vSwitch

Example routed block Hetzner provides:

Subnet:     123.45.67.32/29
Gateway:    123.45.67.33
Usable:     123.45.67.34–38
Broadcast:  123.45.67.39

Attach your dedicated servers to the vSwitch (VLAN ID e.g. 4000).

2. Configure vSwitch VLAN on each node

Each node gets a /32 from the subnet – Hetzner routes the whole /29 to your server.

Important note on routing table IDs

This guide uses routing table 200 as an example.

If you are running Cilium, avoid table 200: Cilium currently flushes all routes in table 200 on startup, which breaks vSwitch routing.

For Cilium-based installations, any other unused routing table ID works (for example 201, 300, or 1001).

Reference: https://github.com/cilium/cilium/issues/38531
Create /etc/netplan/10-vlan-4000.yaml:

network:
  version: 2
  renderer: networkd
  vlans:
    vlan4000:
      id: 4000
      link: eno1
      mtu: 1400
      addresses:
        - 123.45.67.38/32 # node-specific
      routes:
        - to: 0.0.0.0/0
          via: 123.45.67.33
          on-link: true
          table: 200
        - to: 123.45.67.32/29
          scope: link
          table: 200
      routing-policy:
        - from: 123.45.67.32/29
          table: 200
          priority: 10
        - to: 123.45.67.32/29
          table: 200
          priority: 10
        - from: 123.45.67.32/29
          to: 10.233.0.0/18
          table: 254
          priority: 0
        - from: 123.45.67.32/29
          to: 10.233.64.0/18
          table: 254
          priority: 0

Apply:

netplan apply

3. Required sysctl settings

Create /etc/sysctl.d/999-metallb.conf:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0

Why:

Setting	Purpose
`arp_ignore=1`	Only reply to ARP queries for an IP on the correct interface – prevents conflicting replies from Calico/VXLAN.
`arp_announce=2`	Send ARP only from the interface that owns the VIP, required when MetalLB moves VIPs between nodes.
`rp_filter=0`	Disable strict reverse-path filtering – otherwise nodes drop return traffic sourced from VIPs or remote pods.

4. kube-proxy + Calico adjustments

Enable strictARP in kube-proxy (IPVS mode):

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
ipvs:
  strictARP: true

MTU must account for VXLAN + vSwitch + WireGuard overhead:

Calico MTU: 1280 (consistent across nodes)

5. Deploy MetalLB

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: vswitch
  namespace: metallb-system
spec:
  addresses:
    - 123.45.67.34-123.45.67.36 # free VIPs
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2
  namespace: metallb-system
spec:
  ipAddressPools: ["vswitch"]
  interfaces: ["vlan4000"]

Restart MetalLB speakers to pick up interface binding.

6. Ingress service configuration

For ingress-nginx:

spec:
  externalTrafficPolicy: Local

Pro:

Preserves client IP
Prevents traffic hairpin across nodes

Tradeoff:

Only one node handles a given connection (acceptable for ingress)

7. Verification

Confirm that your ingress-nginx Service received a public VIP:

kubectl get svc -n ingress-nginx ingress-nginx-controller

Expected example:

NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   10.233.53.156   123.45.67.35   80:30440/TCP,443:30477/TCP   17d

Inspect the Service events to see which node currently advertises the VIP:

kubectl describe svc -n ingress-nginx ingress-nginx-controller

Look for:

Events:
  Normal  nodeAssigned  ...  metallb-speaker  announcing from node "control-plane-1" with protocol "layer2"

Then verify reachability from outside:

curl -I http://123.45.67.35

Failover test

Identify the active announcer from the above events
Shut that node down abruptly:

sudo poweroff

Re-run:

curl -I http://123.45.67.35

Expected: traffic continues within ~1–2 seconds as another node picks up the VIP.

➡️ Note: VIPs do not appear in ip addr on nodes; they are held in IPVS and advertised via ARP. That is normal.

Acknowledgment

Thanks to Oleksandr Vorona (DevOps at Dysnix) for reporting a Cilium routing table conflict and helping improve this guide.

https://dysnix.com

Wrapping up

This gives:

Public LoadBalancer IPs
Fast failover (~1-2s)
Clean separation: pod networking via VXLAN/WireGuard, external via vSwitch

Alternative approaches:

Hetzner Cloud Load Balancer (simpler, works with Dedicated too)
Cilium with L2 announcements

We run hosting infrastructure, so controlling ingress networking ourselves matters (mostly to prove the point really). Hetzner still runs the vSwitch underneath, but it's more independent than relying on the cloud LB.

And if you'd rather not handle any of this yourself – Hostim.dev is now live. You can deploy your Docker or Compose apps with built-in databases, volumes, HTTPS, and logs – all in one place, ready in minutes.

How to Self-Host n8n with Docker Compose

Pavel — Sat, 04 Oct 2025 12:23:08 +0000

n8n is a popular open-source automation tool – like Zapier, but self-hosted.
Here's how to run it on your own VPS using Docker Compose, and expose it securely over HTTPS using Caddy as the ingress proxy.

If you want a broader primer, check out How to Self-Host a Docker Compose App. But you don't need to read it first – this guide is self-contained.

1. Install Docker on your VPS

Update the system and install Docker + Compose plugin:

apt update && apt upgrade -y
apt-get install ca-certificates curl -y
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" \
  | tee /etc/apt/sources.list.d/docker.list > /dev/null

apt-get update
apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y

2. Write a Docker Compose file

Create a new directory for n8n and add docker-compose.yml:

services:
  n8n:
    image: n8nio/n8n:latest
    restart: always
    ports:
      - "127.0.0.1:5678:5678"
    environment:
      - N8N_HOST=n8n.example.com
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
    volumes:
      - n8n_data:/home/node/.n8n

volumes:
  n8n_data:

💡 Note: This guide assumes you already have a domain (like n8n.example.com) pointing to your VPS's IP address. If not, set that up with your DNS provider before continuing.

Notice we bind to 127.0.0.1:5678 – so it's only accessible locally. Caddy will handle public access.

Start it:

docker compose up -d

3. Make it survive reboots

Create a systemd service:

# /etc/systemd/system/n8n.service
[Unit]
Description=n8n workflow automation (Docker Compose)
After=network.target

[Service]
Type=oneshot
WorkingDirectory=/root/n8n
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Enable it:

systemctl enable n8n
systemctl start n8n

Now n8n restarts automatically after reboots.

4. Install and configure Caddy

Caddy is a modern reverse proxy with automatic HTTPS. Perfect for small setups.

If you're curious about how Caddy compares to Nginx, HAProxy, or Traefik, see The Reverse Proxy Showdown. For n8n, Caddy is the simplest choice.

Make sure your domain (e.g. n8n.example.com) already resolves to your VPS before you configure Caddy. Otherwise, Let's Encrypt won't be able to issue a certificate.

apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | tee /etc/apt/trusted.gpg.d/caddy-stable.asc
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
apt update
apt install caddy -y

Edit the Caddyfile:

nano /etc/caddy/Caddyfile

Add:

n8n.example.com {
    reverse_proxy localhost:5678
}

Reload Caddy:

systemctl reload caddy

That's it – Caddy requests and renews Let's Encrypt certificates automatically.

5. Secure access

Use strong credentials when creating first user.
Keep the docker-compose.yml volume so your workflows persist.
Optionally, restrict access to your IP range using Caddy if it's just for personal use.

Wrapping Up

You now have a self-hosted n8n instance:

Running in Docker Compose
Restarting automatically after reboots
Exposed via Caddy with HTTPS

If you want to avoid managing servers altogether, platforms like Hostim.dev let you paste a Compose file and get HTTPS, metrics, and persistence without touching SSH.

But if you prefer DIY – this setup will take you far.

The Reverse Proxy Showdown: Nginx vs HAProxy vs Caddy vs Traefik

Pavel — Tue, 16 Sep 2025 13:26:48 +0000

Reverse proxies are the unsung heroes of modern infrastructure. They terminate TLS, route traffic, balance loads, and keep your apps reachable. But which one should you choose?
There are four popular options worth comparing head-to-head: Nginx, HAProxy, Caddy, and Traefik. Each comes with its own strengths, trade-offs, and ideal use cases.

Nginx – the classic all-rounder

Best for: general-purpose web serving, static content, simple reverse proxy setups
Strengths: battle-tested, massive ecosystem, tons of tutorials, easy Certbot integration
Weaknesses: verbose configs, not as dynamic as newer tools

Nginx is often the default choice. It's powerful, stable, and widely documented. If you're setting up a straightforward proxy or serving static files alongside your app, Nginx will feel familiar and reliable. Just be prepared to manage slightly more configuration boilerplate.

👉 Full Nginx reverse proxy guide →

HAProxy – the performance beast

Best for: high-traffic sites, low-latency routing, advanced load balancing
Strengths: blazing fast, robust observability, flexible ACL system
Weaknesses: steeper learning curve, TLS setup can be fiddly

HAProxy is famous for performance. It's a favorite in environments where uptime and throughput matter most. Think enterprise setups or any case where you need fine-grained control over routing logic and health checks. It's less beginner-friendly, but extremely powerful once mastered.

👉 Full HAProxy reverse proxy guide →

Caddy – the modern “batteries included” choice

Best for: minimal config, automatic HTTPS, developer-friendly defaults
Strengths: one-line proxy configs, TLS handled automatically, sane defaults
Weaknesses: smaller ecosystem, fewer advanced knobs for complex routing

Caddy made waves by taking the pain out of HTTPS. With a simple Caddyfile, you get automatic TLS, redirects, and reverse proxying. It's ideal for small projects or developers who want secure, working defaults without fiddling with extra tooling.

👉 Full Caddy reverse proxy guide →

Traefik – the container-native router

Best for: Docker and Kubernetes workloads, dynamic environments
Strengths: integrates with container labels, dynamic service discovery, built-in metrics
Weaknesses: YAML configs can get verbose, less popular outside containerized setups

Traefik was built with cloud-native apps in mind. Instead of editing config files, you annotate containers with labels and Traefik routes traffic automatically. It shines in environments where services come and go frequently, making it a natural fit for orchestrators like Kubernetes.

👉 Full Traefik reverse proxy guide →

Quick comparison

Looking for full setup walkthroughs? Check out our guides for Nginx, HAProxy, Caddy, and Traefik.

Feature	Nginx	HAProxy	Caddy	Traefik
Ease of setup	⭐⭐	⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
Performance	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐
Auto HTTPS	Needs Certbot	Manual + hooks	Built-in	Built-in
Container native	No	No	Somewhat	Yes
Ecosystem/docs	Huge	Mature ops-focused	Growing dev-focused	Strong in Docker/K8s space

So which one should you choose?

Just learning or running a blog? → Nginx
Handling big traffic or need reliability? → HAProxy
Want HTTPS with zero config? → Caddy
Running Docker/Kubernetes? → Traefik

Which reverse proxy do you use in your projects and why?

Netlify's New Credit Pricing: When Cloud Rent Comes Due

Pavel — Mon, 08 Sep 2025 08:33:10 +0000

📌 Last week, I wrote about Cloud Rent in Action – how layers of middlemen drive up the cost of running a simple SaaS stack. Netlify's new pricing update feels like the same story, playing out live.

What Changed at Netlify

Netlify just rolled out a credit-based pricing model.

New accounts are now required to buy credits.
Every deploy, function, or gigabyte of bandwidth consumes those credits.
When the credits run out, your projects pause until you top up.
Legacy users can stay on old plans for now, but the future is clear: credits are the new normal.

On paper, this looks like a simplification. In reality, it's the next stage of cloud rent.

Why Netlify Had to Change

For years, companies like Netlify grew fast thanks to venture capital money. Investors subsidized growth: cheap plans, generous free tiers, and aggressive marketing. The mission was simple – capture the market at any cost.

That was the market expansion phase.

VCs were happy to foot the bill as long as user numbers climbed.

Now we're in the market exploration (or sustainability) phase. Investors want returns. And that means:

Free tiers shrink
Simple flat plans get replaced with credit systems
Costs shift from VC wallets to developer wallets

It's not that Netlify suddenly became greedy – it's that the VC playbook always ends this way. Rent has to be collected. And developers end up paying it.

The Problem With Credit Pricing

Credits sound neat – one bucket, one metric. But for most developers, they create more problems than they solve:

Mental overhead – you're forced to budget not just money, but deploys and requests.
Unpredictable bills – a sudden spike in traffic can drain credits overnight.
Complexity creep – hosting a static site shouldn't require a calculator.

This is exactly the dynamic I wrote about in my Cloud Rent post: when platforms optimize for investor returns instead of developer trust, pricing drifts away from simplicity and fairness.

Why Hostim.dev Is Different

I'm building Hostim.dev with a completely different philosophy:

Bootstrapped, not VC-funded. No investors. No pressure to flip pricing later. No "growth at all costs" phase.
Lean team. Right now it's just me – the founder – building and running the platform. That means lower overhead and no bloated payroll to pass on to you.
Fair pricing from the beginning. Plans are simple, predictable, and surge-safe. No credits, no hidden meters, no surprise bills.
Built for developers, not investors. The focus is on usability and transparency. You don't need to rewire your workflow to fit a platform's billing quirks.

Cloud Rent vs. Developer Trust

So if last week's post was the theory, this week is the proof:

Cloud rent always comes due. Netlify's credits are just the latest example.

At Hostim.dev, I'm building the opposite:

Flat, predictable plans
No surprise charges
Databases, volumes, and apps as first-class citizens
A platform you can trust, built for developers, not for VCs

💬 Curious to hear your thoughts:

Do you prefer credit-based pricing models (like Netlify's) or flat, predictable plans?

👉 You can check out Hostim.dev if you're interested – your first project is free for 5 days, no credit card required.

Cloud Rent in Action: How €50 Turns Into €200+

Pavel — Tue, 02 Sep 2025 10:22:16 +0000

When you pay for cloud hosting, you're not just paying for compute.

You're paying rent. And it adds up fast.

🏗️ What You Think You're Paying For

Let's say you need a small SaaS backend:

2 apps (API + worker)
1 Postgres database
1 Redis for caching
A few gigs of storage

Pretty standard stack.

💸 What It Costs on AWS

EC2 (2× t3.medium) → €50 / mo
RDS Postgres (db.t3.small, 10GB) → €22 / mo
ElastiCache Redis (cache.t3.micro) → €10 / mo
EBS storage (100GB) → €10 / mo
Data transfer (200GB egress) → €9 / mo

Total: ~€101 / mo

That’s without backups, monitoring, or any extras.

And without any “friendly” PaaS markup on top.

🏠 What It Costs on Bare Metal

Hetzner: 12 threads, 64GB RAM, 1TB SSD → €44 / mo.

You could run dozens of those same apps + databases on one machine.

But if you don’t want to babysit it, you go through AWS — and suddenly you’re paying 2× more for the same outcome.

Of course, bare metal has risks too (datacenter failures, ops work).

But the price gap is still very real.

🧃 Add a Middleman

Now add a VC-backed PaaS that just resells AWS.

Nice UI, Heroku-like DX… but you’re paying another ×2 markup.

Your ~€100 stack just became €200–250 / mo.

That’s cloud rent: the difference between the infra you’re actually using and the layers of middlemen you’re forced to pay.

🌱 A Fairer Alternative

Here’s the same stack — 2 apps, Postgres, Redis, and a 50GB volume — running on my own PaaS, Hostim.dev:

€34 / mo.
Includes HTTPS, metrics, and logs baked in.
No metering, no hidden costs.

You still get the convenience of a PaaS.

But without subsidizing investors, shareholders, or cloud landlords.

Just me, your humble wannabe hoster.

🚀 Try It Yourself

I recently shipped authless trials:

paste a docker-compose.yml, and you’ll see your app running in seconds — no signup needed.

👉 Try Hostim.dev for free

How We Built a PaaS with Go, Kubernetes, and React

Pavel — Tue, 26 Aug 2025 13:07:09 +0000

Building a PaaS as a solo founder means making choices. Some deliberate, some accidental, all of them tradeoffs.

Every tool has tradeoffs, and the deciding factor is usually the most expensive resource of all: time.

If I can get the job done with something I already know, I'll take that path. I'll learn new tools when the project pays for it. Until then, it's all about moving forward with what works.

Here's how Hostim.dev is put together today – the stack that runs every app, database, and service behind the scenes.

🖥 Infra: Ansible + Kubespray

Before Kubernetes even comes into the picture, there's infrastructure to manage.

I've spent six years working with Ansible, so it was my first pick. Hostim.dev runs on bare metal servers – and the Kubernetes clusters on top of them are provisioned with Kubespray, which itself is a set of Ansible playbooks.

That means everything integrates nicely:

My own playbooks handle server lifecycle (deploy new users, rotate keys, manage credentials).
Kubespray handles cluster lifecycle (deploy, upgrade, or scale clusters).

Could Terraform do this job too? Maybe. But I'd spend more time learning it than deploying clusters. That's the tradeoff.

The upside: flexibility. I can run only the parts I need, when I need them.

The downside: it's not centralized – I run playbooks from my own machine. If two people apply different changes at the same time, you could hit conflicts. For now, that's a human problem, and we'll solve it in a human way.

⚙️ The Kubernetes Operator

The heart of the platform is a custom operator written in Go with Kubebuilder.

If you're not familiar: an operator is basically a program that runs in the cluster and ensures the "desired state" matches the "actual state."

Examples:

Update an app's environment variables → operator notices, triggers a restart.
Create a new database → operator picks a server, provisions it, updates permissions, applies migrations.
Scale an app → operator reconciles replicas until the cluster matches your request.

It also emits the events you see in the dashboard. The backend subscribes to them, stores them, and triggers UI updates so what you see is always fresh.

This piece does a lot – from managing app lifecycles to handling Redis and Postgres placements – and is probably the best candidate for open-sourcing later on. No fixed timeline yet, but it's on my mind.

🔗 Backend API: Schema First

All the business logic sits in the backend. It's written in Go, with:

Gin for the HTTP server
Ent as the ORM
oapi-codegen to generate code from an OpenAPI schema

The workflow looks like this:

Write the OpenAPI schema first.
Generate the server interfaces with oapi-codegen.
Implement the interfaces manually.
Wire them up with Ent models and Kubernetes operator objects.

It's not 100% smooth (Ent and oapi-codegen don't integrate perfectly, so there's some type conversion glue). But overall, it means less boilerplate and more consistency.

At the end of the day, three parts come together:

K8s objects (via the operator's Go package)
Database code (via Ent)
HTTP API (via oapi-codegen)

My job: glue them together and add business logic. Which is exactly what Hostim.dev runs on today.

🎨 Frontend: React + Ant Design

This is where I had the least experience. A good friend helped me bootstrap the project, and I leaned on LLMs for some of the early decisions.

Framework of choice: React + TypeScript.

UI library: Ant Design (Antd) – suggested by an LLM, picked mostly on a gut call. It does the job.

Could I have picked Svelte or Vue or "framework XYZ" instead? Sure. But I had a friend to guide me through React, not those other frameworks. And that meant I could start shipping right away. That's the tradeoff.

What I'm happy about is how code generation carries through to the frontend. The OpenAPI client is autogenerated, so the frontend just calls strongly typed functions.

If I change an object in the backend and re-generate, any breaking changes are immediately visible in the IDE. That feedback loop saved me a ton of time and bugs.

Wrapping Up

So that's the stack:

Ansible + Kubespray for infra
Go + Kubebuilder operator for apps, DBs, Redis, volumes
Go + Gin + Ent + OpenAPI for backend
React + Ant Design for frontend

It's not perfect. Every layer has its tradeoffs. Some tools might be "hotter" or "easier," but experience and context matter more. In the end, the real problem to solve isn't "which framework is coolest" – it's time.

That's true for me building the platform, and it's true for anyone using Hostim.dev instead of wiring up VPS configs or AWS bills.

Think of it like this: you can choose Terraform vs. Ansible, React vs. Vue… and you can also choose "Do I spend Saturday night fixing SSL, or do I just deploy and move on?" 😅

👉 Curious? You can join the beta here.

What about you?

What tradeoffs have you made recently – stuck with a familiar tool, or jumped to something shiny?

From VPS to PaaS: Why I Stopped Managing Servers

Pavel — Tue, 19 Aug 2025 12:39:06 +0000

Most side projects start the same way.

You grab a VPS from Hetzner or DigitalOcean, install Docker, run docker compose up, and boom – you’re live.

It feels cheap. It feels simple.

Until it isn’t.

The VPS Path: The Default Way

Here’s the typical journey I went through (and many devs still do):

Rent a VPS for €5–€10/month
Install Docker + Docker Compose
Run the app
Add Nginx and Let’s Encrypt for HTTPS
Hack together a systemd unit so it restarts after reboot
Manually configure backups, logs, and monitoring

It works. But every new project means repeating the same steps.

And every time, something goes wrong – ports left open, SSL renewal fails, or a config breaks after an update.

Well, unless you properly automate it with something like Ansible or Terraform.

But let’s be honest: do you really want to learn and maintain infra-as-code pipelines… just for side projects?

The Hidden Costs of “Cheap” VPS Hosting

At first glance, VPS looks cheap.

But the costs sneak up on you:

Backups: €2–€5/month
Monitoring/logs: another €5–€10/month or DIY time
Downtime: hours spent debugging instead of coding
Security: one misconfigured firewall can expose your database

The real cost isn’t just money.

It’s time lost repeating setup, patching servers, and fixing mistakes.

And if you’re billing clients? That “cheap” VPS suddenly isn’t cheap anymore.

The PaaS Alternative

A PaaS (Platform-as-a-Service) takes that whole messy checklist and bakes it in:

Deploy directly from Docker, Git, or Compose
Automatic HTTPS and domain management
Built-in databases like Postgres, MySQL, and Redis
Volumes that survive restarts and redeploys
Metrics and logs out of the box
Per-project isolation so one client doesn’t mess up another

Instead of spending hours setting up a VPS, you paste your Compose file or point to a repo, click deploy, and it just works.

Why I Switched (and Why I Built Hostim.dev)

After doing the VPS setup dozens of times – for my own apps, side projects, and client work – I finally hit a wall.

Every project felt like déjà vu.

Spin up server, fight configs, add SSL, fix logging, repeat.

So I built something that skips all of that.

Hostim.dev is a developer-first PaaS.

You paste your docker-compose.yml, or deploy from Git or Docker Hub, and you’re live with HTTPS, metrics, databases, and volumes.

No YAML rewrites. No hidden cloud costs.

Just deploy and move on.

Wrapping Up

VPS hosting isn’t bad. It’s still a good choice if you want full control or you enjoy tweaking configs.

But if you’d rather spend time building apps instead of babysitting servers, a PaaS can save you both money and frustration.

And yes – I’ll still be babysitting servers.

But that’s my job now, not yours. 😉

👉 Hostim.dev is opening soon with a free trial and always-free database tiers.

If you’re tired of fighting servers, join the waitlist – and let’s bring hosting back to earth.