Three machines, a handful of namespaces, and one person who has other work to do — that is the shape of most small clusters. The choice that decides how much of your month Kubernetes eats is not the CNI, the ingress controller, or whether you use Helm or Kustomize. It is who owns the control plane.
We stood up the same stack three ways: k3s on plain Debian VPS nodes, Talos Linux on the same hardware class, and a managed control plane with self-managed workers. The workload was identical each time — a web service, a Postgres StatefulSet on local storage, cert-manager, and an ingress with TLS. All three ran it fine. They differ almost entirely in what happens after day one.
The control plane is the whole decision
Worker nodes are commodity. A kubelet, a container runtime, and a CNI agent are close to interchangeable across every option here, and if a worker dies you drain it, rebuild it, and move on.
The control plane is where small clusters actually break, and the failure modes are boring rather than dramatic:
- Certificate expiry. kubeadm-issued cluster certificates are valid for one year by default and are renewed when you upgrade the control plane. Skip upgrades for 13 months and you get an API server that will not talk to anything. k3s handles this more gracefully — its certificates are also valid for a year, but they rotate automatically on restart once they are within 90 days of expiry.
- etcd disk latency. etcd fsyncs every write. On network-attached storage with variable latency, you get leader elections, slow API responses, and controllers that look broken but are just waiting.
- Upgrade drift. Kubernetes minor releases land roughly three times a year and each is supported for about 14 months. A cluster you forget about for two release cycles is not a cluster you can safely upgrade in one step.
Every option below is a different answer to "who is responsible for those three things."
k3s: the least ceremony
k3s (a CNCF Sandbox project, originally from Rancher) ships Kubernetes as a single binary under 100 MB with containerd, CoreDNS, Flannel, Traefik, a service load balancer, local-path storage, and metrics-server bundled in. Installation is one shell command, and you have a working cluster before the coffee finishes.
The part people miss: k3s replaces etcd with SQLite by default, through a shim called kine. For a single-server cluster that is a genuine simplification — your entire cluster state is one file you can back up with cp. For HA you switch to embedded etcd (three or more servers, odd numbers) or point kine at an external Postgres or MySQL, which is a real option if you already run a managed database.
What k3s does not do is manage the operating system. You still own kernel upgrades, SSH keys, unattended-upgrades, firewall rules, and whatever else accumulates on a long-lived Debian box. That is the whole tradeoff: minimum Kubernetes ceremony, unchanged Linux ceremony.
If you run k3s in HA mode with embedded etcd, put the datastore on local SSD or NVMe, not on network block storage. etcd's write path is fsync-bound, and the symptom of slow storage is not a clear error — it is intermittent leader elections and API calls that hang for seconds. Single-server k3s on SQLite is far more tolerant of slow disks than three-server k3s on etcd, which is a genuinely counterintuitive reason to stay single-node longer than you expect.
Talos: the OS is the API
Talos Linux takes the opposite position — instead of making Kubernetes smaller, it makes the operating system disappear. There is no shell, no SSH daemon, no package manager, and no systemd. The root filesystem is read-only and immutable. You manage the machine entirely over a gRPC API with talosctl, and the machine's configuration is a single YAML document applied at boot.
The upside is concrete rather than philosophical. There is no shell to compromise and no package set to patch, so the attack surface shrinks to the API and the kubelet. Upgrades swap the whole system image and reboot, so a node is either on the new version or it rolled back — there is no half-upgraded state. And because the machine config is one file, node provisioning is genuinely reproducible from git.
The cost is that your Linux muscle memory stops working. You cannot SSH in and tail a log. Debugging a node means talosctl logs, talosctl dmesg, talosctl read, and accepting that anything you cannot express in machine config does not exist on that machine. Teams that already treat nodes as cattle find this liberating. Anyone who habitually fixes production by editing a file in place will find it hostile.
Unlike k3s, Talos runs upstream Kubernetes components — the distribution is the OS, not a repackaged control plane.
Managed control planes: paying to not care
The third option is to buy the control plane and keep the workers. What you get is cert rotation, etcd backups, and version upgrades handled by someone whose job that is, plus an SLA you can point at.
The pricing splits cleanly into two camps at the time of writing. EKS and GKE Standard both bill cluster management at $0.10 per hour — roughly $73 per month per cluster, before a single node. DigitalOcean Kubernetes and Linode Kubernetes Engine offer a free non-HA control plane and charge only for worker nodes, with high-availability control planes as a paid add-on.
That difference decides the argument for small clusters. If your entire workload runs on $40 of compute, a $73 control plane more than doubles the bill and you should be running k3s or Talos. If the free-control-plane providers cover your region and compliance needs, a managed control plane is close to free money — you skip the entire class of problems in the first section.
What you give up is control plane customization. Custom API server flags, admission configuration, and alternate datastores are mostly off the table, and you inherit the provider's upgrade cadence and supported version window. You still own the worker nodes, their OS, and their upgrade schedule — a managed control plane is half the job, not all of it.
A reasonable default: single-node k3s for anything a brief outage would not hurt, Talos for a three-node cluster you intend to keep for years, and a free-tier managed control plane whenever your provider offers one and you would rather spend the time on the application.
Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.
Top comments (0)