DEV Community

Celso Nery
Celso Nery

Posted on

On-Premise Kubernetes: Adding a New Node to an Already-Running Cluster

#kubernetes #k8s #kubeadm #devops

🇧🇷 Leia a versão em português aqui.

Once an on-premise Kubernetes cluster is up and running, it's common, sooner or later, to need more available capacity — whether to support more load, or to improve resilience by spreading applications across more workers. In this article, I show the process of adding a new node to a cluster that's already in operation.

This flow is similar to joining workers for the first time, but with some extra care — mainly around machine identity (hostname, SSH keys) — since, in virtualized environments, it's common to clone an existing template machine to speed up provisioning.

Starting from a template

If your environment uses virtualization (VMware, Proxmox, etc.), the fastest path is usually to keep a node template already prepared — with the operating system, containerd, and the Kubernetes packages (kubelet, kubeadm) already installed, following the same steps used on the cluster's original nodes.

To provision a new machine, just clone that template. However, a machine cloned from a template carries certain identities that need to be regenerated before it joins the cluster — otherwise, the new node might conflict with the original node the template was created from.

Regenerating SSH keys

Machines cloned from a template share the same SSH keys as the source host, which poses a security risk (the same SSH identity "exists" on two different machines). Regenerate the keys from scratch:

ssh-keygen -t rsa -f ssh_host_rsa_key
ssh-keygen -t ecdsa -f ssh_host_ecdsa_key
ssh-keygen -t ed25519 -f ssh_host_ed25519_key
Enter fullscreen mode Exit fullscreen mode

These commands should be run in the /etc/ssh/ directory, overwriting the keys inherited from the template.

Adjusting the hostname

Still carrying over the template's identity, the new machine's hostname is likely duplicated from the source. Set a new, unique hostname for this node:

hostnamectl set-hostname <new-name>
Enter fullscreen mode Exit fullscreen mode

Then, add the new node to the /etc/hosts file on all machines in the cluster (master and other workers) — just as we did during the cluster's initial setup —, ensuring every node can resolve the new server's name.

Updating system packages

Before joining the new node to the cluster, it's good practice to make sure it has up-to-date OS packages:

apt update
apt upgrade
Enter fullscreen mode Exit fullscreen mode

Joining the new node to the cluster

With the machine's identity fixed and the system updated, the joining process is the same used to add any worker: generate a valid join command on the master and run it on the new node.

On the master server, generate a new token and the full join command:

kubeadm token create --print-join-command
Enter fullscreen mode Exit fullscreen mode

On the new node, run the returned command (the token and hash values below are just illustrative):

kubeadm join 192.168.121.100:6443 --token r5jtqh.pek7... --discovery-token-ca-cert-hash sha256:518233918ee0353672...
Enter fullscreen mode Exit fullscreen mode

Confirming the integration

Back on the master, check whether the new node shows up in the list and has Ready status:

kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

If the new worker stays NotReady for more than a few minutes, it's worth investigating whether the pod network (CNI) was propagated correctly to it and whether containerd is running without errors on the new machine.

Final thoughts

Adding nodes to an already-running cluster is, in essence, the same process used in the initial setup — the difference lies in the extra care needed when the machine is provisioned from a template or clone, ensuring it has its own identity (SSH and hostname) before joining the cluster's network. Keeping an up-to-date template with containerd and Kubernetes already installed is a practical way to speed up this horizontal scaling process over time.

Top comments (0)