DEV Community

Celso Nery
Celso Nery

Posted on

Building an On-Premise Kubernetes Cluster — Part 2: Installing Containerd and Kubernetes

#kubernetes #k8s #containerd #devops

🇧🇷 Leia a versão em português aqui

In Part 1 of this series, we prepared the environment: defined the hardware, configured /etc/hosts, adjusted the firewall, and disabled SWAP on all nodes. Now that the foundation is ready, it's time to install the container runtime (containerd) and the Kubernetes packages themselves (kubelet and kubeadm).

All the steps below should be run on all servers in the cluster — master and workers — unless stated otherwise.

Loading kernel modules

Kubernetes, through containerd, depends on two Linux kernel modules: overlay (for the layered filesystem used by containers) and br_netfilter (so that bridge network traffic passes through iptables rules).

For these modules to load automatically on every boot, create the file /etc/modules-load.d/containerd.conf:

overlay
br_netfilter
Enter fullscreen mode Exit fullscreen mode

And, to load them immediately (without needing a reboot), run:

$ sudo modprobe overlay
$ sudo modprobe br_netfilter
Enter fullscreen mode Exit fullscreen mode

Adjusting kernel network parameters

Create the file /etc/sysctl.d/99-kubernetes-k8s.conf with the following parameters:

net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
Enter fullscreen mode Exit fullscreen mode

These parameters ensure that network traffic between pods and services is correctly routed and filtered by Kubernetes.

To apply the settings without restarting the server:

$ sudo sysctl --system
Enter fullscreen mode Exit fullscreen mode

Installing containerd

Containerd is the container runtime used by the cluster. In this case, we'll install it through Docker's official repository, using only the containerd.io package (without installing full Docker).

1. Download the repository's GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/docker.gpg
Enter fullscreen mode Exit fullscreen mode

2. Create the repository file at /etc/apt/sources.list.d/docker.list:

deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable
Enter fullscreen mode Exit fullscreen mode

3. Update the package list and install containerd:

sudo apt-get update
sudo apt-get install containerd.io
Enter fullscreen mode Exit fullscreen mode

4. Generate the default containerd config file:

$ sudo containerd config default > /etc/containerd/config.toml
Enter fullscreen mode Exit fullscreen mode

5. Enable the systemd cgroup driver

Edit the file /etc/containerd/config.toml:

sudo vim /etc/containerd/config.toml
Enter fullscreen mode Exit fullscreen mode

And change the SystemdCgroup option from false to true. This adjustment matters because the kubelet, by default, also uses systemd as its cgroup driver — keeping them aligned avoids instability in container resource management.

6. Restart containerd to apply the changes:

sudo systemctl restart containerd
Enter fullscreen mode Exit fullscreen mode

Installing Kubernetes

With containerd running, the next step is installing the Kubernetes components: kubelet and kubeadm.

1. Add the Kubernetes repository GPG key:

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/trusted.gpg.d/kubernetes-gpg-keyring.gpg
Enter fullscreen mode Exit fullscreen mode

2. Add the Kubernetes repository, creating the file /etc/apt/sources.list.d/kubernetes.list:

deb [signed-by=/etc/apt/trusted.gpg.d/kubernetes-gpg-keyring.gpg] http://pkgs.k8s.io/core:/stable:/v1.31/deb/ /
Enter fullscreen mode Exit fullscreen mode

Note: this series uses Kubernetes version 1.31. If you want to install a different version, just swap the v1.31 in the URL above for the desired version.

3. Update the package list:

sudo apt update
Enter fullscreen mode Exit fullscreen mode

4. Install Kubernetes:

sudo apt install kubelet kubeadm
Enter fullscreen mode Exit fullscreen mode

5. Hold the packages' version, to prevent them from being automatically upgraded (which could break the cluster's compatibility):

sudo apt-mark hold kubelet kubeadm
Enter fullscreen mode Exit fullscreen mode

6. Enable the kubelet to auto-start on boot:

sudo systemctl enable kubelet
Enter fullscreen mode Exit fullscreen mode

Limiting image storage

By default, the kubelet can accumulate container images over time, consuming disk space. To prevent this, you can configure image garbage collection limits in the /var/lib/kubelet/kubeadm-flags.env file:

vim /var/lib/kubelet/kubeadm-flags.env
Enter fullscreen mode Exit fullscreen mode

Add the flags:

--image-gc-high-threshold=60 --image-gc-low-threshold=50
Enter fullscreen mode Exit fullscreen mode

This tells the kubelet to start removing unused images once the disk reaches 60% usage, stopping the cleanup once it drops to 50%.

Next steps

With containerd installed and configured with the correct cgroup driver, and with kubelet and kubeadm ready on all nodes, we now have everything needed to initialize the cluster.

In Part 3 of this series, we'll use kubeadm init to create the control-plane on the master server and then join the workers to the cluster.

Continued in Part 3.

Top comments (0)