DEV Community

Ana Villar
Ana Villar

Posted on

Building a Kubernetes Cluster on Red Hat Enterprise Linux 10: A kubeadm Guide

#tutorial #kubernetes #rhel #kubeadm

Introduction

In this post, I'll walk you through deploying a production-ready Kubernetes cluster on Red Hat Enterprise Linux 10 using kubeadm. This lab was inspired by Anthony E. Nocentino's excellent Certified Kubernetes Administrator (CKA): Using kubadm to Install a Basic Cluster training course, which is part of the official Certified Kubernetes Administrator (CKA) path on Pluralsight.

⭐ Shout-out: Anthony is a fantastic trainer! His course uses Ubuntu 22.04 as the base OS. I adapted his approach to work on RHEL 10, adding some additional considerations specific to Red Hat's ecosystem.

One intentional decision in this setup: I deployed Kubernetes v1.35 and CRI-O v1.35, which wasn't the latest version available at installation time.

This was purposeful. Anthony's course includes a dedicated section on upgrading clusters, and using a slightly older baseline makes that learning path clearer.

The upgrade procedures (not covered here) are what really solidify your understanding of cluster lifecycle management.

Lab Infrastructure Overview

Nodes Configuration

Node Role RAM vCPUs IP Address
rh-cp1 Control Plane 12 GiB 2 192.168.110.120
rh-node1 Worker 6 GiB 2 192.168.110.121
rh-node2 Worker 6 GiB 2 192.168.110.122
rh-node3 Worker 6 GiB 2 192.168.110.123

Note: The IP address schema is just an example and what was more convenient for me.

Supporting Infrastructure

A dedicated utilities VM (also RHEL 10) provides essential services:

  • DNS (BIND/named)
  • NTP (chrony)
  • HTTP (Apache/httpd)
  • DHCP (Kea)

This centralized infrastructure simplifies name resolution across all cluster nodes. But this is not essential for this project. You can, instead, ensure the nodes are able to reach each other updating the file /etc/hosts on all nodes.

Prerequisites & OS Preparation

Before diving into Kubernetes, we need consistent node preparation across all machines.

1. System Registration and Updates 

$ sudo subscription-manager register --username <username> --password <password>
$ sudo dnf update redhat-release
$ sudo dnf upgrade
$ sudo reboot
Enter fullscreen mode Exit fullscreen mode

2. Disable Swap (Required by Kubernetes)

Edit /etc/fstab to comment out swap entries:

UUID=xxxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxxxx none swap defaults 0 0
# ^ Comment this line out
Enter fullscreen mode Exit fullscreen mode

Verify:

$ sudo swapoff -a
$ free
Enter fullscreen mode Exit fullscreen mode

3. Disable Firewalld

Disable firewalld, as indicated in the Calico System requirements for Kubernetes:

$ sudo systemctl stop firewalld
$ sudo systemctl disable firewalld
$ sudo systemctl mask firewalld
Enter fullscreen mode Exit fullscreen mode

⚠️ Production Note: Use Calico to maintaining security and enforce network policies later.

4. Load Kernel Modules and Enable IP Forwarding 

$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
Enter fullscreen mode Exit fullscreen mode 
$ sudo modprobe overlay
$ sudo modprobe br_netfilter
Enter fullscreen mode Exit fullscreen mode

Configure sysctl parameters:

$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
Enter fullscreen mode Exit fullscreen mode 
$ sudo sysctl --system
Enter fullscreen mode Exit fullscreen mode

Verify the modules loaded correctly:

$ lsmod | grep overlay
$ lsmod | grep br_netfilter
Enter fullscreen mode Exit fullscreen mode

Installing Kubernetes Components

Setting Version Variables 

$ KUBERNETES_VERSION=v1.35
$ CRIO_VERSION=v1.35
Enter fullscreen mode Exit fullscreen mode

Adding Repositories

Create the Kubernetes repo:

$ cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/repodata/repomd.xml.key
EOF
Enter fullscreen mode Exit fullscreen mode

Create the CRI-O repo:

$ cat <<EOF | sudo tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/repodata/repomd.xml.key
EOF
Enter fullscreen mode Exit fullscreen mode

Installing Packages 

$ sudo dnf install -y kubelet kubeadm kubectl cri-o container-selinux
Enter fullscreen mode Exit fullscreen mode

Configuring CRI-O

Create a cgroup manager configuration file:

$ cat <<EOF | sudo tee /etc/crio/crio.conf.d/02-cgroup-manager.conf
[crio.runtime]
conmon_cgroup = "pod"
cgroup_manager = "cgroupfs"
EOF
Enter fullscreen mode Exit fullscreen mode

Enable and start services:

$ sudo systemctl enable --now crio kubelet
$ sudo systemctl restart crio
Enter fullscreen mode Exit fullscreen mode

Version Locking

To prevent accidental upgrades:

$ sudo dnf install 'dnf-command(versionlock)'
$ sudo dnf versionlock add kubeadm-1.35.4 kubelet-1.35.4 kubectl-1.35.4 cri-o-1.35.2
Enter fullscreen mode Exit fullscreen mode

Note: Review the output from the installation of the packages kubeadm, kubelet, kubectl and cri-o, and update the versions to lock in the command above.

Initializing the Control Plane

On rh-cp1, download and configure Calico networking. To know the current lastest version, check Tigera documentation, in the Manifest tab:

$ wget https://raw.githubusercontent.com/projectcalico/calico/v3.32.0/manifests/calico.yaml
Enter fullscreen mode Exit fullscreen mode

Edit the CALICO_IPV4POOL_CIDR value to match your pod network plan:

- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"
Enter fullscreen mode Exit fullscreen mode

Initialize the cluster, to use the same subnet:

$ sudo kubeadm init \
  --kubernetes-version v1.35.4 \
  --pod-network-cidr=10.244.0.0/16 \
  --cri-socket unix:///var/run/crio/crio.sock \
  --upload-certs
Enter fullscreen mode Exit fullscreen mode

Once successful, save the join commands that appear at the end of the output—you'll need these for worker nodes!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.110.120:6443 --token xxxxxx.xxxxxxxxxxxxxxxx \
 --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  
$
Enter fullscreen mode Exit fullscreen mode

Your Kubernetes control-plane has initialized successfully!

Configuring kubectl 

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
Enter fullscreen mode Exit fullscreen mode

Deploy Calico Network 

$ kubectl apply -f calico.yaml
Enter fullscreen mode Exit fullscreen mode

Wait a few minutes and verify pods are running:

$ kubectl get pods --all-namespaces
$ kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

Expected output:

$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-6b4b6457d5-p98c2   1/1     Running   0          2m47s
kube-system   calico-node-5vjrr                          1/1     Running   0          2m47s
kube-system   coredns-7d764666f9-r4vqp                   1/1     Running   0          4m9s
kube-system   coredns-7d764666f9-vh7df                   1/1     Running   0          4m9s
kube-system   etcd-rh-cp1                                1/1     Running   0          4m28s
kube-system   kube-apiserver-rh-cp1                      1/1     Running   0          4m28s
kube-system   kube-controller-manager-rh-cp1             1/1     Running   0          4m27s
kube-system   kube-proxy-4r6h8                           1/1     Running   0          4m10s
kube-system   kube-scheduler-rh-cp1                      1/1     Running   0          4m28s
$ kubectl get nodes
NAME     STATUS   ROLES           AGE     VERSION
rh-cp1   Ready    control-plane   4m40s   v1.35.4
$
Enter fullscreen mode Exit fullscreen mode

Joining Worker Nodes

On each worker node (rh-node1, rh-node2, rh-node3), run the join command saved during kubeadm init:

$ sudo kubeadm join 192.168.110.120:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash>
Enter fullscreen mode Exit fullscreen mode

Verify the cluster health from the control plane:

$ kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

All nodes should show Ready status.

$ kubectl get nodes
NAME       STATUS   ROLES           AGE     VERSION
rh-cp1     Ready    control-plane   10m     v1.35.4
rh-node1   Ready    <none>          2m11s   v1.35.4
rh-node2   Ready    <none>          113s    v1.35.4
rh-node3   Ready    <none>          102s    v1.35.4
$
Enter fullscreen mode Exit fullscreen mode

Install bash completion for kubectl: 

$ sudo dnf install bash-completion
$ echo "source <(kubectl completion bash)" >> ~/.bashrc
$ source ~/.bashrc
Enter fullscreen mode Exit fullscreen mode

Testing the Deployment

Deploy a test application:

$ kubectl create deployment hello-world --image=psk8s.azurecr.io/hello-app:1.0
$ kubectl get pods -o wide
Enter fullscreen mode Exit fullscreen mode

Expose it via a service:

$ kubectl expose deployment hello-world --port=80 --target-port=8080
$ kubectl get service hello-world
Enter fullscreen mode Exit fullscreen mode

For example:

$ kubectl get service hello-world
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
hello-world   ClusterIP   10.99.168.218   <none>        80/TCP    8s
$
Enter fullscreen mode Exit fullscreen mode

Then, test it. You should see a response from your running container!

$ curl http://10.99.168.218:80
Hello, world!
Version: 1.0.0
hello-world-b5b7f67cc-d26dt
$
Enter fullscreen mode Exit fullscreen mode

Clean up after testing:

$ kubectl delete service hello-world
$ kubectl delete deployment hello-world
Enter fullscreen mode Exit fullscreen mode

Key Considerations When Moving from Ubuntu to RHEL

Here are the main differences I encountered adapting Anthony's tutorial:

Aspect Ubuntu Approach RHEL 10 Adaptation
Package Manager apt/dpkg dnf/rpm
Firewall Management ufw/firewalld optional firewalld disabled (use Calico policies)
Subscription N/A subscription-manager required
SELinux Permissive mode default Need to handle SELinux context
CRI Runtime containerd CRI-O

What's Next?

This setup provides a solid foundation for learning Kubernetes administration. From here, you could explore:

  • Cluster upgrades (covered extensively in Anthony's course)
  • Network policy enforcement with Calico
  • High availability with multiple control plane nodes
  • Storage classes and persistent volumes
  • Monitoring stack with Prometheus/Grafana

If you found this walkthrough helpful, I'd highly recommend checking out the original Pluralsight course. Anthony's explanations are crystal clear, and adapting them to different distributions is an excellent way to deepen your understanding of what happens under the hood.

Resources

Pluralsight: Certified Kubernetes Administrator Certification Path
Official Kubernetes Documentation
Calico Project GitHub

Thanks for reading! Feel free to share your own experiences with Kubernetes on RHEL in the comments below.

Top comments (0)