In this guide, we will walk through the steps to set up a Kubernetes cluster on bare metal servers running Ubuntu. This guide covers the installation of necessary tools, configuration of the master and worker nodes, and setting up network and storage solutions.
Machine Requirements
To set up the Kubernetes cluster, you will need the following machines:
- 1 Master Node on Ubuntu
- 3 Worker Nodes on Ubuntu
- 1 NFS host on Ubuntu
Step 1: Install SSH Server and Net-tools
First, ensure that SSH server and net-tools are installed on all machines. This will allow you to remotely manage the servers and check network configurations.
sudo apt update
sudo apt install openssh-server
sudo systemctl status ssh
sudo apt install net-tools
Step 2: Cluster Volume Setup
Objectives:
- Install NFS Server on a separate host
- Install NFS client on all Kubernetes cluster nodes
- Test if all clients are connected to the NFS host
NFS Installation
i. On NFS Host Machine
Update system packages and install the NFS server:
sudo apt update
sudo apt install nfs-kernel-server
sudo mkdir -p /mnt/nfs_share
sudo chown -R nobody:nogroup /mnt/nfs_share/
sudo chmod 777 /mnt/nfs_share/
sudo vim /etc/exports
ii. Grant Access to Clients
Configure the NFS server to allow access from the client machines:
/mnt/nfs_share <nfs_host_ip>/24(rw,sync,no_subtree_check)
sudo exportfs -a
sudo systemctl restart nfs-kernel-server
sudo ufw allow from <nfs_host_ip>/20 to any port nfs
sudo ufw enable
sudo ufw status
cd /mnt/nfs_share/
touch sample1.text sample2.text
iii. & iv. On NFS Client Machines
Install the NFS client and mount the shared directory:
sudo apt update
sudo apt install nfs-common
sudo mkdir -p /mnt/nfs_clientshare
sudo mount <nfs_host_ip>:/mnt/nfs_share /mnt/nfs_clientshare
ls -l /mnt/nfs_clientshare/
Step 3: Master Node Setup
i. Assign Static IP and Update Machine
If the machine IP is not static, assign a static IP and update the machine:
vim /etc/netplan/00-installer-config.yaml
network:
ethernets:
enp0s3:
dhcp4: true
enp0s8:
addresses: [<master_node_ip>/20]
version: 2
Apply the changes and reboot:
sudo netplan apply
sudo shutdown -r now
sudo -i
sudo apt update
sudo apt -y full-upgrade
[ -f /var/run/reboot-required ] && sudo reboot –f
ii. Install Kubernetes Tools
Install kubelet, kubeadm, and kubectl:
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt -y install vim git curl wget kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
kubectl version --client && kubeadm version
iii. Disable Swap
Disable swap to ensure Kubernetes runs smoothly:
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a
sudo mount -a
free –h
Enable kernel modules and configure IP tables:
sudo modprobe overlay
sudo modprobe br_netfilter
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
iv. Install CRI-O
Install the Container Runtime Interface:
sudo apt update && sudo apt upgrade
sudo systemctl reboot
OS=xUbuntu_20.04
CRIO_VERSION=1.23
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$CRIO_VERSION/$OS/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION.list
curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION/$OS/Release.key | sudo apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -
sudo apt update
sudo apt install cri-o cri-o-runc
sudo systemctl enable crio.service
sudo systemctl start crio.service
systemctl status crio
sudo apt install cri-tools
sudo crictl info
v. Install Podman
Install Podman, an alternative to Docker:
sudo apt-get -y update
sudo apt-get -y install podman
vi. Initialize Master Node
Enable kubelet and pull Kubernetes container images:
sudo lsmod | grep br_netfilter
sudo systemctl enable kubelet
sudo kubeadm config images pull --cri-socket unix:///var/run/crio/crio.sock
vii. Spin Up Master Node Pods
Initialize the master node:
sudo kubeadm init --apiserver-advertise-address=<master_node_ip> --pod-network-cidr=192.168.0.0/16 --cri-socket unix:///var/run/crio/crio.sock --upload-certs --control-plane-endpoint=<master_node_ip> --v=5
Expose the master node pods:
export KUBECONFIG=/etc/kubernetes/admin.conf
viii. Install Network Plugins
Install network plugins using Calico:
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/tigera-operator.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/custom-resources.yaml
ix. Install HELM
Install HELM for managing Kubernetes applications:
snap install helm --classic
x. Install NFS Subdirectory External Provisioner Helm Chart
Install the NFS subdirectory external provisioner:
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server=<nfs_host_ip> --set nfs.path=/mnt/nfs_share
xi. Install and Configure External IP Service
Create a config.yaml file for the external IP service:
vim config.yaml
Add the following configuration (replace xxx with your IP range):
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: my-ip-space
protocol: layer2
addresses:
- xxx.xxx.xxx.210-xxx.xxx.xxx.250
Apply the configuration:
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
kubectl get configmap kube-proxy -n kube-system -o yaml | sed -e "s/strictARP: false/strictARP: true/" | kubectl apply -f - -n kube-system
kubectl apply -f config.yaml
Step 4: Worker Node Setup
Follow steps i to v of the master node setup on each worker node. Then, join the worker nodes to the master node:
kubectl get nodes
kubeadm token generate
kubeadm token create <token> --print-join-command
Run the join command on each worker node:
kubeadm join <master_node_ip>:6443 --token <token> --discovery-token-ca-cert-hash <hash>
Step 5: Install Ingress via HELM
Install the NGINX Ingress controller:
kubectl create namespace ingress-nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.replicaCount=3 \
--set controller.nodeSelector."kubernetes\.io/os"=linux
Step 6: Create Persistent Volume Profiles
Create a Persistent Volume:
vim ClusterPersistentVolume.yaml
Add the following configuration:
apiVersion: v1
kind: PersistentVolume
metadata:
name: dev-pv
labels:
type: local
spec:
storageClassName: nfs-client
capacity:
storage: 50Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /mnt/nfs_share
Apply the configuration:
kubectl get sc
kubectl apply –f ClusterPersistentVolume.yaml
kubectl get pv
Create a Persistent Volume Claim:
vim ClusterPersistentVolumeClaim.yaml
Add the following configuration:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: dev-pvc
spec:
storageClassName: nfs-client
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 25Gi
Apply the configuration:
kubectl create namespace dev
kubectl apply –f ClusterPersistentVolumeClaim.yaml -n dev
kubectl get pvc –n dev
By following these steps, you will have a fully functional Kubernetes cluster running on bare metal servers. This setup provides a robust environment for deploying and managing containerized applications.
Top comments (0)