DEV Community

Cover image for Raspberry Pi Kubernetes Cluster: Start the Cluster (Issue #4)
Jules
Jules

Posted on • Originally published at juliantellez.com

Raspberry Pi Kubernetes Cluster: Start the Cluster (Issue #4)

#kubernetes #homelab #raspberrypi

Welcome to the Raspberry Pi Kubernetes home lab series. A bite sized informative guide to help you provision a cluster from scratch. By now we should have 3+ nodes running the latest version of Ubuntu and we should be able to SSH into them.

Today's issue will walk you through installing Kubernetes into your newly created cluster.

1. Requirements

  • At least 3 nodes with Ubuntu installed on them

The following 12 steps should be replicated on every node. We will set them up manually in this lab session, but will provide an automated way of setting up nodes via Ansible in a coming issue.

2. SSH into the node 

ssh ubuntu@192.168.0.25
Enter fullscreen mode Exit fullscreen mode

3. Set up hostnames and static IPs

Slave nodes need to provide a unique name to the master node upon registration.

# On each node
NODE_NAME=k8s-master # k8s-node-1 etc..
sudo hostnamectl set-hostname $NODE_NAME
Enter fullscreen mode Exit fullscreen mode

4. Edit the hosts file so it maps IP addresses to hostnames 

sudo nano /etc/hosts
192.168.0.25 k8s-master
192.168.0.26 k8s-node-1
192.168.0.24 k8s-node-2
Enter fullscreen mode Exit fullscreen mode

5. Disable swap memory

Swap is space on the host disk that is used when the amount of physical RAM is full. Even though swap support has been added to Kubernetes, our Raspberry Pis have limited resources and we should account for such limitations.

sudo swapoff -a
sudo apt purge -y dphys-swapfile
# Test swap has been disabled
free -m
Enter fullscreen mode Exit fullscreen mode

6. Edit the boot configuration

Append the following config to the boot file.

sudo nano /boot/firmware/cmdline.txt
ipv6.disable=1 cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1
Enter fullscreen mode Exit fullscreen mode

7. Set up a container runtime (CRI)

Kubernetes has moved away from Docker. There are a few options out there (Docker included). We will be setting up containerd as our CRI.

sudo apt update -y && sudo apt dist-upgrade -y
sudo apt install -y containerd containernetworking-plugins
Enter fullscreen mode Exit fullscreen mode

8. Create a config for containerd 

sudo mkdir /etc/containerd
cat <<EOF | sudo tee /etc/containerd/config.toml
version = 2
[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".containerd]
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          runtime_type = "io.containerd.runc.v2"
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            SystemdCgroup = true
EOF
Enter fullscreen mode Exit fullscreen mode

9. Forward IPv4 and set up IP tables 

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system
Enter fullscreen mode Exit fullscreen mode

10. Install kubeadm

Update the apt package index and install packages needed to use the Kubernetes apt repository:

sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl
Enter fullscreen mode Exit fullscreen mode

Download the Google Cloud public signing key:

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
Enter fullscreen mode Exit fullscreen mode

Add the Kubernetes apt repository:

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
Enter fullscreen mode Exit fullscreen mode

Update the apt package index, install kubelet, kubeadm and kubectl, and pin their version:

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
Enter fullscreen mode Exit fullscreen mode

11. Install a Container Network Interface (CNI)

A CNI is required to implement the Kubernetes network model. We are going to install Flannel in this lab as it is compatible with MetalLB. Be sure to check out different networking models too, such as Calico.

wget https://github.com/flannel-io/flannel/releases/download/v0.19.2/flanneld-arm64
sudo chmod +x flanneld-arm64
sudo cp flanneld-arm64 /usr/local/bin/flanneld
sudo mkdir -p /var/lib/k8s/flannel/networks
Enter fullscreen mode Exit fullscreen mode

We will install Flannel and apply its configuration once the master node is running.

12. Reboot (optional)

Reboot the nodes and start fresh. It will ensure that the boot file from step 6 is loaded in.

sudo reboot
Enter fullscreen mode Exit fullscreen mode

Start the cluster

SSH into the master node and run:

sudo kubeadm config images pull
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --token-ttl=0
Enter fullscreen mode Exit fullscreen mode

Save the token and discovery token provided by the initialisation. You can always generate a new token later by doing:

KUBE_TOKEN=$(kubeadm token generate)
kubeadm token create $KUBE_TOKEN --print-join-command --ttl=0
Enter fullscreen mode Exit fullscreen mode

Create the kube config 

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Enter fullscreen mode Exit fullscreen mode

Node verification 

kubectl get nodes
NAME       STATUS   ROLES  AGE VERSION
k8s-master NotReady master 55m v1.25.1
Enter fullscreen mode Exit fullscreen mode

For the node to become ready we need to install our CNI from step 11.

Apply the CNI config

Assuming we are connected to our master node from the previous section, download Flannel's configuration:

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Enter fullscreen mode Exit fullscreen mode

Inspect the YAML configuration and replace the backend. More info is in the Flannel backend documentation.

sed -i 's/vxlan/host-gw/' kube-flannel.yml
Enter fullscreen mode Exit fullscreen mode

Apply the config:

kubectl create -f kube-flannel.yml
Enter fullscreen mode Exit fullscreen mode

Add slave nodes to the cluster

SSH into the slave nodes (k8s-node-1, k8s-node-2, ...) and run:

K8S_MASTER_IP=192.168.0.25
sudo kubeadm join $K8S_MASTER_IP:6443 \
--token <token here> \
--discovery-token-ca-cert-hash sha256:<hash value here>
# Output:
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
Enter fullscreen mode Exit fullscreen mode

We are now ready to start inspecting and extending our cluster.

Leave a comment if you have any questions. We will be automating this setup next. Happy coding.

First published in my LinkedIn newsletter, Built from Scratch.

This was originally published on juliantellez.com. It is part of my newsletter on video, streaming, and building reliable systems from scratch: the architecture, the tradeoffs, and the occasional 3am incident.

If it was useful, subscribe for the next one. No spam, unsubscribe anytime. If you are building something similar and want to compare notes, say hello. I read every message.

Built from scratch, like everything here.

Top comments (0)