Installing Kubernetes with dual-stack (ipv4/ipv6) networking

Introduction

This post describes how you can install kubernetes with dual-stack (ipv4/ipv6) networking. The target setup is a single node kubernetes cluster on ubuntu 22.04 (Jammy Jellyfish), which is also the latest LTS release of ubuntu, using calico as CNI implementation. This cluster will be bootstrapped using kubeadm.

Changes required for Ubuntu OS

Enabled packet forwarding for both ipv4 and ipv6

• Edit /etc/sysctl.conf file and make sure following lines are uncommented -

  • net.ipv4.ip_forward=1
  • net.ipv6.conf.all.forwarding=1

• Apply changes -
  
  

sudo sysctl -p

Disable Swap

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a

Docker and Containerd

Installing docker, docker cli and containerd

• Use latest instructions available on docker website to install docker, docker cli and containerd.

• (Optional) Pin packages' versions -
  

sudo apt-mark hold docker-ce docker-ce-cli containerd.io

• Create containerd configuration file -

containerd config default > /etc/containerd/config.toml

• Add SystemdCgroup = true line under [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] configuration in the above file.

• Restart containerd -
  

sudo systemctl restart containerd

Kubernetes Setup

Bootstrap the cluster using kubeadm

• Use the instructions here to install kubeadm, kubelet and kubectl.

• Bootstrap cluster using kubeadm init command -
  

sudo kubeadm init --pod-network-cidr=192.168.0.0/16,2001:db8:42:0::/56 --service-cidr=10.96.0.0/16,2001:db8:42:1::/112

Setup Calico on the cluster

• Install the Tigera Calico operator and custom resource definitions. Make sure you replace v3.27.0 in the below command with the latest version of Calico cni implementation -

kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/tigera-operator.yaml

• Download https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/custom-resources.yaml file. Make sure you replace v3.27.0 with the latest version of Calico cni implementation before downloading this file.

• Make following changes in the above file -

  • If you are using pod network cidr which is different from 192.168.0.0/16 then update this value in this file.
  • Make sure ipPools has entry for IPv6 -

ipPools:
    - blockSize: 26
      cidr: 192.168.0.0/16
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()
    - blockSize: 122 
      cidr: 2001:db8:42:0::/56
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()

• Install Calico -

kubectl create -f custom-resources.yaml

• Make sure all the pods are running and in ready state -

watch kubectl get pods --all-namespaces

Setup ingress controller

• Install an ingress controller (e.g. ingress nginx) of your choice.

• Inspect the service associated with your ingress controller using kubectl describe service <ingress-controller-service> and make sure it has both ipv4 and ipv6 IPs assigned. IP addresses shown below may be different depending on the ip ranges selected by you during calico setup -
  

IPs:                      10.96.227.47,2001:db8:42:1::3310

• Also inspect the ingress controller pod using kubectl describe pod <ingress-controller-pod> and make sure it has both types of IPs assigned. IP addresses shown below may be different depending on the ip ranges selected by you during calico setup -

IPs:
  IP:           192.168.218.201
  IP:           2001:db8:42:1d:71c6:f49d:679d:3789