Bringing in and updating an initial post written two years ago on my other profile here. These are the defined and authentic steps you can use to run a Three, Five, Seven...etc node cluster on GCP using Linux VMs.
Choose your Linux Flavor, recommended is Ubuntu 22.04 (Jammy) (This will work on local dev boxes and on Cloud Compute VMs with Jammy)
On a Google Cloud Web Console, pick your desired project that has billing enabled and setup your cli tool (Google Cloud CLI) and create a VPC, Subnet and Firewall to allow traffik.
(Replace resource names in square brackets without the brackets):
Create a Virtual Private Cloud Network
gcloud compute networks create [vpc name] --subnet-mode custom
Create a Subnet with a specific range (10.0.96.0/24)
gcloud compute networks subnets create [subnet name] --network [vpc name] --range 10.0.96.0/24
Create Firewall Rule that allows internal communication accros all protocols (10.0.96.0/24, 10.0.92.0/22)
gcloud compute firewall-rules create [internal network name] --allow tcp,udp,icmp --network [vpc name] --source-ranges 110.0.96.0/24, 10.0.92.0/22
Create a firewall rule that allows external SSH, ICMP, and HTTPS:
gcloud compute firewall-rules create [external network name] --allow tcp,icmp --network [vpc name] --source-ranges 0.0.0.0/0
List the firewall rules in the VPC network:
gcloud compute firewall-rules list --filter="network:[vpc name]"
Provision Nodes:
Create 3, 5 or 7 compute instances which will host the Kubernetes Proxy, control plane and worker nodes respectively (A proxy is recommended if you are creating 5 nodes or more):
Proxy Plane Node (Optional):
gcloud compute instances create proxynode --async --boot-disk-size 50GB --can-ip-forward --image-family ubuntu-2204-lts --image-project ubuntu-os-cloud --machine-type n2-standard-2 --private-network-ip 10.0.96.10 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet [subnet name] --tags kubevms-node,proxy
Master Control Plane Node:
gcloud compute instances create masternode --async --boot-disk-size 200GB --can-ip-forward --image-family ubuntu-2204-lts --image-project ubuntu-os-cloud --machine-type n2-standard-2 --private-network-ip 10.0.96.11 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet [subnet name] --tags kubeadm-node,controller
Worker Nodes: (10.0.96.21+ for the other worker nodes)
gcloud compute instances create workernode1 --async --boot-disk-size 100GB --can-ip-forward --image-family ubuntu-2204-lts --image-project ubuntu-os-cloud --machine-type n2-standard-2 --private-network-ip 10.0.96.20 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet [subnet name] --tags kubeadm-node,worker
Print the Internal IP address and Pod CIDR range for each worker node
gcloud compute instances describe workernode1 --format 'value[separator=" "](networkInterfaces[0].networkIP,metadata.items[0].value)'
List the compute instances in your default compute zone:
gcloud compute instances list --filter="tags.items=kubeadm-node"
Test SSH Into Google Cloud VM Instance (You will need to SSH into all the VMs/Nodes to install software)
gcloud compute ssh [compute instance name]
RUN THESE INSTALLATIONS ON ALL NODES
a. sudo -i
b. apt-get update && apt-get upgrade -y
c. apt install curl apt-transport-https vim git wget gnupg2 software-properties-common apt-transport-https ca-certificates uidmap lsb-release -y
d. swapoff -a
INSTALL AND CONFIGURE CONTAINER RUNTIME PREREQUISITES ON ALL NODES
Verify that the br_netfilter module is loaded by running
lsmod | grep br_netfilter
In order for a Linux node's iptables to correctly view bridged traffic, verify that net.bridge.bridge-nf-call-iptables is set to 1 in your sysctl by running the following commands:
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
Apply sysctl params without reboot
sudo sysctl --system
INSTALL CONTAINER RUNTIME ON ALL NODES
a. mkdir -p /etc/apt/keyrings
b. curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
c. echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
d. apt-get update
e. apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
CONFIGURE CGROUP DRIVER FOR CONTAINERD ON ALL NODES (We will use the more advanced SystemD that comes with Ubuntu 2204)
a. stat -fc %T /sys/fs/cgroup/ (Check to see if you are using the supported cgroupV2)
b. sudo containerd config default | sudo tee /etc/containerd/config.toml (Make sure that the config.toml is present with defaults)
c. Set the SystemDGroup = true to use the CGroup driver in the config.toml [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
...
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
d. sudo systemctl restart containerd (Restart ContainerD)
INSTALL KUBEADM, KUBELET & KUBECTL ON ALL NODES
Download the Google Cloud public signing key:
a. sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
Add the Kubernetes apt repository:
b. echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
c. sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
CONFIGURE CGROUP DRIVER FOR MASTER NODE (Add section to kubeadm-config.yaml if you are using a supprted OS for SystemD and change the kubernetesVersion to the actual one installed by kubeadm)
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: 1.30.0
controlPlaneEndpoint: "masternode:6443"
networking:
podSubnet: 10.200.0.0/16
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
CONFIRGURE HOSTNAME FOR MASTER NODE
Open file : nano /ect/hosts
Add Master Node's Static IP and preferred Hostname (10.0.96.11 masternode)
INITIALIZE KUBEADM ON MASTER NODE (Remember to save the token hash)
kubeadm init --config=kubeadm-config.yaml --upload-certs | tee kubeadm-init.out
Logout from ROOT if you are stilL ROOT
To make kubectl work for your non-root user, run these commands, which are also part of the kubeadm init output:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
INSTALL A POD NETWORKING INTERFACE ON MASTER NODE
Download and Install the Tigera Calico operator and custom resource definitions.
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/tigera-operator.yaml
Download and Install Calico by creating the necessary custom resource.
Before installing remember to change the CALICO_IPV4POOL_CIDR to the POD_CIDR (10.200.0.0/16)
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/custom-resources.yaml
JOIN WORKER NODES TO THE CONTROL PLANE (MASTER NODE)
Run the command below inside each worker node with the token you got from the cli when you initialized kubeadm on Master Node:
kubeadm join masternode:6443 --token n0smf1.ixdasx8uy109cuf8 --discovery-token-ca-cert-hash sha256:f6bce2764268ece50e6f9ecb7b933258eac95b525217b8debb647ef41d49a898
Top comments (0)