This started off due to frustration with my current Single Board Computers setup and limitations using Docker Desktop meaning I couldn't fully utilise all the features of Kubernetes.
I'd previously setup up a k3s cluster using k3sup on my Raspberry Pi's and ASUS Tinkerboard running Ubuntu 20.04. Due to using older models – Pi 2 & 3 -, with limited resources, it wasn't a great experience for testing demanding applications or load testing.
Docker Desktop Kubernetes is great for starting out but the limitations kick in when you want to test out multiple nodes and installing additional features like Ingress Controllers. Plus, running it on a MacBook Pro 2019 I found the battery life draining quickly – running just Docker Desktop is bad enough at times.
So here comes in Multipass. I'd seen it a few times and was very curious. I'd previously used Vagrant combined with VirtualBox with various levels of success and agony. So the idea of being able to run an Ubuntu VM and Kubernetes Cluster without additional programs to manage and debug got me hooked. I like things to be as native as possible and Multipass can run on HyperKit and Hyper-V.
Installing Multipass is pretty slick and having a CLI – like Vagrant – meant there was a way to wrap it all up in a script for repeatability. And install k3sup is even easier.
Bonus feature of Multipass is being able to pass in cloud-init files to do additional configuration setup, which adds many benefits beyond this example.
The plan is to create a 3 node – 1 master, 2 workers – k3s cluster, which is disposable and portable.
Tools used:
So let's begin!
k3sup uses SSH to install and by default Multipass uses a predefined User – ubuntu – and SSH Key-Pair, which can be found after bit of digging - see Issue #913. This is where cloud-init comes in.
Using cloud-init, we can easily create a user, assigned it to groups, and pass in a Public SSH Key of our choice. And this is the minimal to do so:
users:
- name: tom
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa...
Next is to spin up the nodes, which can all be done with the Multipass CLI. We'll leave the VMs with the default values for CPU, RAM, Disk Space and OS - latest LTS - but include our cloud-init configuration:
multipass launch -n master --cloud-init - <<EOF
users:
- name: tom
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa...
EOF
multipass launch -n node1 --cloud-init - <<EOF
users:
- name: tom
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa...
EOF
multipass launch -n node2 --cloud-init - <<EOF
users:
- name: tom
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa...
EOF
Using the Multipass CLI, we can then view the IP Addresses of our new VMs:
multipass list
Name State IPv4 Image
master Running 192.168.64.8 Ubuntu 20.04 LTS
node1 Running 192.168.64.9 Ubuntu 20.04 LTS
node2 Running 192.168.64.10 Ubuntu 20.04 LTS
Now to get our k3s cluster built, again using the default values, first by setting up our master node:
k3sup install --ip 192.168.64.8 --context k3s-cluster --user tom --ssh-key ./demo-key
This will install k3s, setup the instance as the master node and return the kubeconfig to the current directory.
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ...
server: https://192.168.64.8:6443
name: k3s-cluster
contexts:
- context:
cluster: k3s-cluster
user: k3s-cluster
name: k3s-cluster
current-context: k3s-cluster
kind: Config
preferences: {}
users:
- name: k3s-cluster
user:
client-certificate-data: ...
client-key-data: ...
Next, to set up our worker nodes to join the cluster:
k3sup join --server-ip 192.168.64.8 --ip 192.168.64.9 --user tom --ssh-key demo-key
k3sup join --server-ip 192.168.64.8 --ip 192.168.64.10 --user tom --ssh-key demo-key
Finally, we can now check our k3s cluster is up and running:
KUBECONFIG=kubeconfig kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 7m17s v1.19.7+k3s1
node1 Ready <none> 6m47s v1.19.7+k3s1
node2 Ready <none> 6m24s v1.19.7+k3s1
And now time to tear it all down now:
multipass delete --all && multipass purge
The Next Level
I mentioned that I wanted this setup to be disposable and portable. So let's wrap this all up in a good ol' Shell script.
So with a bit of refactoring and Command Line tooling - Piping & Parsing - we can end up with a script like so:
#!/bin/sh
master="master"
nodes=("node1" "node2")
context="k3s-cluster"
createInstance () {
multipass launch -n "$1" --cloud-init - <<EOF
users:
- name: ${USER}
groups: sudo
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- $(cat "$PUBLIC_SSH_KEY_PATH")
EOF
}
getNodeIP() {
echo $(multipass list | grep $1 | awk '{print $3}')
}
installK3sMasterNode() {
MASTER_IP=$(getNodeIP $1)
k3sup install --ip "$MASTER_IP" --context "$context" --user "$USER" --ssh-key "${PRIVATE_SSH_KEY_PATH}"
}
installK3sWorkerNode() {
NODE_IP=$(getNodeIP $1)
k3sup join --server-ip "$MASTER_IP" --ip "$NODE_IP" --user "$USER" --ssh-key "${PRIVATE_SSH_KEY_PATH}"
}
createInstance $master
for node in "${nodes[@]}"
do
createInstance "$node"
done
installK3sMasterNode $master
for node in "${nodes[@]}"
do
installK3sWorkerNode "$node"
done
And with only needing two Environment Variables, we can now bring up a disposable cluster within minutes - I clocked just under 2 minutes!
PUBLIC_SSH_KEY_PATH=./demo-key.pub PRIVATE_SSH_KEY_PATH=./demo-key ./minimal-k3s-multipass-bootstrap.sh
Making a script as well allows for easy use on multiple systems without needing to install more than the necessary.
Multipass offers extra configuration options for the VMs and cloud-init can do more internally on the OS. Plus k3sup has even more options, but I'll keep it simple for now!
Enjoy!
Top comments (0)