Setting up multi-node Kubernetes cluster locally with K3s and Multipass

There are a lot of tools that allow you to setup a local Kubernetes cluster in no time. But with a full-blown K8s running on your local machine, you will soon hit a wall if you want to play with multi-node cluster.

For tackling this very issue, we will be looking into how we can setup a lightweight Kubernetes cluster using K3s and Multipass for our VMs.

If you are interested in what makes K3s so light, you can watch the talk on K3s under the hood.

Prerequisites

Installing Multipass

Multipass is a command line tool to orchestrate virtual Ubuntu instances on your local machine. With the CLI, you can spawn VMs within minutes allocating optimal CPU, memory and disk space.

Download Multipass

Installing K3sup (ketchup)

We talked about K3s but it can be daunting to set up if you just want a quick hands on with Kubernetes. To abstract all that we will be installing a handy CLI called k3sup. It will bootstrap Kubernetes cluster on our VMs using K3s within a minute! 🚀

k3sup uses SSH under the hood, so make sure you have SSH service on your machine.

Install k3sup

Creating Virtual machines

Nodes are just virtual machines on Kubernetes working in sync. For our cluster we will be creating 3 virtual machines, each with 1CPU, 1GB RAM, 2GB Storage.

Generating keys

If you have SSH installed and already have ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub , you can skip this part and move to **creating config file **section.

Create Private/Public key:

$ ssh-keygen

This will create the aforementioned files on your system. Copy the contents of ~/.ssh/id_rsa.pub

$ cat ~/.ssh/id_rsa.pub

Creating config file

Create a file called multipass.yaml and place your public key in ssh-rsa.

# multipass.yaml
ssh_authorized_keys:

  - ssh-rsa <add-your-public-key>

This config file makes sure the public key is stored on the virtual machine once it’s created. Let’s create our VMs with proper names based on their roles we will be assigning (master/worker).

$ multipass launch --cpus 1 --mem 1G --disk 2G --name master-node --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-master --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-worker --cloud-init multipass.yaml

$ multipass ls

In K3s terms, a master node is called the server and the rest of the nodes are called agents. Agents are simply the nodes that gets added to the master node; they can be another master node or a worker node.

Adding K8s sauce with k3sup

Now that we have our VMs ready, let’s install Kubernetes on them. First we will be creating a master node to setup a control plane.

Adding a master node

We will need the IP and username of our virtual machine to SSH into and install Kubernetes. Run multipass ls and take note of IP of master-node . All the usernames for VMs are ubuntu by default.

$ k3sup install --ip <IP> --user ubuntu --k3s-extra-args "--cluster-init"

We are passing the --k3s-extra-args "--cluster-init" to make sure this node is prepared to connect with another master node, else it might cause errors.

Once installed it downloads the kubeconfig file on the directory where you invoked your command. You can set environment variable KUBECONFIG with path to recently downloaded kubeconfig file.

# mac / linux
$ export KUBECONFIG=<path-to-kubeconfig>

# windows
$ setx KUBECONFIG <path-to-kubeconfig>

Now you can kubectl get nodes to view the nodes on your cluster.

Adding a second master node (HA)

Multi master setup on local machine is an overkill for most use cases but for the purpose of this tutorial we are going to add it anyway. To join a new master node in the cluster we use the join command.

Here, all we have to do is introduce the server IP and username (previously created) that this node should connect to . And additionally pass the --server flag specifying this will be a server node (master).

$ k3sup install --ip <IP-of-agent-master> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu --server

And that’s about it. Now you can kubectl get nodes again, and you’ll see two master nodes that is Highly Available (HA).

Adding a worker node

Finally, we have to setup our worker node where we will be deploying our applications and services.

For this grab the IP of agent-worker and pass on the same command as before minus the --server flag.

$ k3sup join --ip <IP-of-agent-worker> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu

Now if you kubectl get nodes , you will find two master nodes and a single worker node composing your multi node cluster setup. You can create new VMs and add more master or worker nodes depending on how you plan to utilize your cluster.

Conclusion

Thank you for reading. Hope this guide has been helpful for you to setup and play with multi-node Kubernetes cluster. If you have any corrections, suggestions, or feedback feel free to DM me on Twitter or comment below.

Happy hacking! 🚀

Comments

[Mert Can Bilgiç]

Hi, thanks for this awesome article. But the command you've used while creating second master node is wrong, you have written "k3sup install" instead of "k3sup join".