DEV Community: Michael Braverman

Kubernetes from Scratch: Bootstrapping a Cluster

Michael Braverman — Mon, 08 May 2023 08:24:03 +0000

Creating a Kubernetes Cluster can be a complex process that entails so many option, but complex things can become simpler to grasp when broken down and investigated bit by bit. In this tutorial we will demystify the process of bootstrapping a Kubernetes cluster by understanding the bare-minimum components that are required to get a Kubernetes node running inside a Virtual Machine (VM).

Introduction

To begin with, it is important to have a Virtual Machine (VM) that is running a Debian-based or a RHEL-based Linux distribution. For this tutorial, we will be using a Debian 11 VM that is running inside a KVM virtual environment. The VM was provisioned using Terraform, which is a tool for building, changing, and versioning infrastructure safely and efficiently. Additionally, cloud-init was used to configure a VM with any necessary settings, such as networking and SSH access. A riced-up terminal configuration was applied that provides a more bearable terminal experience using my personal so called "dotfiles". FinallyTerraform configs for deploying ready-made virtual machines are also available at this link.

My riced up terminal heavily inspired by Garuda's configuration, an Arch-based Linux distribution.

Prerequisites

With our VM set up, we can move on to the next steps in bootstrapping a Kubernetes Cluster with cri-o as the container runtime. To begin with, we first have to make a few necessary changes in order for kubeadm init flight checks to pass. I will expand on both of these points below.

As the very first step, we should follow the number one best practice when it comes to software — checking for updates and installing the latest packages:

sudo apt update
sudo apt upgrade

As well as a few dependencies that are necessary to download

sudo apt install software-properties-common apt-transport-https ca-certificates gnupg2 gpg sudo

1. Disabling Swap

In Linux, swap is a useful way to extend the available RAM when the physical memory has been exhausted, allowing processes to continue running. However, when setting up a node for a Kubernetes cluster, it is generally recommended to disable swap for several reasons.

Firstly, Kubernetes requires a significant amount of memory to operate effectively, and any performance degradation due to swap usage can impact the performance of the entire cluster. Additionally, Kubernetes assumes that the node has a fixed amount of available memory, and if swap is enabled, it can cause confusion and unexpected behavior.

Furthermore, disabling swap can help prevent the risk of the so-called "OOM killer" from being invoked. The OOM killer is a Linux kernel process that is responsible for terminating processes when the system runs out of memory. While this is intended as a safeguard to prevent the system from crashing, it can lead to unpredictable behavior when running Kubernetes workloads, as the OOM killer may terminate critical components of the cluster.

We can see if our machine uses swap memory using the htop command:

In this screenshot swap is equal to 0. However. if its was otherwise, we would need to disable it.

Overall, while swap can be a useful tool for extending the available memory on a Linux machine, it is generally recommended to disable it when setting up a node for a Kubernetes cluster to ensure reliable and predictable performance. We can do so by issuing the following command:

swapoff -a

To make sure that swap remains disabled after startup, we have to uncomment a line in /etc/fstab that initialized swap memory upon boot:

In this this specific case, a file called swap.img was used as a swap partition which we can go ahead and delete afterwards with root privileges:

sudo rm /swap.img

Note: In modern Linux distributions, a swap file is often used instead of a separate swap partition. If your system is configured with a separate swap partition, it is important to take that into consideration when setting up a Kubernetes cluster and avoid setting up a swap partition when installing a VM.

Now you can go ahead and reboot the machine and swap should now be disabled. Use htop once again to confirm that this is the case.

2. Enabling Linux kernel modules

Enabling the necessary Linux kernel modules is a crucial step in setting up a container runtime for a Kubernetes cluster. These modules are essential for providing networking and storage functionality to the Kubernetes Pods, which are the smallest and simplest units in the Kubernetes system. Networking modules enable Kubernetes to provide network connectivity between the different Pods in a cluster, while storage modules enable the persistent storage of data across different Pods and Nodes in the cluster.

In order to enable these kernel modules, we typically need to modify the Linux kernel parameters and load the relevant kernel modules using the modprobe utility. This ensures that the necessary functionality is available to the Kubernetes cluster and that the pods can communicate and store data effectively. By enabling these modules, we can ensure that our Kubernetes cluster is well-equipped to handle a range of tasks and can provide a reliable and scalable platform for running containerized applications.

Before we proceed, we will login under root account and follow all the steps below with superuser privileges:

sudo su - root

With that out of the way, here are the two Linux kernel modules we need to enable:

br_netfilter — This module is required to enable transparent masquerading and to facilitate Virtual Extensible LAN (VxLAN) traffic for communication between Kubernetes Pods across the cluster.
overlay — This module provides the necessary kernel-level support for the overlay storage driver to function properly. By default, the overlay module may not be enabled on some Linux distributions, and therefore it is necessary to enable it manually before running Kubernetes.

We can enable these modules by issuing the modprobe command along with the -v (verbose) flag to see the results:

modprobe overlay -v
modprobe br_netfilter -v

After which we should get the following output:

In order to make sure that the kernel modules get loaded after a reboot, we can also add them to the /etc/modules file:

echo "overlay" >> /etc/modules
echo "br_netfilter" >> /etc/modules

After enabling the br_netfilter module, we must enable IP forwarding on the Linux kernel in order to enable networking between Pods and Nodes. IP forwarding allows the Linux kernel to route packets from one network interface to another. By default, IP forwarding is disabled on most Linux distributions for security reasons, since it allows a machine to be used as a router.

However, in a Kubernetes cluster, we need IP forwarding to be enabled to allow Pods to communicate with each other, as well as to allow traffic to be forwarded to the outside world. Without IP forwarding, Pods would not be able to access external resources or communicate with each other, effectively breaking the cluster.

To do so, we must write "1" to a configuration file called "ip_forward":

echo 1 > /proc/sys/net/ipv4/ip_forward

With these necessary steps for setting up the requirements our for Kubernetes cluster out of the way, we can proceed installing Kubelet — the beating heart of Kubernetes.

Installing Kubelet

Installing Kubelet is perhaps the easiest step since it is very well documented in the official Kubernetes documentation. Basically, we need to issue the following commands:

mkdir /etc/apt/keyrings
sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

The line apt-mark hold kubelet kubeadm kubectl tells our package manager to avoid upgrading these components since this is something we would want to do manually when upgrading a cluster.

Run the command, cross your fingers and hope it works:

Not that we installed kubelet, kubeadm, and kubectl, we can now proceed installing a container runtime that will run Kubernetes components, Pods and containers.

Installing our container runtime

Kubernetes is an orchestration system for containerized workloads. It manages the deployment, scaling, and operation of containerized applications across a cluster of nodes. However, Kubernetes itself does not run containers directly. Instead, it relies on a container runtime, which is responsible for starting, stopping, and managing the containers. The container runtime is the software that runs the containers on the nodes in the Kubernetes cluster.

There are several container runtimes that can be used with Kubernetes, including Docker, cri-o, containerd, and others. The choice of container runtime depends on factors such as performance, security, and compatibility with other tools in the infrastructure. For our purposes, we will chose cri-o as our container runtime.

By following the official cri-o documentation, we first need to specify the variables that are necessary to download the desired cri-o version for our specific Linux distribution. Given that we are running Debian 11 and cri-o 1.24 is the latest version at the time of writing, we will export a few variables:

export OS=Debian_11
export VERSION=1.24

We can also double check if these variables were save into our current terminal session by piping the env command to grep:

Now we can proceed installing the container runtime:

echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/Release.key | apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | apt-key add -
apt-get update
apt-get install -y cri-o cri-o-runc

If successful, we should get the following output:

Now that we have cri-o packages installed, we must enable and start cri-o as a service:

systemctl enable crio
systemctl start crio
systemctl status crio

The command systemctl status crio should will output the current service state:

Congrats! Our Kubernetes node is ready to be bootstrapped.

Bootstrapping our First node

I hope you strapped your boots tight because our Pod network will have a CIDR of 10.100.0.0/16.

What the hell is a Network CDR?

A network CIDR (Classless Inter-Domain Routing) is a notation used to represent a network prefix in IP addressing. It is a combination of an IP address and a subnet mask, represented in the form of "<IP address>/<subnet mask>". The subnet mask defines what part of the IP address is the network portion and which is the host portion.

In the case of the range 10.100.0.0/16, it means that the IP address is 10.100.0.0 and the subnet mask is 16 bits long. The subnet mask of 16 bits indicates that the first 16 bits of the IP address are the network portion and the remaining 16 bits are the host portion. This means that our host can manage up to 65,536 IP addresses, ranging from 10.100.0.0 to 10.100.255.255.

Now that we decided that our small but mighty Kubernetes cluster will have a network of 65,536 IP addresses, we can test our configuration.

Dry run

For bootstrapping our cluster will be using the official kubeadm utility. Before applying our changes we can go ahead and run our Network CIDR setting with the --dry-run flag without making any changes:

kubeadm init --pod-network-cidr=10.100.0.0/16 --dry-run

If our VM was set up properly, we should get a long output after a minute:

If the so called "pre-flight checks" output an error, then by making a quick Google search we can fix the issue and the apply these changes without the --dry-run flag.

Initializing our Cluster

One we have a VM that passes the pre-flight checks, we can initialize our cluster.

kubeadm init --pod-network-cidr=10.100.0.0/16

After issuing this command, kubeadm will turn our VM into a Kubernetes Control Plane node consisting of out of the following main components:

etcd — A key-value database store used for storing the state of the whole Kubernetes cluster;
kube-scheduler — A control plane component that watches for newly created Pods with no assigned node, and selects a node for them to run on;
kube-controller-manager — A Control plane component that runs controller processes.

If our run was successful, we should get an output with a command that can be used to join other nodes:

Take note of this join command:

kubeadm join 192.168.122.97:6443 --token nljqps.vypo4u9y07lsw7s2 \
        --discovery-token-ca-cert-hash sha256:f820767cfac10cca95cb7649569671a53a2240e1b91fcd12ebf1ca30c095c2d6

Note: By default, this join command's token is only valid for 2 hours, after which you would have to tell kubeadm to issue a new token for joining other nodes.

Once we have our first Control Plane node bootstrapped, we can use crictl the same way we use the docker command and see what components are running in our cri-o container runtime:

As we can see, the above mentioned Kubernetes components are all running as containers inside our first Control Plane node.

Adding our first worker

By default, the Control Plane node only runs containers that are part of the Kubernetes system, but no Pods a container applications. Now that we have a working Control Plane node we can proceed joining our first worker node.

Before doing so, we must follow the same exact steps as we did when setting up our control plane node. I went ahead and opened a second vm called "worker1" on the the right pane of my tmux terminal window manager:

After going though the same procedure I copy the join token from the steps above:

kubeadm join 192.168.122.97:6443 --token nljqps.vypo4u9y07lsw7s2 \
        --discovery-token-ca-cert-hash sha256:f820767cfac10cca95cb7649569671a53a2240e1b91fcd12ebf1ca30c095c2d6

And paste this into the worker1 window. The kubeadm will once again take a moment to go through all the flight checks until joining the worker to the Control Plane node.

Once that is done we can congratulate ourselves with setting a Kubernetes cluster from scratch! 🥳

Accessing our cluster with `kubectl`

The last step we have to do is to copy admin credentials that kubectl will use to manage our cluster's resource through the Kubernetes API server component running on our Control Plane node.

For the purposes of this tutorial, we will be accessing our cluster through the Control Plane node. We will copy the configuration to our user's home directory like so:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

Once we do so we can use the kubectl command to manage, create, edit, delete and provision our Kubernetes cluster. To start with, we can get a list of nodes that are currently part of our Kubernetes cluster:

kubectl get nodes

And we get a list consisting of our Control Plane node and a master worker:

Now we can create Pods, Services, Namespaces and all the good Kubernetes stuff!

Conclusion

In conclusion, the process of bootstrapping a Kubernetes cluster with cri-o as the container runtime may seem daunting at first, but with the proper understanding and knowledge of the individual components involved, it can be achieved with relative ease. By following the steps outlined in this tutorial, we have gained a deeper understanding of how Kubernetes operates and the requirements necessary to set up a functional cluster. Armed with this knowledge, we can now confidently experiment and explore the full potential of Kubernetes in our development and production environments.

1. Applying Design Patterns in Python: GZIP Decorator

Michael Braverman — Tue, 15 Nov 2022 20:11:44 +0000

Understanding design patterns is a powerful skill that any programmer should have in their arsenal. It is as powerful as learning grammar of a spoken language. Although most native speakers go about their lives without giving second thought to how they write and utter sentences, those who learn the grammar of a language possess an advantageous power, regardless of whether they are native speakers or not. Not only does this power translate into a more intricate understanding of a given language, it also translates into an ability to understand universal rules that are common among all spoken languages.

The same analogy applies to Design Patterns in programming languages. And as with any skill, one can really learn something only through practice. So today we will be creating a GZIP compressor by implementing the Decorator Design Pattern, while also learning a few things about bytes, strings and UTF-8 encoding on the side.

The Decorator Design Pattern

The Decorator design pattern is used to dynamically add a new feature to an object without changing its implementation. It differs from inheritance because a new feature is added only to that particular object, not to the entire subclass.

Similar to how decorators in Python are functions that wrap other functions, the Decorator pattern on the other hand, is essentially a class for wrapping other classes and objects.

Creating a Base Class for our Data

Before we proceed compressing strings with GZIP, we can create a helper class that will allow us to manage our data that we will later “decorate”. This is necessary because GZIP works with the bytes (also referred to as a “blob”) type, not with the str type. The class Data will handle the conversion from strings to bytes and vice versa, as well as handle the encoding to and from UTF-8:

# Define a common encoding
ENCODING_TYPE = 'utf-8'

class Data:
    """Data Class
    Stores data in string or bytes. 
    Always returns bytes
    """

    # Decrease memory usage from 48 to 40 bytes
    __slots__ = ('_data')

    def __init__(self, data: str | bytes) -> None:
        self._data = data

    def _get_str(self) -> str:
        """Get string from stored data and convert using the set encoding"""
        if type(self._data) is not str:
            return str(self._data, encoding=ENCODING_TYPE)
        return self._data

    def _get_bytes(self) -> bytes:
        """Get bytes from stored data and convert using the set encoding"""
        if type(self._data) is not bytes:
            return bytes(self._data, encoding=ENCODING_TYPE)
        return self._data

    @property
    def text(self) -> str:
        return self._get_str()

    @text.setter
    def text(self, value: str | bytes) -> None:
        self._data = value

    @property
    def blob(self) -> bytes:
        return self._get_bytes()

    @blob.setter
    def blob(self, value: str | bytes) -> None:
        self._data = value

Our Data class is able to handle any data type, whether it is a byte or a str.

Creating an Abstract Class for our GZIP Decorator

As an extra step, we can create an abstract class called DataSourceDecorator. Abstract classes in Python are helpful when we want to define a blueprint for other classes. By defining an abstract base class, you can define a common Application Program Interface(API) for a set of subclasses. In this case, we want to create a class that handles GZIP compression and decompression. By defining a base class for our GZIP class, we can have the ability to extend how we want to manipulate our data later on. For example, we could later add a class that can handle bz2, lzma, zlib, tar, or zip compression — or perhaps, encrypt and decrypt our data.

from abc import ABC, abstractmethod

class DataSourceDecorator(ABC):
    """Abstract Data Source Decorator"""

    @abstractmethod
    def __init__(self, data: Data) -> None:
        self._data = data

    @abstractmethod
    def _compress(self, blob: bytes) -> None:
        ...

    @abstractmethod
    def _inflate(self, blob: bytes) -> None:
        ...

    def read(self) -> Data:
        return self._data

Regardless of our case, we can define __init__(), _compress(), and _inflate() abstract methods. By specifying @abstractmethod for a given method, we tell Python’s interpreter that this method can only be used once it is redefined in a child class. This makes sense because, for every compression algorithm that we would later want to implement, we have to define the various libraries that we might have to be initialized, as well as define the diverging inflation and compression steps. While the read() method is likely to be used everywhere the same way regardless of the compression algorithm we decide to implement and therefore does not have to be abstract.

Implementing GZIP

Python’s standard library has an excellent built-in gzip module that we will be using for implementing out GZIPDecorator class based on the DataSourceDecorator base class.

import gzip

class GZIPDecorator(DataSourceDecorator):
    """GZIP Compression Decorator"""

    def __init__(self, data: Data, compressed: bool = False, compresslevel: int = 9) -> None:
        if type(data) is Data:
            blob = data.blob
        else:
            raise TypeError(f'Unsupported type "{type(data)}"')

        self._compresslevel = compresslevel

        if compressed:
            self._inflate(blob)
        else:
            self._compress(blob)

    def _compress(self, blob: bytes):
        self._data = Data(gzip.compress(blob, self._compresslevel))

    def _inflate(self, blob: bytes):
        self._data = Data(gzip.decompress(blob))

In our GZIPDecorator class, we redefined our __init__() method so that we can specify whether the data we want to wrap is compressed or inflated, set the compression level (from 0 to 9), and require that the input data is of a Data type so that we can comfortably use the bytes of our data.

Seeing our Decorator in action

Now that we have our classes in place, we can test whether our compression decorator works as intended:

# Create a long string with duplicated data
our_data = Data('Our Data' * 1000)
print(f'Uncompressed size:\t{len(our_data.blob)}')

# Compress our data
compressed_data = GZIPDecorator(our_data)
print(f'Compressed size:\t{len(compressed_data.read().blob)}')

# Calculate compression ratio
ratio = len(our_data.blob) / len(compressed_data.read().blob)
print(f'Compressed by:  \t{ratio:.2f}X!')

# Test decompressed data
decompressed_data = GZIPDecorator(compressed_data.read(), True)
status = 'SUCCESS' if decompressed_data.read().blob == our_data.blob else 'FAILURE'
print(f'Decompression   \t{status}')

And we get the following result:

Uncompressed size:      8000
Compressed size:        57
Compressed by:          140.35X!
Decompression           SUCCESS

On a set of data that has a 8 byte-long string duplicated 1000 times we got a 140X compression ratio. Not too bad!

Conclusion

The Decorator can be a helpful pattern in the Python language. This example demonstrated how the GZIPDecorator class can effectively be used to wrap our Data class without changing its underlying functionality. By using the same pattern, we then create other types of decorator classes that could manipulate our Data class.

Code is available on GitHub.

DEV Community: Michael Braverman

Kubernetes from Scratch: Bootstrapping a Cluster

Introduction

Prerequisites

1. Disabling Swap

2. Enabling Linux kernel modules

Installing Kubelet

Installing our container runtime

Bootstrapping our First node

What the hell is a Network CDR?

Dry run

Initializing our Cluster

Adding our first worker

Accessing our cluster with kubectl

Conclusion

1. Applying Design Patterns in Python: GZIP Decorator

The Decorator Design Pattern

Creating a Base Class for our Data

Creating an Abstract Class for our GZIP Decorator

Implementing GZIP

Seeing our Decorator in action

Conclusion

Accessing our cluster with `kubectl`