Cluster API Theoretical and Hands-On Breakdown

There are various methods available when it comes to creating infrastructure. Whether you go with an Infrastructure-as-Code tool like Terraform, an SDK, a Cloud Development Kit (CDK), a Configuration Management tool like Ansible, or even the console/UI, there’s no shortage of ways in which you can spin up various sources on-prem and in the cloud.

But what about a more declarative approach? A method that you can use to create Kubernetes clusters with Kubernetes.

In this blog post, you’ll learn how to do exactly that with Cluster API.

Prerequisites

To follow along with the hands-on portion of this blog post, you should have an Azure account and the ability to create an Azure Kubernetes Service (AKS) cluster. If you don’t have an Azure account, you can still read through the code and see how the implementation of Cluster API works as it’ll be very similar with other Providers.

What Is Cluster API

When you want to create a Kubernetes cluster, there are a few different common methods.

1. An Infrastructure-as-Code tool like Terraform
2. The UI
3. An SDK

and a few other methods.

The thing is, none of these methods are declarative like creating and running a Kubernetes Manifest or closer to the method of which you work with Kubernetes on a daily basis.

That’s where Cluster API comes into play.

Cluster API is a declarative approach to creating Kubernetes clusters from another Kubernetes cluster. The way it works is you have a Management Cluster that acts almost like a Control Plane. The Management Cluster has Cluster API installed based on the provider you’re using (you’ll learn about Providers in the subsection below). Once Cluster API exists on the Management Cluster, you can use it to generate Kubernetes Manifests and those Kubernetes Manifests can be used to create new clusters via API calls.

When you run kubectl apply -f on an existing Kubernetes Manifest to create a Pod, Service, or another Kubernetes resource, you’re making an API call. The same approach is used with Cluster API.

Cluster API provides declarative APIs for creating, updating, modifying, and deleting Kubernetes clusters with Kubernetes. It’s essentially using Kubernetes to manage Kubernetes.

What’s A Provider?

When you work with Cluster API, you’ll be creating clusters from a Kubernetes cluster. The Kubernetes cluster will be running on the cloud or on-prem whether it’s Azure, AWS, another cloud, or in your datacenter.

To work with the specific cloud provider or on-prem datacenter, you’ll need a method of interacting with it. That method is with a Provider. A Provider is built to interact via API calls with a cloud provider or an on-prem environment. That way, the Kubernetes cluster that you’re using to create the other clusters has permission to do so.

You’ll see how it’s built and configured in the Cluster API With Azure section.

Cluster API With Azure

Now that you know what Cluster API is and why you’d want to use it, let’s see it in action. For the purposes of this section, you’ll learn how Cluster API is used with Azure, but as mentioned in the Prerequisites, the process across various providers is pretty similar.

First, ensure that you install Cluster API. There are two examples below - one for Linux and one for Mac. Other installation methods can be found here.

## Linux
curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.4.4/clusterctl-linux-amd64 -o clusterctl

sudo install -o root -g root -m 0755 clusterctl /usr/local/bin/clusterctl

## Mac
brew install clusterctl

Once installed, you should be able to run clustectl on your terminal and see the output.

Now that the clusterctl is installed, you’ll need to set some environment variables and Kubernetes Secrets to interact with the cloud provider. In the example below, the environment variables are being set to authenticate to Azure.

The CLUSTER_TOLOGY environment variables set feature gates, which are used for experimental features. You can find out more information here.

export CLUSTER_TOPOLOGY=true

export AZURE_SUBSCRIPTION_ID=""

# Create an Azure Service Principal and paste the output here
export AZURE_TENANT_ID=""
export AZURE_CLIENT_ID=""
export AZURE_CLIENT_SECRET=""

# Base64 encode the variables
export AZURE_SUBSCRIPTION_ID_B64="$(echo -n "$AZURE_SUBSCRIPTION_ID" | base64 | tr -d '\n')"
export AZURE_TENANT_ID_B64="$(echo -n "$AZURE_TENANT_ID" | base64 | tr -d '\n')"
export AZURE_CLIENT_ID_B64="$(echo -n "$AZURE_CLIENT_ID" | base64 | tr -d '\n')"
export AZURE_CLIENT_SECRET_B64="$(echo -n "$AZURE_CLIENT_SECRET" | base64 | tr -d '\n')"

# Settings needed for AzureClusterIdentity used by the AzureCluster
export AZURE_CLUSTER_IDENTITY_SECRET_NAME="cluster-identity-secret"
export CLUSTER_IDENTITY_NAME="cluster-identity"
export AZURE_CLUSTER_IDENTITY_SECRET_NAMESPACE="default"

Once the environment variables are set, create a Kubernetes Secret for authenticating to Azure from AKS.

kubectl create secret generic "${AZURE_CLUSTER_IDENTITY_SECRET_NAME}" --from-literal=clientSecret="${AZURE_CLIENT_SECRET}" --namespace "${AZURE_CLUSTER_IDENTITY_SECRET_NAMESPACE}"

Initialize the Management Cluster, which indicates you’re running in Azure.

clusterctl init --infrastructure azure

After the Management Cluster is initialized, you can generate the Kubernetes Manifest to create a new cluster.

To create a new cluster, specify a few environment variables like the size of the cluster and where you want it to run. This cluster will be bootstrapped with Kubeadm.

export AZURE_LOCATION="eastus"

export AZURE_CONTROL_PLANE_MACHINE_TYPE="Standard_D2s_v3"
export AZURE_NODE_MACHINE_TYPE="Standard_D2s_v3"

export AZURE_RESOURCE_GROUP="devrelasaservice"

Run the generate command via clusterctl which specifies the Kubernetes version that you wish to run.

clusterctl generate cluster capi-azure --kubernetes-version v1.27.0 > capi-azurekubeadm.yaml

You should now see a new file called capi-azurekubeadm.yaml.

Create the cluster by running the Kubernetes Manifest that contains the declarative code to create a new cluster.

kubectl apply -f capi-azurekubeadm.yaml

Run the command below and you should now see the new cluster available.

kubectl get clusters

Once the cluster is running, you can set up the Cloud Controller. The Cloud Controller is a Kubernetes Controller that’s typically abstracted away from you when running in the cloud, but it’s something you have to install and manage with Cluster API.

helm install --kubeconfig=./capi-azure.kubeconfig --repo https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo cloud-provider-azure --generate-name --set infra.clusterName=capi-azure --set cloudControllerManager.clusterCIDR="192.168.0.0/16"

The last step is to get a Container Network Interface (CNI) up and running in your cluster. That way, you have internal Kubernetes networking. An example of getting a CNI installed is below with Calico.

helm repo add projectcalico https://docs.tigera.io/calico/charts --kubeconfig=./capi-azure.kubeconfig && \

helm install calico projectcalico/tigera-operator --kubeconfig=./capi-azure.kubeconfig -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-azure/main/templates/addons/calico/values.yaml --namespace tigera-operator --create-namespace

You should now be able to see your cluster fully up and running.

clusterctl get kubeconfig capi-azure > capi-azure.kubeconfig

Congrats! You have successfully deployed a Kubernetes cluster with Cluster API.