What are Kubernetes Operators and Custom Resources?

Kubernetes API Resources

I assume that you have a basic understanding of Kubernetes and know how to use kubectl.

Pod, ReplicaSet, Service, and even Namespace are actually types of API Resources.

You can use kubectl api-resources to view all the API Resources currently available in your Kubernetes cluster.

$ kubectl api-resources
NAME                     SHORTNAMES   APIVERSION   NAMESPACED   KIND
bindings                              v1           true         Binding
componentstatuses        cs           v1           false        ComponentStatus
configmaps               cm           v1           true         ConfigMap
endpoints                ep           v1           true         Endpoints
events                   ev           v1           true         Event
limitranges              limits       v1           true         LimitRange
namespaces               ns           v1           false        Namespace
nodes                    no           v1           false        Node
persistentvolumeclaims   pvc          v1           true         PersistentVolumeClaim
persistentvolumes        pv           v1           false        PersistentVolume
pods                     po           v1           true         Pod
podtemplates                          v1           true         PodTemplate
replicationcontrollers   rc           v1           true         ReplicationController
resourcequotas           quota        v1           true         ResourceQuota
secrets                               v1           true         Secret
serviceaccounts          sa           v1           true         ServiceAccount
services                 svc          v1           true         Service
...

Kubernetes Custom Resources

Custom Resources, as the name suggests, refer to custom API Resources that we can create and install in Kubernetes to extend its functionality.

According to the official documentation, there are two ways to create Custom Resources, but the most common method is using Custom Resource Definitions (CRD), so we will focus on CRDs here.

In fact, CRD itself is a type of API Resource, and this can be clearly seen with the following command:

$ kubectl api-resources | grep -i custom
customresourcedefinitions   crd,crds   apiextensions.k8s.io/v1   false   CustomResourceDefinition

Think about how we usually create Resources; typically, we write a YAML file and then create it with kubectl apply. The steps to create a CRD are the same.

First, create a file called chishengliu-crd.yaml with the following content:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: chishenglius.chishengliu.com
spec:
  group: chishengliu.com
  scope: Namespaced
  names:
    plural: chishenglius
    singular: chishengliu
    kind: ChiShengLiu
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                mood:
                  type: string

Then run the following command:

$ kubectl apply -f chishengliu-crd.yaml
customresourcedefinition.apiextensions.k8s.io/chishenglius.chishengliu.com created

$ kubectl api-resources | grep -i chishengliu
chishenglius   chishengliu.com/v1   true   ChiShengLiu

You will notice that there is now a new Resource in the cluster called ChiShengLiu.

To create this kind of ChiShengLiu Resource, you also write a YAML file and then use kubectl apply. Create a file called happy-chishengliu.yaml with the following content:

apiVersion: chishengliu.com/v1
kind: ChiShengLiu
metadata:
  name: happy-chishengliu
spec:
  mood: "happy"

Then run the following command:

$ kubectl apply -f happy-chishengliu.yaml
chishengliu.chishengliu.com/happy-chishengliu created

$ kubectl describe chishengliu happy-chishengliu
Name:         happy-chishengliu
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  chishengliu.com/v1
Kind:         ChiShengLiu
Metadata:
  Creation Timestamp:  2024-08-28T17:37:57Z
  Generation:          1
  Resource Version:    1609
  UID:                 3cbc6964-2c7b-4e60-b113-4fa5f4dd0b3c
Spec:
  Mood:  happy
Events:  <none>

You can see that we have created a ChiShengLiu Resource named happy-chishengliu.

Kubernetes Operator

Having a Custom Resource alone is not very useful. You might notice that after creating this ChiShengLiu Resource, nothing changes in the Kubernetes cluster, unlike with built-in Resources.

Built-in Resources have functionality because there are corresponding Controllers running in the cluster. The Controller's job is to monitor certain Resources in the cluster and adjust the cluster to the state declared in the Resource's spec. For example, the Replication Controller monitors all ReplicaSets in the cluster, and when any ReplicaSet is created, updated, or deleted, it adjusts the cluster to the corresponding state, which means creating the correct number of Pods.

Therefore, if we want the ChiShengLiu Resource we created earlier to have an effect, we need to install a Controller for ChiShengLiu in the cluster. This Controller is not built-in and needs to be written by ourselves, so it's called a Custom Controller, also known as a Kubernetes Operator. The official documentation refers to this pattern as the Operator Pattern.

The most common use case for Kubernetes Operators is to encapsulate underlying logic, especially for framework or tool developers, allowing users to install everything by simply applying a YAML file with kubectl apply, with the Operator handling the rest.

So, how do we write a Kubernetes Operator? Stay tuned for future updates in this series.

References

• Custom Resources | Kubernetes
• Operator pattern | Kubernetes