Exploring the HelmChart custom resource in k3s

k3s is a lightweight Kubernetes distribution developed by Rancher which is suitable for edge, IoT, and resource-constrained computing environments.

Helm is a package manager for Kubernetes which enables users to find, share, and use software built for Kubernetes.

Traditionally, Helm is comprised of a client (helm) and server-side component, Tiller, which manages the deployment and lifecycle of Helm charts. In k3s, Tiller is replaced by Helm Controller. Helm Controller defines a new HelmChart custom resource definition, or CRD, for managing Helm charts.

You can verify the HelmChart resource is present on a k3s cluster by using kubectl to list the available API resources.

 $ sudo k3s kubectl api-resources --api-group=helm.cattle.io
NAME         SHORTNAMES   APIGROUP         NAMESPACED   KIND
helmcharts                helm.cattle.io   true         HelmChart

The HelmChart resource

Below is an example of a HelmChart resource definition.

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata: # Kubernetes object metadata
  name: # Chart name (required)
  namespace: kube-system # Namespace Helm Controller is monitoring to deploy charts (required)
spec: # resource specification

The apiVersion and kind fields together identify this as a HelmChart resource to be managed by Helm Controller.

The name field under metadata specifies a name for the HelmChart resource. namespace specifies the Kubernetes namespace in which Helm Controller is monitoring CRDs. For k3s, this must be kube-system as Helm Controller is only configured to watch this namespace for new HelmChart resources.

Lastly, the spec, or specification, field is used to specify the desired state of the HelmChart resource.

HelmChart resource spec

The HelmChart resource supports several configuration parameters. These parameters correspond to helm install command line arguments and are passed to Helm during chart deployment.

• chart (string: null) - Chart reference or a URL.
• targetNamespace (string: null) - Namespace to install the Chart into. Defaults to the namespace used by the k3s controller (i.e., kube-system) if omitted.
• version (string: null) The exact Chart version to install. If this is not specified, the latest version is installed.
• repo (string: null) - Chart repository URL where to locate the requested Chart.
• set (string: null) - A dictionary/map consisting of configuration values to provide to Helm. Keys should be specified as strings. Values may be either Integers or Strings. If more complex values are required, use valuesContent.
• valuesContent (string: null) - A list of configuration values to provide to Helm. This should be a multi-line string.

Below is an example of a HelmChart resource definition with all available configuration values defined.

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: example-helmchart
  namespace: kube-system
spec:
  chart: test-helmchart
  targetNamespace: test-namespace
  version: 0.0.1
  repo: https://example.com/helm-charts
  set:
    key1: "value1"
    key2: "value2"
  valuesContent: |-
    config:
      application:
        admin_username: admin
        admin_password: insecure_password
      server:
        port: 8080
        tls_enabled: false

The corresponding CLI arguments for this would be:

$ helm install test-helmchart --namespace test-namespace \
                              --version 0.0.1 \
                              --repo https://example.com/helm-charts \
                              --set-string key1=value1 \
                              --set-string key2=value2 \
                              --values values.yaml

Where values.yaml contains the content present under the valuesContent field.

Note: Both set and valuesContent may be used to configure the chart. set only supports strings or integers as values. If other data types are needed, use valuesContent.

Creating a HelmChart

We will now walk through the process of creating a HelmChart resource definition to deploy Node-RED – a flow-based programming for the Internet of Things.

The following steps were performed on a Raspberry Pi 4 running the Raspian operating system. You may need to modify the commands if using a different environment.

First, create a file named node-red-helmchart.yaml using your default editor.

$ editor node-red-helmchart.yaml

Copy and paste the following block of text into the file.

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: node-red
  namespace: kube-system
spec:
  chart: stable/node-red
  targetNamespace: node-red
  set:
    image.tag: rpi
  valuesContent: |-
    ingress:
      enabled: "true"
      annotations:
        kubernetes.io/ingress.class: traefik
        traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
      hosts:
        - raspberrypi1.local
      path: /node-red/

Note: Be sure to modify the ingress.hosts field to contain the hostname and/or IP address of one or more node in your Kubernetes cluster.

The deployment of the chart has been customized by modifying several of the available configuration values. Particularly, Ingress support has been enabled which exposes Node-RED to hosts outside of the Kubernetes cluster at http://raspberrypi1.local/node-red/.

Installing a HelmChart

There are two methods to install a HelmChart in k3s.

1. Install the resource definition using kubectl apply.
2. Copy the resource definition to the server's manifest directory at /var/lib/rancher/k3s/server/manifests/.

Installing via kubectl

Instruct Kubernetes to install the specified resource definition using kubectl apply.

$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
helmchart.helm.cattle.io/node-red created

This is the standard method of provisioning resources within Kubernetes.

Installing using Auto-Deploying Manifests

k3s includes a feature called Auto-Deploying Manifests which allows for the automated deployment of Helm charts (and Kubernetes manifests) placed in k3s manifest directory (/var/lib/rancher/k3s/server/manifests). Helm Controller continually watches this directory for new CRDs. When it detects a new resource has been added, it will automatically deploy the chart. Therefore, you can initiate the installation by simply copying node-red-helmchart.yaml into the server's manifest directory.

$ sudo cp node-red-helmchart.yaml /var/lib/rancher/k3s/server/manifests/node-red-helmchart.yaml

Installation process under the hood

Regardless of the installation method used, when a chart is deployed Helm Controller creates a Kubernetes Job which installs the chart into the cluster using helm install.

Execute kubectl get jobs in the kube-system namespace and search for Jobs associated with the deployed chart.

$ sudo k3s kubectl --namespace kube-system get jobs \
                   --selector helmcharts.helm.cattle.io/chart=node-red
NAME                    COMPLETIONS   DURATION   AGE
helm-install-node-red   0/1           21s        23s

You can view the command the Job is executing by re-running kubectl get jobs, and appending additional arguments to select and display Job-specific information using kubectl's custom-column output.

$ export CUSTOM_COLUMN_OUTPUT="COMMAND:.spec.template.spec.containers[0].name,INSTALL_ARGS:.spec.template.spec.containers[0].args"
$ sudo k3s kubectl --namespace kube-system get job helm-install-node-red \
                   --output=custom-columns=$CUSTOM_COLUMN_OUTPUT
COMMAND   INSTALL_ARGS
helm      [install --name node-red stable/node-red --namespace node-red --set-string image.tag=rpi]

The Job installs the chart by executing helm install with the list of installation arguments provided in the HelmChart resource, as noted earlier.

Verifying the chart installation

Verify the chart has been successfully deployed by ensuring the associated resources have been created within the node-red namespace.

$ sudo kubectl --namespace node-red get ingress,service,pods
NAME                          HOSTS                ADDRESS      PORTS   AGE
ingress.extensions/node-red   raspberrypi1.local   10.0.0.133   80      2m2s

NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/node-red   ClusterIP   10.43.135.177   <none>        1880/TCP   2m2s

NAME                            READY   STATUS    RESTARTS   AGE
pod/node-red-7c5c564cb6-sz8b8   1/1     Running   0          2m1s

Updating a HelmChart

The configuration for a chart may be updated using kubectl apply.

$ sudo k3s kubectl apply --filename node-red-helmchart.yaml

Deleting a HelmChart

To delete a Helm Chart, use kubectl delete helmchart.

$ sudo k3s kubectl --namespace kube-system delete helmchart node-red

Conclusion

You have learned about the HelmChart custom resource available in K3s, how to construct a HelmChart resource definition, install it on the cluster, and perform post-deployment operations (update/delete).

That's it for this walkthrough.

I have a written program, k3s-helmchart-generate, which simplifies the process of building HelmChart manifests. You can find a link to the repo on GitHub below.

blake / k3s-helmchart-generate

Helper script to generate HelmChart CRDs for use with k3s

Comments

[2stacks]

Great article. I'm curious if you had to pre-create the 'node-red' namespace for this to work. I'm trying to deploy cert-manager in the same way but it fails due to the name space not already existing.