k3s is a lightweight Kubernetes distribution developed by Rancher which is suitable for edge, IoT, and resource-constrained computing environments.
Helm is a package manager for Kubernetes which enables users to find, share, and use software built for Kubernetes.
Traditionally, Helm is comprised of a client (
helm) and server-side component, Tiller, which manages the deployment and lifecycle of Helm charts. In k3s, Tiller is replaced by Helm Controller. Helm Controller defines a new
HelmChart custom resource definition, or CRD, for managing Helm charts.
You can verify the
HelmChart resource is present on a k3s cluster by using
kubectl to list the available API resources.
$ sudo k3s kubectl api-resources --api-group=helm.cattle.io NAME SHORTNAMES APIGROUP NAMESPACED KIND helmcharts helm.cattle.io true HelmChart
The HelmChart resource
Below is an example of a
HelmChart resource definition.
--- apiVersion: helm.cattle.io/v1 kind: HelmChart metadata: # Kubernetes object metadata name: # Chart name (required) namespace: kube-system # Namespace Helm Controller is monitoring to deploy charts (required) spec: # resource specification
kind fields together identify this as a HelmChart resource to be managed by Helm Controller.
name field under
metadata specifies a name for the HelmChart resource.
namespace specifies the Kubernetes namespace in which Helm Controller is monitoring CRDs. For k3s, this must be
kube-system as Helm Controller is only configured to watch this namespace for new HelmChart resources.
spec, or specification, field is used to specify the desired state of the HelmChart resource.
The HelmChart resource supports several configuration parameters. These parameters correspond to
helm install command line arguments and are passed to Helm during chart deployment.
chart(string: null) - Chart reference or a URL.
targetNamespace(string: null) - Namespace to install the Chart into. Defaults to the namespace used by the k3s controller (i.e., kube-system) if omitted.
version(string: null) The exact Chart version to install. If this is not specified, the latest version is installed.
repo(string: null) - Chart repository URL where to locate the requested Chart.
set(string: null) - A dictionary/map consisting of configuration values to provide to Helm. Keys should be specified as strings. Values may be either Integers or Strings. If more complex values are required, use
valuesContent(string: null) - A list of configuration values to provide to Helm. This should be a multi-line string.
Below is an example of a HelmChart resource definition with all available configuration values defined.
--- apiVersion: helm.cattle.io/v1 kind: HelmChart metadata: name: example-helmchart namespace: kube-system spec: chart: test-helmchart targetNamespace: test-namespace version: 0.0.1 repo: https://example.com/helm-charts set: key1: "value1" key2: "value2" valuesContent: |- config: application: admin_username: admin admin_password: insecure_password server: port: 8080 tls_enabled: false
The corresponding CLI arguments for this would be:
$ helm install test-helmchart --namespace test-namespace \ --version 0.0.1 \ --repo https://example.com/helm-charts \ --set-string key1=value1 \ --set-string key2=value2 \ --values values.yaml
Where values.yaml contains the content present under the
valuesContent may be used to configure the chart.
set only supports strings or integers as values. If other data types are needed, use
Creating a HelmChart
We will now walk through the process of creating a HelmChart resource definition to deploy Node-RED – a flow-based programming for the Internet of Things.
The following steps were performed on a Raspberry Pi 4 running the Raspian operating system. You may need to modify the commands if using a different environment.
First, create a file named
node-red-helmchart.yaml using your default editor.
$ editor node-red-helmchart.yaml
Copy and paste the following block of text into the file.
--- apiVersion: helm.cattle.io/v1 kind: HelmChart metadata: name: node-red namespace: kube-system spec: chart: stable/node-red targetNamespace: node-red set: image.tag: rpi valuesContent: |- ingress: enabled: "true" annotations: kubernetes.io/ingress.class: traefik traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip hosts: - raspberrypi1.local path: /node-red/
Note: Be sure to modify the
ingress.hosts field to contain the hostname and/or IP address of one or more node in your Kubernetes cluster.
The deployment of the chart has been customized by modifying several of the available configuration values. Particularly, Ingress support has been enabled which exposes Node-RED to hosts outside of the Kubernetes cluster at http://raspberrypi1.local/node-red/.
Installing a HelmChart
There are two methods to install a HelmChart in k3s.
- Install the resource definition using
- Copy the resource definition to the server's manifest directory at
Installing via kubectl
Instruct Kubernetes to install the specified resource definition using
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml helmchart.helm.cattle.io/node-red created
This is the standard method of provisioning resources within Kubernetes.
Installing using Auto-Deploying Manifests
k3s includes a feature called Auto-Deploying Manifests which allows for the automated deployment of Helm charts (and Kubernetes manifests) placed in k3s manifest directory (
/var/lib/rancher/k3s/server/manifests). Helm Controller continually watches this directory for new CRDs. When it detects a new resource has been added, it will automatically deploy the chart. Therefore, you can initiate the installation by simply copying
node-red-helmchart.yaml into the server's manifest directory.
$ sudo cp node-red-helmchart.yaml /var/lib/rancher/k3s/server/manifests/node-red-helmchart.yaml
Installation process under the hood
Regardless of the installation method used, when a chart is deployed Helm Controller creates a Kubernetes Job which installs the chart into the cluster using
kubectl get jobs in the kube-system namespace and search for Jobs associated with the deployed chart.
$ sudo k3s kubectl --namespace kube-system get jobs \ --selector helmcharts.helm.cattle.io/chart=node-red NAME COMPLETIONS DURATION AGE helm-install-node-red 0/1 21s 23s
You can view the command the Job is executing by re-running
kubectl get jobs, and appending additional arguments to select and display Job-specific information using kubectl's custom-column output.
$ export CUSTOM_COLUMN_OUTPUT="COMMAND:.spec.template.spec.containers.name,INSTALL_ARGS:.spec.template.spec.containers.args" $ sudo k3s kubectl --namespace kube-system get job helm-install-node-red \ --output=custom-columns=$CUSTOM_COLUMN_OUTPUT COMMAND INSTALL_ARGS helm [install --name node-red stable/node-red --namespace node-red --set-string image.tag=rpi]
The Job installs the chart by executing
helm install with the list of installation arguments provided in the HelmChart resource, as noted earlier.
Verifying the chart installation
Verify the chart has been successfully deployed by ensuring the associated resources have been created within the node-red namespace.
$ sudo kubectl --namespace node-red get ingress,service,pods NAME HOSTS ADDRESS PORTS AGE ingress.extensions/node-red raspberrypi1.local 10.0.0.133 80 2m2s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/node-red ClusterIP 10.43.135.177 <none> 1880/TCP 2m2s NAME READY STATUS RESTARTS AGE pod/node-red-7c5c564cb6-sz8b8 1/1 Running 0 2m1s
Updating a HelmChart
The configuration for a chart may be updated using
$ sudo k3s kubectl apply --filename node-red-helmchart.yaml
Deleting a HelmChart
To delete a Helm Chart, use
kubectl delete helmchart.
$ sudo k3s kubectl --namespace kube-system delete helmchart node-red
You have learned about the HelmChart custom resource available in K3s, how to construct a HelmChart resource definition, install it on the cluster, and perform post-deployment operations (update/delete).
That's it for this walkthrough.
I have a written program,
k3s-helmchart-generate, which simplifies the process of building HelmChart manifests. You can find a link to the repo on GitHub below.
Top comments (1)
Great article. I'm curious if you had to pre-create the 'node-red' namespace for this to work. I'm trying to deploy cert-manager in the same way but it fails due to the name space not already existing.