Upgrading Kubernetes addons automatically as cluster runtime state changes

Kubernetes itself is not a complete solution. To build a production cluster, you need various additional addons. If you are managing multiple clusters, that is not an easy task.
Sveltos wants to figure out the best way to install, manage and deliver cluster addons. It does so by providing declarative APIs allowing you to deploy Kubernetes addons across multiple Kubernetes clusters.

Sveltos was introduced here. In this post I am going to deep dive into how Sveltos can automatically react to cluster runtime state changes.

Sometimes what addons you need and/or their versions depend on cluster runtime state. For instance, let's say you want to:
deploy a given version of Gatekeeper on any cluster running a Kubernetes version v1.24.x;
different version of Gatekeeper on any cluster running a kubernetes version v1.25.x.
You could manually upgrade Gatekeeper any time you upgrade a cluster. But that is error prone (you might forget) or might require some coordination (the admin managing the cluster Kubernetes and the admin managing Gatekeeper might be different admins).
Sveltos introduced Classifier to help with that.

Classifier allows you to define a set of constraints. Each constraint can be based on:

1. Cluster Kubernetes version;
2. Any resource deployed in the cluster (filtering by labels and/or properties is supported).

Classifier also allows you to define the set of labels that need to be added to a cluster if its runtime state matches the Classifier instance.

Some examples here.

Once a Classifier instance is created, a Sveltos agent running in each managed cluster, keeps watching the cluster runtime state. When it detects a cluster runtime state is a match, that information is sent back to the management cluster. Sveltos will then update cluster labels accordingly. As soon as cluster labels are modified, cluster might start matching a new ClusterProfle and so Kubernetes addons are upgraded automatically.

Combing Classifier with ClusterProfile, you can have:

1. Sveltos watch for each cluster runtime state;
2. Sveltos update cluster labels as cluster runtime state changes;
3. Sveltos deploy/upgrade Kubernetes addons.

Some use cases:

1. upgrade Kubernetes helm charts as soon as cluster Kubernetes version is upgraded;
2. deploy new addons as soon as certain resources are deployed in a cluster;
3. automatically remove an hot fix when cluster is upgraded.

Sveltos is an open source project. Contributions are more welcome.

If you would like to know more about it or would like to see a new feature added to sveltos, please reach to us on slack. Any feedback is very much appreciated.