Deploy Kubernetes add-ons: statically and dynamically

Kubernetes itself is not a complete solution. To build a production cluster, you need various additional add-ons. If you are managing multiple clusters, that is not an easy task.

Sveltos is an open source project that provides declarative APIs allowing to deploy Kubernetes add-ons across multiple Kubernetes clusters. Both Helm charts and resource YAMLs can be passed to Sveltos.

For instance, just posting following ClusterProfile instance is sufficient to deploy Kyverno Helm chart in a set of managed clusters:

Or if you have Kubernetes resource YAMLs, just create a ConfigMap (or Secret if data is confidential) with the data and then have Sveltos reference such ConfigMap (Secret) instance.

kubectl create configmap contour-gateway --from-file=<File with contour YAML>

To make things more complex, some of the time, the add-ons that you need to deploy depends on cluster run-time state.

For instance, you have deployed Calico v3.24 in a set of clusters. As those clusters get upgraded to Kubernetes v1.25, you want Calico to be upgraded to v3.25 as well.

If you are managing tens of such clusters, manually upgrading Calico when Kubernetes version is upgraded is not ideal. You need an automated solution for that.

Sveltos cluster classification offers a solution for such scenario. Define two ClusterProfiles:

• one ClusterProfile instance will deploy calico v3.24.5 in any cluster with label kubernetes: v1–24

• other ClusterProfile instance will deploy calico v3.25.0 in any cluster with label kubernetes: v1–25

Then simply create following Classifier instances:

Above Classifier instances will have Sveltos manage Cluster labels by automatically adding:

• label kubernetes: v1–24 to any cluster running Kubernetes version v1.24.x

• label kubernetes: v1–25 to any cluster running Kubernetes version v1.25.x.

Because of those labels and the above ClusterProfile instances:

• calico version v3.24.5 will be deployed in any cluster running Kubernetes version v1.24.x

• calico version v3.25.0 will be deployed in any cluster running Kubernetes version v1.25.x

No action is required on your side. As clusters are upgraded, Sveltos will upgrade Calico as well.

Things might get more complicated when the add-ons need to be deployed as result of an event in a managed cluster. For instance, any time a Service in certain namespace is created, adds an HTTPRoute to expose such a service via Gateway API.

Sveltos Events is an event-driven workflow automation framework for Kubernetes which helps you trigger K8s add-on deployments on various events.

1. Define what an Event is (Sveltos supports Lua script for that);
2. Define what add-ons to deploy when such an event happen. Add-ons can be expressed as template, and Sveltos will instantiate those at deployment time using information from managed clusters.

This EventSource is defining an event as the creation/deletion of a Service in the namespace eng exposing either port 443 or port 8443.

When such an event happens in a managed cluster, we want to deploy an HTTPRoute instance:

As you can see, what to deploy is defined in a ConfigMap and it is expressed as a template.

Resources contains all the Service instances in each managed cluster matching the EventSource defined above.

To know more about Sveltos event driven framework please see documentation.

Sveltos is an open source project. We would love your suggestions, contributions, and help!

If you would like to know more about it or would like to see a new feature added to sveltos, please reach to us on slack. Any feedback is very much appreciated.