Sveltos: Simplifying Kubernetes Add-on Deployment and Constraints

Sveltos, an open-source project, enables the deployment of Kubernetes add-ons across multiple clusters. It provides support for various deployment mechanisms such as Helm charts, kustomize resources, and resource YAMLs. With Sveltos, add-ons can be fetched from diverse sources for enhanced flexibility.

When managing add-ons across numerous clusters, it becomes crucial to validate and enforce constraints before their deployment. In this article we will delve into how Sveltos simplifies this process. With Sveltos, add-ons undergo rigorous validation to ensure they meet the specified criteria and adhere to the defined constraints. By enforcing these constraints, Sveltos guarantees the maintenance of consistency and reliability in add-on management across all clusters.

Add-on deployment

To deploy add-ons with Sveltos, all you need to do is define which Kubernetes add-ons to deploy and where to deploy them:

1. Select one or more clusters using a Kubernetes label selector.
2. List the Kubernetes add-ons that need to be deployed on the selected clusters.

Add-on Constraints

When deploying numerous add-ons across multiple clusters, it becomes crucial to ensure that all deployed add-ons adhere to certain constraints. These constraints can vary between clusters or sets of clusters, with production clusters typically having stricter requirements compared to non-production ones.

OpenAPI allows you to define and describe the expected structure, properties, and constraints of the add-on configurations. By leveraging OpenAPI, you can specify the required parameters, data types, format validations, and any additional constraints necessary for the add-ons.

By defining configuration constraints using OpenAPI, you establish a standardized and validated configuration schema that can be utilized during the deployment and configuration process.

Add-on Constraints with Sveltos

With Sveltos, defining constraints and specifying their enforcement becomes effortless. For example, you can define a ConfigMap that contains an OpenAPI policy to establish constraints. Here’s an example of the ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: openapi-deployment
  namespace: default
data:
  openapi.yaml: |
    openapi: 3.0.0
    info:
      title: Kubernetes Replica Validation
      version: 1.0.0

    paths:
      /apis/apps/v1/namespaces/{namespace}/deployments:
        post:
          parameters:
            - in: path
              name: namespace
              required: true
              schema:
                type: string
                minimum: 1
              description: The namespace of the resource
          summary: Create/Update a new deployment
          requestBody:
            required: true
            content:
              application/json:
                schema:
                  $ref: '#/components/schemas/Deployment'
          responses:
            '200':
              description: OK
            '400':
              description: Invalid Deployment. Each deployment in a production cluster requires at least 3 replicas

    components:
      schemas:
        Deployment:
          type: object
          properties:
            metadata:
              type: object
              properties:
                name:
                  type: string
            spec:
              type: object
              properties:
                replicas:
                  type: integer
                  minimum: 3

Once the ConfigMap is defined, you can create an AddonConstraint Custom Resource Definition (CRD) instance to enforce the constraints. The clusterSelector field specifies the target cluster(s) where the constraints should be enforced. The openAPIValidationRefs field references the resources (such as ConfigMap, Secret, GitRepository, etc.) that contain the OpenAPI policies. Here's an example of the AddonConstraint CRD instance:

apiVersion: lib.projectsveltos.io/v1alpha1
kind: AddonConstraint
metadata:
 name: depl-replica
spec:
  clusterSelector: env=prod
  openAPIValidationRefs:
  - namespace: default
    name: openapi-deployment
    kind: ConfigMap

The example provided demonstrates how Sveltos enforces a constraint that requires each deployment within a production cluster to have a minimum of 3 replicas. With this constraint in place, Sveltos ensures that the Kyverno Helm chart is not deployed unless the replicas for each deployment contained within the Helm chart are set to 3.

In testing clusters where no constraint is enforced, the deployment of the Helm chart will succeed regardless of the replica count specified for the deployments. This flexibility allows for easier experimentation and testing in non-production environments.

By leveraging Sveltos, you can enforce cluster-specific constraints, such as replica counts, to maintain consistency and compliance in your production clusters while having the flexibility to adjust these constraints in different environments.

Choosing this Approach over Using an Admission Controller

Let’s explore the advantages of choosing the approach provided by Sveltos for add-on deployment and constraint enforcement over relying on an admission controller like Kyverno or OPA.

1. Synchronization without Hassle: When using an admission controller, it is crucial to ensure that no add-ons are deployed until the controller is up and running. This often requires implementing synchronization mechanisms to coordinate the deployment process. With Sveltos, this is taken care of for you. The add-on controller in Sveltos patiently waits for the add-on constraint controller to load all existing constraints specific to each cluster. This guarantees a smooth and orderly deployment process without the need for additional synchronization mechanisms.

2. Consistency in Resource Deployment: When deploying resources using an Helm chart or similar mechanisms, multiple resources are typically deployed together as part of a cohesive unit. With the approach offered by Sveltos, a strict rule applies: either all the resources in the deployment satisfy the existing constraints and are valid, or none of them are deployed. This ensures consistency and prevents partial or incomplete deployments, leading to a more reliable and predictable deployment process.

👏 Support this project

If you like the project, please give us a 🌟 if you haven’t done so yet. Your support means a lot to us. Thank you 🙏.