Introduction
In the previous post, we demonstrated how easy it is to deploy ArgoCD with the Cilium Gateway API and move away from the Kubernetes Ingress. In today’s post, we will explore how the Projectsveltos Event Framework can be utilised to seamlessly perform the same deployment based on an event.
The Sveltos Event Framework was designed to automate the deployment of Kubernetes add-ons triggered on specific cluster events. This functionality proves invaluable in managing environments spanning across multiple clusters.
Sveltos supports an event-driven workflow with the below steps:
- Define what the event is
- Select the clusters to watch for such events
- Define an event trigger. Which add-ons/applications to deploy when the events occur
Once we identify the event we want Sveltos to watch for, we can deploy the EventSource and EventTrigger Kubernetes CRDs (Custom Resource Definitions). More information about these resources can be found later on.
Diagram
Prerequisites
To follow along, it is strongly advised to take a look at a previous post detailing the deployment of Sveltos on a cluster. To install Sveltos, just follow the instructions outlined in steps 1 and 2 of the mentioned post. Additionally, as we will use the Cilium Gateway API, have a look at the previous post to enable this capability on an RKE2 cluster.
Lab Setup
— — — — — -+ — — — — — — — — — — + — — — — — — — — — — — — -+
| Cluster Name | Type | Version |
+ — — — — — — -+ — — — — — — — — — — + — — — — — — — — — — -+
| rke2-mgmt01 | Management Cluster | RKE2 v1.26.12+rke2r1 |
| rke2-sles-demo | Managed Cluster | RKE2 v1.26.12+rke2r1|
+ — — — — — -+ — — — — — — — — — — + — — — — — — — — — — — — -+
- — — — — — -+ — — — — -+
| Deployment | Version |
+ — — — — — — -+ — — — — -+
| ArgoCD | v2.9.3 |
| Cilium | v1.14.5 |
| GatewayAPI | v0.7.0 |
+ — — — — — — -+ — — — — -+
Step 1: Register and label the RKE2 cluster with Sveltos
Once the RKE2 cluster is in a “Running” state, we will use the sveltosctl to register it. For the registration, we need three things: a service account, a kubeconfig associated with that account and a namespace. If you are unsure how to create a Service Account and an associated kubeconfig, there is a script publicly available to help you out.
Registration
$ sveltosctl register cluster --namespace=projectsveltos --cluster=rke2-sles-demo --kubeconfig=rke2-sles-demo.yaml
Verification
$ kubectl get sveltoscluster -n projectsveltos
NAME READY VERSION
rke2-sles-demo true v1.26.12+rke2r1
Assing Label
$ kubectl label sveltoscluster rke2-sles-demo env=staging -n projectsveltos
Verification
$ kubectl get sveltoscluster -n projectsveltos --show-labels
NAME READY VERSION LABELS
rke2-sles-demo true v1.26.12+rke2r1 env=staging,sveltos-agent=present
For the demonstration, we will use the label ‘env=staging’ to specify the cluster we want to deploy the Gateway and the HTTPRoute resources.
Step 2: Pre-Work (management cluster)
Before we move on with the Gateway API implementation, we need to create additional Kubernetes resources for the managed cluster (rke2-sles-demo).
- ArgoCD namespace
- ArgoCD TLS Secret
- Cilium IP Pool
- Cilium GatewayClass
- Gateway
- HTTPRoute
To utilise the power of Sveltos, we will create the above resources in the management cluster as a Secret and configMap resource and use the Sveltos ClusterProfile to deploy them to the managed cluster.
More information on how to create the above resources can be found here.
ArgoCD Namespace
If the namespace does not exist in the managed cluster, Sveltos will create it!
ArgoCD TLS Secret
apiVersion: v1
data:
tls.crt: {base64 encoded .crt}
tls.key: {base64 encoded .key}
kind: Secret
metadata:
name: argocd-server-tls
namespace: argocd
type: kubernetes.io/tls
$ kubectl create secret generic argocd-server-tls --from-file=argocd_server_tls.yaml --type=addons.projectsveltos.io/cluster-profile
Cilium IP Pool
---
apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
name: "rke2-pool"
spec:
cidrs:
- cidr: "10.10.10.0/24"
$ kubectl create configmap ipampool --from-file=ipam-pool.yaml
Cilium GatewayClass
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
name: cilium
spec:
controllerName: io.cilium/gateway-controller
$ kubectl create configmap gatewayclass --from-file=gatewayclass.yaml
Gateway
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: argocd
namespace: argocd
spec:
gatewayClassName: cilium
listeners:
- hostname: argocd.example.com
name: argocd-example-com-http
port: 80
protocol: HTTP
- hostname: argocd.example.com
name: argocd-example-com-https
port: 443
protocol: HTTPS
tls:
certificateRefs:
- kind: Secret
name: argocd-server-tls
$ kubectl create configmap gateway --from-file=gateway.yaml
HTTPRoute
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
creationTimestamp: null
name: argocd
namespace: argocd
spec:
hostnames:
- argocd.example.com
parentRefs:
- name: argocd
rules:
- backendRefs:
- name: argocd-server
port: 80
matches:
- path:
type: PathPrefix
value: /
status:
parents: []
$ kubectl create configmap gateway --from-file=httproute.yaml
Verification
$ kubectl get secret
NAME TYPE DATA AGE
argocd-server-tls addons.projectsveltos.io/cluster-profile 1 20m
$ kubectl get cm
NAME DATA AGE
gateway 1 35s
gatewayclass 1 20m
httproute 1 3s
ipam-pool 1 22m
kube-root-ca.crt 1 141m
ClusterProfile
---
apiVersion: config.projectsveltos.io/v1alpha1
kind: ClusterProfile
metadata:
name: cilium-gateway-api
spec:
clusterSelector: env=staging
syncMode: Continuous
policyRefs:
- name: ipampool
namespace: default
kind: ConfigMap
- name: gatewayclass
namespace: default
kind: ConfigMap
- name: argocd-server-tls
namespace: default
kind: Secret
The ClusterProfile definition will match the managed cluster with the label selector set to ‘env=staging’ and deploy the Cilium IP Pool, the GatewayClass and the ArgoCD TLS secret.
Apply ClusterProfile
$ kubectl apply -f "clusterprofile.yaml"
Verification
$ sveltosctl show addons
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
| CLUSTER | RESOURCE TYPE | NAMESPACE | NAME | VERSION | TIME | PROFILES |
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
| projectsveltos/rke2-sles-demo | cilium.io:CiliumLoadBalancerIPPool | | rke2-pool | N/A | 2024-02-11 17:39:58 +0100 CET | ClusterProfile/cilium-gateway-api |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:GatewayClass | | cilium | N/A | 2024-02-11 17:39:58 +0100 CET | ClusterProfile/cilium-gateway-api |
| projectsveltos/rke2-sles-demo | :Secret | argocd | argocd-server-tls | N/A | 2024-02-11 17:39:58 +0100 CET | ClusterProfile/cilium-gateway-api |
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
Step 3: Define the Event Framework (management cluster)
As mentioned at the beginning of the post, we will use the EventSource to define which events Sveltos will watch for and the EventTrigger resource to perform actions based on an event.
EventSource
---
apiVersion: lib.projectsveltos.io/v1alpha1
kind: EventSource
metadata:
name: http-https-service
spec:
collectResources: true
resourceSelectors:
- group: ""
version: "v1"
kind: "Service"
namespace: argocd
evaluate: |
function evaluate()
hs = {}
hs.matching = false
if obj.spec.ports ~= nil then
for _,p in pairs(obj.spec.ports) do
if p.port == 80 or p.port == 443 then
hs.matching = true
end
end
end
return hs
end
The EventSource uses the Lua language to search for any services with ports set to 80 or 443 in the ‘argocd’ namespace. More examples can be found here.
EventTrigger
---
apiVersion: lib.projectsveltos.io/v1alpha1
kind: EventTrigger
metadata:
name: service-network-policy
spec:
sourceClusterSelector: env=staging
eventSourceName: http-https-service
oneForEvent: true
policyRefs:
- name: httproute
namespace: default
kind: ConfigMap
- name: gateway
namespace: default
kind: ConfigMap
Apply Resources
$ kubectl apply -f "eventsource.yaml"
eventsource.lib.projectsveltos.io/loadbalancer-service created
$ kubectl apply -f "eventtrigger.yaml"
eventtrigger.lib.projectsveltos.io/service-network-policy created
Once an event is spotted by Sveltos, the HTTPRoute and Gateway Kubernetes add-ons will get deployed on the managed cluster with the label selector set to ‘env=staging’. In our case, the cluster ‘rke2-sles-demo’ will match the ClusterSelector and will automatically get the specified resources.
Step 4: Deploy ArgoCD Manifests (managed cluster)
Now, it is time to deploy the ArgoCD manifests and wait Sveltos to notice the event and trigger the deployment of the HTTPRoute and Gateway resources in the ‘argocd’ namespace.
$ kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Step 5: Verification and Tests
As we have installed the ArgoCD manifests, the ‘argocd-server’ service should have been created in the ‘argocd’ namespace. That means the Sveltos EventSource should have been activated and triggered the deployment of the HTTPRoute and Gateway Kubernetes resources.
Sveltos Verification
$ sveltosctl show addons
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
| CLUSTER | RESOURCE TYPE | NAMESPACE | NAME | VERSION | TIME | PROFILES |
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
| projectsveltos/rke2-sles-demo | cilium.io:CiliumLoadBalancerIPPool | | rke2-pool | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/cilium-gateway-api |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:GatewayClass | | cilium | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/cilium-gateway-api |
| projectsveltos/rke2-sles-demo | :Secret | argocd | argocd-server-tls | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/cilium-gateway-api |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:HTTPRoute | argocd | argocd | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/sveltos-1lbzxnp4cs96gpo38g8b |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:Gateway | argocd | argocd | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/sveltos-1lbzxnp4cs96gpo38g8b |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:HTTPRoute | argocd | argocd | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/sveltos-i0nmkq3ebsvpis8ubd3s |
| projectsveltos/rke2-sles-demo | gateway.networking.k8s.io:Gateway | argocd | argocd | N/A | 2024-02-11 19:55:52 +0100 CET | ClusterProfile/sveltos-i0nmkq3ebsvpis8ubd3s |
+-------------------------------+----------------------------------------+-----------+-------------------+---------+-------------------------------+---------------------------------------------+
$ sveltosctl show usage
+----------------+--------------------+------------------------------+-------------------------------+
| RESOURCE KIND | RESOURCE NAMESPACE | RESOURCE NAME | CLUSTERS |
+----------------+--------------------+------------------------------+-------------------------------+
| ClusterProfile | | cilium-gateway-api | projectsveltos/rke2-sles-demo |
| ClusterProfile | | sveltos-fvc3vr84yqusfzwpobcl | projectsveltos/rke2-sles-demo |
| ConfigMap | default | gateway | projectsveltos/rke2-sles-demo |
| ConfigMap | default | ipampool | projectsveltos/rke2-sles-demo |
| ConfigMap | default | gatewayclass | projectsveltos/rke2-sles-demo |
| ConfigMap | default | httproute | projectsveltos/rke2-sles-demo |
+----------------+--------------------+------------------------------+-------------------------------+
Managed Cluster Verification
$ kubectl get gateway,httproute -n argocd
NAME CLASS ADDRESS PROGRAMMED AGE
gateway.gateway.networking.k8s.io/argocd cilium 10.10.10.147 True 78s
NAME HOSTNAMES AGE
httproute.gateway.networking.k8s.io/argocd ["argocd.example.com"] 78s
Test ArgoCD Accessibility
curl -ki https://argocd.example.com
HTTP/1.1 200 OK
accept-ranges: bytes
content-length: 788
content-security-policy: frame-ancestors 'self';
content-type: text/html; charset=utf-8
vary: Accept-Encoding
x-frame-options: sameorigin
x-xss-protection: 1
date: Sun, 11 Feb 2024 18:59:57 GMT
x-envoy-upstream-service-time: 0
server: envoy
<!doctype html><html lang="en"><head><meta charset="UTF-8"><title>Argo CD</title><base href="/"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" type="image/png" href="assets/favicon/favicon-32x32.png" sizes="32x32"/><link rel="icon" type="image/png" href="assets/favicon/favicon-16x16.png" sizes="16x16"/><link href="assets/fonts.css" rel="stylesheet"><script defer="defer" src="main.9ecae91d8fd1deedf944.js"></script></head><body><noscript><p>Your browser does not support JavaScript. Please enable JavaScript to view the site. Alternatively, Argo CD can be used with the <a href="https://argoproj.github.io/argo-cd/cli_installation/">Argo CD CLI</a>.</p></noscript><div id="app"></div></body><script defer="defer" src="extensions.js"></script></html>
Note: As we use a self-sign certificate the argocd-cmd-params-cm ConfigMap in the argocd namespace should include the following setting server.insecure: “true”.
As expected, Sveltos took care of the complete lifecycle of the different Kubernetes deployments in a simple and straightforward manner! The example might seem simplistic but think about it on a bigger scale with hundreds of clusters across a multicloud setup. Imagine the flexibility of setting Sveltos labels to a cluster, defining the event source to trigger an action and having a Kubernetes deployment on a cluster. In the next post, we will demonstrate the power of Events in a cross multi-cluster environment.
👏 Support this project
Every contribution counts! If you enjoyed this article, check out the Projectsveltos GitHub repo. You can star 🌟 the project if you find it helpful.
The GitHub repo is a great resource for getting started with the project. It contains the code, documentation, and many more examples.
Thanks for reading!
Top comments (0)