I think lots have been written about, and everyone intuitively understands the idea of a service dependency hierarchy. Some services are critical lower tiered services, that everything depends on, and some services are less critical higher tiered services. So, like the network is a lower tier service that everything depends on, and DNS depends on the network, and your service depends on DNS to communicate with other services like your database. Your database is more critical then your service, as maybe multiple services talk to the same database, and so your database shouldn't have a dependency on your service, and thus go down when your service goes down, likewise, your network shouldn't be dependent on any of your services, especially if those services are considered non-critical, or low SLA, because then your network adopts the same SLA as a non-critical service.
I think there's more to unpack here, as we think about DevOps and automation. ArgoCD is a relatively heavyweight service, at least as compared to a command-line tool like Terraform. Should critical services like Karpenter or Ist.IO be deployed using, and thus have a dependency on a less critical service like ArgoCD? You could define ArgoCD as a more critical service, but then you've increased the load on the teams that have to manage it. And maybe ArgoCD isn't even architected to be a tier-0 service. There are some vulnerabilities in the ways that CRD's are managed that make it such that uninstalling Argocd might uninstall or make your service unmanageable.
The specific question that got me going in this route was around automatically creating Alerts for critical EKS services. Wouldn't it be great if installing a critical service automatically created the alerts to tell you when it was down. I have a helm chart that does this, but what if we could abstract this functionality behind a platform engineering workflow tool like Koreo or KPT?
Well now we've taken a fairly critical function, that is core service alerting, and made it dependent on a possibly much less critical service, Koreo. If Koreo goes down, then we're not creating core service alerts, including possibly the alert for telling when Koreo goes down!
The point being that there's a lot of fun magic in automating Kubernetes and Platform engineering, but some thought has to be given to the dependency hierarchy. Maybe you only want one deploy tool -- Crossplane or ArgoCD, but then who watches the watchmen? There should always be a critical tier later that has as few dependencies as possible for bootstrapping, and later operating the core features of the infrastructure - say the Kubernetes Environment.
For sure, this advice is variable. I use Kubernetes core services as an example, but if you have automation to create and manage the lifecycle of kubernetes clusters, than maybe even core services like CoreDNS aren't those top tier-0 services, because the management of CoreDNS on a leaf Kubernetes cluster isn't dependent on anything about that Kubernetes Cluster itself, it's dependent on something upstream functioning that may itself be tier0.
Top comments (0)