DEV Community: Jan Lepsky

Pulumi vs. Terraform: Choosing the Best Infrastructure as Code Solution

Jan Lepsky — Mon, 10 Feb 2025 16:19:08 +0000

This article was originally posted on the mogenius blog by Ben Force.

Building an application in the cloud can become complex quickly. There are multiple cloud providers, each with hundreds of different resources. Even the same type of resources on different providers can have their unique quirks. All of this complexity makes defining your infrastructure – and deploying it – a complex task of its own.
‍
Over the years, two infrastructure as code (IaC) tools have become popular solutions to this problem: Terraform and Pulumi. Terraform is the more mature of the two, with a huge community and support for a lot of resource types. Pulumi is more of a newcomer, trying to make it easier for application developers to define infrastructure.

This article compares the two tools based on the following:‍

‍Basic use: focusing on programming models, the learning curve, and testing‍
Behind the scenes: comparing their state management and performance
Support: looking at available cloud providers, the ecosystem, and the community
Use cases and best-fit scenarios: highlighting when you should use one over the other

Basic Use

In this first section, you'll explore how day-to-day interactions with the two tools vary, and you'll begin to understand the different types of developers each tool caters to.

Programming Model
The first notable difference between Pulumi and Terraform is the programming language. Terraform uses a custom language called HashiCorp configuration language (HCL) to define infrastructure. In contrast, Pulumi offers more options, allowing you to choose from a list of popular programming languages.

The two different approaches create a trade-off between power and simplicity. Since Pulumi uses general-purpose programming languages, you can implement complex logic and abstractions. While HCL has few advanced features by comparison, its simple declarative nature allows you to look at a terraform file and easily understand what it's created.

Both Terraform and Pulumi are improving their programming models so they're more accessible. Terraform now offers CDK for Terraform, which allows you to define your infrastructure using a common programming language, and Pulumi has a YAML-based configuration option.

Learning Curve and Developer Experience
Both Terraform and Pulumi require you to understand the cloud resources you're creating. At the time of writing, AWS lists 246 services that each have multiple resource types you can create. On top of that, you need to understand how to give each of these resources permission to access other resources and how to connect them together. So if you're new to infrastructure development, there's a bit of a learning curve before you start using either tool.

If you're a software engineer familiar with cloud resources, Pulumi will be a natural fit. As mentioned, it supports multiple popular programming languages, such as TypeScript and Python, so there's likely one you're already familiar with. Pulumi also offers Pulumi AI, which allows you to describe the infrastructure you want, and it will generate the code needed to create it.

Terraform, on the other hand, has a steeper initial learning curve since you'll have to learn HCL. Creating simple configurations doesn't take long to figure out, but the syntax for dynamically creating multiple resources can take longer to get used to.

Testing and Validation
Terraform and Pulumi both offer testing to make sure you're deploying the resources that you expect to.

Pulumi supports three different types of tests: unit, integration, and property. You can write unit and integration tests for your project using the testing framework of your choice. It even offers tools to mock resources. Integration tests deploy a temporary copy of your defined infrastructure and then run your tests against the deployed resources. Unit tests mock any calls to external resources so they run much faster.

Pulumi's property tests allow you to define company-wide policies and check all your infrastructure to ensure compliance. However, these tests are only supported in a few languages and don't use standard testing frameworks.

Terraform provides a built-in testing framework to validate your project. The testing framework supports both unit and integration tests. It handles unit tests by creating a deployment plan and running the tests against that plan. While Terraform's testing framework allows you to validate basic properties, you may need to write more advanced tests to verify your virtual machines are set up correctly. You can create advanced tests using open source testing tools like Terratest and Kitchen-Terraform.

Behind the Scenes

Now that you've seen some of the trade-offs in the everyday use of the two tools, let's look at how they differ behind the scenes, focusing on their state management and performance.
‍
State Management
Every time you change your IaC project, both Terraform and Pulumi compare the desired end state with the current state. To do this, they store the state of every resource after each deployment as a reference for the next deployment. Both Pulumi and Terraform offer multiple methods to store this state.

Pulumi offers a managed service, a self-hosted service, and the option to store the state in an object store like AWS S3. All of these state-storage backends support state locking, which ensures only one deployment can run at a time for a specific project. The lock is checked before a deployment is started. If one is found, the deployment will wait until the lock is released. This allows you to safely execute multiple deployments at the same time by ensuring only one set of changes is applied at a time.

Terraform stores its state in a local file by default, but you can configure your project to use several different services, like AWS S3, using the backend block. Alternatively, if you subscribe to HashiCorp's cloud service, you can configure it using the cloud block. While Terraform offers many ways to store your state, not all of them support locking.

Performance and Scalability
Pulumi handles large updates efficiently by relying on its saved state when planning updates. When applying the updates, it runs all possible change operations in parallel.

Similarly to Pulumi, Terraform applies updates in parallel. By default, however, it will make requests to refresh the state of every resource for each plan and apply operation. If you have an exceptionally large project, Terraform can cache all resource properties in the state file and use it as the source of truth instead of constantly making calls to your cloud provider to check resource properties on every deployment.

Support

This section looks at Terraform's and Pulumi's support for different cloud providers and resources and how you can get help when you need it.

Cloud Provider Support
Terraform has been around since 2014 and has built up solid support for different providers, which have been developed by the service providers themselves, HashiCorp, and third-party developers. As of right now, there are almost 4,500 providers available in the Terraform Registry.

In contrast, Pulumi has only about 160 providers available in the Pulumi registry. However, since the major cloud providers are supported, this should be enough. If Pulumi lacks a provider for a specific resource that Terraform supports, you can create a Pulumi Terraform bridge. These bridges can adapt to any Terraform provider for use with Pulumi. While this process isn't as painless as adding a provider to your dependencies, it's easier than creating a new provider from scratch.

Pulumi is also currently testing a Terraform provider in beta testing that will automatically generate one of the Terraform bridge packages mentioned above, making the process a lot easier.

Ecosystem and Integrations
In addition to supporting a bunch of cloud providers, Pulumi has several first-party integrations that you can add to your CI/CD pipelines. These integrations not only deploy your infrastructure changes but also add comments to merge request reviews so you can easily see what will be changed when the code is committed.

As mentioned above, Terraform has many providers available. It has even more modules, about 18,000 at the time of writing. While this may seem like it makes Terraform the clear winner, remember that some of these modules already exist for your favorite programming language and don't need to be in the Pulumi registry. For example, a module to generate a UUID doesn't need to be in the Pulumi registry.

License
Pulumi is released under the Apache 2.0 license, which means you can build products using it and sell those products to customers. Terraform, on the other hand, used to be released under the Mozilla Public License but has since changed to the Business Source Licnese. This license still allows you to use Terraform internally, but if you want to build your own product on top of it, then you’re going to run into legal issues.

Community and Support
Both Terraform and Pulumi are backed by for-profit companies, which provides some assurance that they'll be able to support their products for a while. In comparison, HashiCorp is more stable since it was acquired by IBM, while Pulumi is a venture-funded startup.

Terraform is generally accepted as the de facto standard IaC solution, so it has extensive support options available. The internet is filled with years of Stack Overflow answers, blog posts, and sample projects on GitHub. If you can't find a solution in one of those places, you can probably find someone at your company with some Terraform experience.

Though it hasn't been around as long, Pulumi still has over 150,000 end users as of October 2023. While there are fewer overall resources, some of your experience in your chosen language can be used. They also have well-written documentation.

Both Terraform and Pulumi offer enterprise subscriptions that include support plans. Both charge based on the number of resources that you're managing. Terraform's Standard subscription offers support and costs about $0.10 per resource per month. To get a support plan for Pulumi, however, you have to splurge for the Enterprise subscription, which will set you back $1.10 per resource per month.

Use Cases and Best-Fit Scenarios

Now that you've seen some of the pros and cons of Terraform and Pulumi, how do you decide which one will work best in a given situation? It comes down to what kind of project you're creating.

Serverless Applications: Pulumi
If you have a TypeScript serverless application with several small functions, Pulumi is the way to go. It allows you to define your function's code in-line with the infrastructure definition, and it will handle the build step for you.

Applications with Virtual Networks: Terraform
When building a more traditional application with a lot of virtual networks and virtual machines, Terraform has a strong track record. Because it's been around for so long, it's likely that the people defining virtual networks will already be familiar with Terraform, making it a natural fit.

Custom Resources: Pulumi
Pulumi offers dynamic resource providers, which enable you to easily support custom resources by defining a few functions to perform CRUD operations. You can also use a dynamic resource provider to load test data into a developer environment.

Audited/Regulated Projects: Terraform
If you're developing a project for a regulated industry, Terraform can simplify the process of auditing your infrastructure. Due to its declarative nature, it's easier to understand what the code is doing. It also makes it more difficult for noncompliant changes to sneak in.

Quick Comparison

Here's a table that quickly summarizes each of the comparison points:

Conclusion

In this article, you've seen that Terraform and Pulumi aren't one-size-fits-all solutions. Each has its strengths and weaknesses that need to be considered depending on your situation. Terraform is the established solution that works best for a large CloudOps team. Pulumi, on the other hand, is a great solution if your backend (or full-stack) developers need to manage a project's infrastructure.

If you use Terraform or Pulumi to create a Kubernetes cluster, it doesn't help much beyond getting the computing resources in place. To gain better visibility into your Kubernetes services across multiple environments, check out mogenius. Its free plan offers a full feature set to deploy and troubleshoot applications on any Kubernetes cluster, and it also provides intuitive onboarding.

How and When to Use Terraform with Kubernetes

Jan Lepsky — Fri, 24 Jan 2025 15:19:13 +0000

This article was originally posted on the mogenius blog by Kovid Rathee.

Terraform is an infrastructure as code tool that replaces the ClickOps method of defining, deploying, and managing infrastructure locally, on-premises, or in the cloud. Its declarative method of defining infrastructure lets you focus on the target state of the infrastructure rather than the steps needed to achieve that state, which makes managing infrastructure easier.

Terraform has its own declarative language called the Hashicorp configuration language (also called the Terraform language) for defining the infrastructure you want, and its command line tool makes it easy to perform operations on your infrastructure.

In this article, you'll learn how and when to use Terraform with Kubernetes with an example of a local nginx deployment.

When to Use Terraform with Kubernetes‍

Terraform is a platform-agnostic tool, so you can use it to manage your entire stack irrespective of the underlying frameworks, libraries, cloud platforms, etc. It enables multicloud and hybrid-cloud development by bringing all your infrastructure definitions into one place, which makes managing infrastructure, costs, and security easier.

Combining Terraform with your version control system and CI/CD pipeline also helps you automate infrastructure deployment. If you store your Terraform project in a Git repository, any changes to the project are accompanied by the output of the Terraform plan describing the changes that the PR would bring to the infrastructure. The reviewer can then approve the code changes, triggering a terraform apply -auto-approve command.

Using Kubernetes with Terraform lets you define all your infrastructure – such as networking, storage volumes, databases, security groups, firewalls, DNS, etc. – within the Terraform project. Terraform comes in handy when you're dealing with all the infrastructure in multicloud and hybrid-cloud environments, especially when you're managing dependencies between different Kubernetes objects and cloud resources.

Using Terraform with Kubernetes

Let's now look at how to use Terraform with Kubernetes by walking through an example of setting up two separate nginx clusters.

For simplicity, you'll be deploying the clusters on your local machine, which is great for development, but in a real-world scenario, the clusters will likely be hosted in the cloud.

Setting Up the Environment

To use Terraform with Kubernetes, you'll need the following tools installed on your system:

Docker Engine: lays the foundation for deploying containers using Minikube on your local machine
Minikube: acts as your local Kubernetes cluster, which you'll use to run separate nginx deployments and services
kubectl: lets you access Kubernetes resources – such as pods, deployments, and services – from your command line
Terraform CLI: lets you work with a Terraform project from your command line (i.e. you can use this tool to plan, apply, and destroy infrastructure)
Git: the version control system that lets you download and manage the source code for the module you'll be working with in this tutorial

Ensure Docker Engine and Minikube are up and running. Use the following commands to start Minikube:

minikube start
minikube dashboard

You'll see an output resembling the following:

➜  ~ minikube dashboard
🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...
🎉  Opening http://127.0.0.1:57560/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...

‍
Make a note of the URL http://127.0.0.1:57560 as you'll need it later in this tutorial.

Check the status of your Docker machine by running the following command:

docker info

Lastly, clone this GitHub repository that contains the Terraform nginx module for this tutorial:

git clone https://github.com/kovid-r/terraform-nginx-k8s.git

‍
Though you can write all your Terraform code in a single .tf file, it's considered a bad practice for code management. In the repository, you'll find the code split into the following three files in the GitHub repository:

main.tf: contains the definitions of the Kubernetes namespace, deployment, and service
variables.tf: contains the definitions of Terraform variables, for which you'll provide values via parameters from the Minikube configuration file
versions.tf: contains the definition of the Kubernetes provider, along with its source and version

You can find more information on how files and directories in the GitHub repo can be structured.

Configuring the Terraform Kubernetes Provider

This tutorial uses Hashicorp's official Kubernetes provider to create the two local Kubernetes clusters you'll be using.

When you run the terraform init command for the first time, the following snippet from the versions.tf file in the GitHub repository will import the provider module into your local Terraform project:

terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.11" # Update based on latest stable version
    }
  }

  required_version = ">= 1.0.0"
}

Creating and Configuring a Terraform nginx Module

To deploy nginx using Kubernetes and Terraform, you first need to create a Terraform nginx module. Terraform lets you create reusable modules when creating new resources in your infrastructure. Although creating modules is easy, Terraform generally advises using them in moderation, especially when they don't create a new abstraction to your application architecture.

You could follow Terraform's official tutorial to build your own module, but for this tutorial, you'll use the prebuilt terraform-nginx-k8s module from the GitHub repository. This module is based on the official Kubernetes provider from the Terraform registry.

Before deploying the terraform-nginx-k8s module, let's walk through how to configure its core components and how it defines various Kubernetes resources.

Creating a Kubernetes Namespace for Deploying Resources

You can use Kubernetes namespaces to manage clusters and subclusters individually. The module uses the following code to create a new namespace:

## resource "kubernetes_namespace" "nginx" {
  metadata {
    name = var.namespace
  }
}

‍
This creates a new kubernetes_namespace with the default name set as nginx, which you can override in your variables.tf file. Later in the tutorial, you'll use this code to create two namespaces, nginx-cluster-one-ns and nginx-cluster-two-ns, one for each Kubernetes cluster. In each of those clusters, you'll create nginx deployments.

Defining the nginx Deployment

Next, the module defines the nginx deployment using the kubernetes_deployment resource:

resource "kubernetes_deployment" "nginx" {
  metadata {
    name      = "nginx-deployment"
    namespace = kubernetes_namespace.nginx.metadata[0].name
    labels = {
      app = "nginx"
    }
  }

  spec {
    replicas = var.replicas

    selector {
      match_labels = {
        app = "nginx"
      }
    }

    template {
      metadata {
        labels = {
          app = "nginx"
        }
      }

      spec {
        container {
          name  = "nginx"
          image = "nginx:${var.nginx_version}"

          port {
            container_port = 80
          }

          resources {
            limits = {
              cpu    = var.cpu_limit
              memory = var.memory_limit
            }
          }
        }
      }
    }
  }
}

This resource takes arguments – such as the namespace, number of replicas, nginx version, CPU limit, and memory limit – from the variables.tf file. You can override the default variable definitions when you call the module. For example, when you import the module to configure the Kubernetes deployment later in the tutorial, you'll define some of the variables as follows:

namespace                         = "nginx-cluster-one-ns"
nginx_version                     = "1.21.1"
replicas                          = 2
cpu_limit                         = "250"
memory_limit                      = "128Mi"

‍
This specifies the nginx version and the resources you want to provide the server.

Defining a Kubernetes Service for nginx

Finally, the module defines a Kubernetes service, an application that will run in your Kubernetes cluster behind a single outward-facing endpoint:

resource "kubernetes_service" "nginx" {
  metadata {
    name      = "nginx-service"
    namespace = kubernetes_namespace.nginx.metadata[0].name
  }

  spec {
    selector = {
      app = "nginx"
    }

    port {
      port        = 80
      target_port = 80
    }

    type = var.service_type
  }
}

Deploying the Module and Applying the Configuration

Now that you've run through all the core components of the Kubernetes nginx module, you can deploy it and apply your own configuration.

First, define the two separate nginx clusters by placing the following snippet in a .tf file in a new directory of your choice:

module "nginx_cluster_one" {
  source = "/path/to/nginx-module/"
  kubernetes_host                   = "http://127.0.0.1:57560/" # replace with your minikube dashboard address
  kubernetes_client_certificate     = "/path/to/.minikube/profiles/minikube/client.crt"
  kubernetes_client_key             = "/path/to/.minikube/profiles/minikube/client.key"
  kubernetes_cluster_ca_certificate = "/path/to/.minikube/ca.crt"
  namespace                         = "nginx-cluster-one-ns"
  nginx_version                     = "1.21.1"
  replicas                          = 2
  cpu_limit                         = "250m"
  memory_limit                      = "128Mi"
  service_type                      = "ClusterIP"
}

module "nginx_cluster_two" {
  source = "/path/to/nginx-module/"
  kubernetes_host                   = "http://127.0.0.1:57560/" # replace with your minikube dashboard address
  kubernetes_client_certificate     = "/path/to/.minikube/profiles/minikube/client.crt"
  kubernetes_client_key             = "/path/to/.minikube/profiles/minikube/client.key"
  kubernetes_cluster_ca_certificate = "/path/to/.minikube/ca.crt"
  namespace                         = "nginx-cluster-two-ns"
  nginx_version                     = "1.21.1"
  replicas                          = 3
  cpu_limit                         = "250m"
  memory_limit                      = "128Mi"
  service_type                      = "ClusterIP"
}

This imports the module for both clusters and provides it with cluster-specific parameters for variables like the Kubernetes namespace, nginx version, CPU limit, memory limit, and service type, among other things. Make sure to update the kubernetes_client_certificate, kubernetes_client_key, and kubernetes_cluster_ca_certificate variables with the correct paths from your Minikube config file, which will be in the .minikube folder in the home directory where you installed Minikube.

Once you've updated the variables, run the following Terraform commands one after the other:‍

terraform init
terraform plan 
terraform apply

When you run the terraform apply command, you'll be asked to confirm the actions; type yes. Once you press enter, you'll see an output that resembles the following:

Plan: 6 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.nginx_cluster_one.kubernetes_namespace.nginx: Creating...
module.nginx_cluster_one.kubernetes_namespace.nginx: Creation complete after 0s [id=nginx-cluster-one-ns]
module.nginx_cluster_one.kubernetes_service.nginx: Creating...
module.nginx_cluster_one.kubernetes_deployment.nginx: Creating...
module.nginx_cluster_two.kubernetes_namespace.nginx: Creating...
module.nginx_cluster_two.kubernetes_namespace.nginx: Creation complete after 0s [id=nginx-cluster-two-ns]
module.nginx_cluster_one.kubernetes_service.nginx: Creation complete after 0s [id=nginx-cluster-one-ns/nginx-service]
module.nginx_cluster_two.kubernetes_service.nginx: Creating...
module.nginx_cluster_two.kubernetes_deployment.nginx: Creating...
module.nginx_cluster_two.kubernetes_service.nginx: Creation complete after 0s [id=nginx-cluster-two-ns/nginx-service]
module.nginx_cluster_one.kubernetes_deployment.nginx: Creation complete after 4s [id=nginx-cluster-one-ns/nginx-deployment]
module.nginx_cluster_two.kubernetes_deployment.nginx: Creation complete after 4s [id=nginx-cluster-two-ns/nginx-deployment]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

‍
Once the deployment is complete, use the terraform show command to verify for yourself if the intended infrastructure was deployed. A sample output of the terraform show command can be found in this GitHub Gist.

Alternatively, use the kubectl CLI to get information about both your clusters and the resources deployed in them. Here's how you get all the resources deployed in the namespace nginx-cluster-one-ns:

➜ kubectl get all -n nginx-cluster-one-ns
NAME                                    READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-7d85df7dbc-ff2wz   1/1     Running   0          101m
pod/nginx-deployment-7d85df7dbc-g8xqh   1/1     Running   0          101m

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/nginx-service   ClusterIP   10.109.36.68   <none>        80/TCP    128m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   2/2     2            2           128m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-7d85df7dbc   2         2         2       128m
replicaset.apps/nginx-deployment-d55b49455    0         0         0       116m

‍
If the second nginx cluster deployed successfully, you should get the same for nginx-cluster-two-ns.

This information is also available on the Kubernetes dashboard, as shown in the image below:

Scaling the nginx Deployment

Scaling your nginx deployment is very straightforward in Terraform. To scale up the nginx deployment in this tutorial, simply go to the **.tf **file and make the following changes:
‍

In nginx_cluster_one, change the number of replicas from 2 to 4 and the CPU limit from 250m to 450m.
In nginx_cluster_two, change the number of replicas from 3 to 5 and the CPU limit from 250m to 450m.

These changes allow your nginx deployments to address more requests because of the increased resources. After making these changes, your .tf should look something like the following:

module "nginx_cluster_one" {
  source = "/path/to/nginx-module/"
  kubernetes_host                   = "http://127.0.0.1:57560/" # replace with your minikube dashboard address
  kubernetes_client_certificate     = "/path/to/.minikube/profiles/minikube/client.crt"
  kubernetes_client_key             = "/path/to/.minikube/profiles/minikube/client.key"
  kubernetes_cluster_ca_certificate = "/path/to/.minikube/ca.crt"
  namespace                         = "nginx-cluster-one-ns"
  nginx_version                     = "1.21.1"
  replicas                          = 4
  cpu_limit                         = "450m"
  memory_limit                      = "128Mi"
  service_type                      = "ClusterIP"
}

module "nginx_cluster_two" {
  source = "/path/to/nginx-module/"
  kubernetes_host                   = "http://127.0.0.1:57560/" # replace with your minikube dashboard address
  kubernetes_client_certificate     = "/path/to/.minikube/profiles/minikube/client.crt"
  kubernetes_client_key             = "/path/to/.minikube/profiles/minikube/client.key"
  kubernetes_cluster_ca_certificate = "/path/to/.minikube/ca.crt"
  namespace                         = "nginx-cluster-two-ns"
  nginx_version                     = "1.21.1"
  replicas                          = 5
  cpu_limit                         = "450m"
  memory_limit                      = "128Mi"
  service_type                      = "ClusterIP"
}

‍
Apply the changes using the terraform apply command:

Plan: 0 to add, 2 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.nginx_cluster_two.kubernetes_deployment.nginx: Modifying... [id=nginx-cluster-two-ns/nginx-deployment]
module.nginx_cluster_one.kubernetes_deployment.nginx: Modifying... [id=nginx-cluster-one-ns/nginx-deployment]
module.nginx_cluster_two.kubernetes_deployment.nginx: Still modifying... [id=nginx-cluster-two-ns/nginx-deployment, 10s elapsed]
module.nginx_cluster_one.kubernetes_deployment.nginx: Still modifying... [id=nginx-cluster-one-ns/nginx-deployment, 10s elapsed]
module.nginx_cluster_two.kubernetes_deployment.nginx: Modifications complete after 15s [id=nginx-cluster-two-ns/nginx-deployment]
module.nginx_cluster_one.kubernetes_deployment.nginx: Modifications complete after 15s [id=nginx-cluster-one-ns/nginx-deployment]

Apply complete! Resources: 0 added, 2 changed, 0 destroyed.

After applying the infrastructure changes, run the kubectl get all -n nginx-cluster-one-ns command to confirm that you have four instead of two pods running in your cluster:

➜ kubectl get all -n nginx-cluster-one-ns
NAME                                   READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-d55b49455-2sqwg   1/1     Running   0          33s
pod/nginx-deployment-d55b49455-n87wf   1/1     Running   0          33s
pod/nginx-deployment-d55b49455-ww6l8   1/1     Running   0          25s
pod/nginx-deployment-d55b49455-zlntp   1/1     Running   0          26s

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/nginx-service   ClusterIP   10.109.36.68   <none>        80/TCP    141m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   4/4     4            4           141m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-7d85df7dbc   0         0         0       141m
replicaset.apps/nginx-deployment-d55b49455    4         4         4       128m
‍

Managing Kubernetes Resources with Terraform

You now know how to set up and deploy nginx on two different Kubernetes clusters, and you can start exploring additional Kubernetes resources that help manage your infrastructure.

To better manage multiple deployments of the same resource, use a ConfigMap to inject configuration files and environment variables into your pods that nginx can read during startup. For example, a sample Kubernetes ConfigMap resource definition stores the nginx.conf file that can be loaded when nginx is first deployed:

resource "kubernetes_config_map" "nginx_configmap" {
  metadata {
    name      = "nginx-configmap"
    namespace = var.namespace
  }

  data = {
    "nginx.conf" = <<-EOF
      server {
        listen 80;
        server_name localhost;

        location / {
          root /usr/share/nginx/html;
          index index.html index.htm;
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
          root /usr/share/nginx/html;
        }
      }
    EOF
  }
}

‍
Another important Kubernetes resource is kubernetes_ingress, which allows you to define rules for inbound connections to reach the endpoints defined by the backend. Here's an example from Terraform's official documentation of how to describe this resource in your Terraform project:

resource "kubernetes_ingress" "example_ingress" {
  metadata {
    name = "example-ingress"
  }

  spec {
    backend {
      service_name = "myapp-1"
      service_port = 8080
    }

    rule {
      http {
        path {
          backend {
            service_name = "myapp-1"
            service_port = 8080
          }

          path = "/app1/*"
        }

        path {
          backend {
            service_name = "myapp-2"
            service_port = 8080
          }

          path = "/app2/*"
        }
      }
    }

    tls {
      secret_name = "tls-secret"
    }
  }
}

‍
While these resources are not part of the GitHub repository you cloned at the beginning of the tutorial, you can add these resources to your module as per your requirements.

Conclusion

This article demonstrated some of the useful features of Terraform and how to use it to deploy nginx on Kubernetes using a module built with the official Kubernetes provider from the Terraform registry.

Kubernetes can be difficult to manage, especially when you're bringing changes made in local environments to production. Because it's difficult to mirror actual production setups, teams often face the infamous "works on my machine" dilemma.

mogenius helps tackle this challenge by enabling you to create self-service Kubernetes environments with enhanced visibility. With a unified view of both application and infrastructure components, simplified Kubernetes interaction, and guided troubleshooting, mogenius supports better local testing for developers. It also allows you to integrate seamlessly with CI/CD pipelines, configure external domains, manage SSL, set up load balancing, and handle certificate management. This approach enhances the developer experience, particularly in cloud-agnostic or multicloud environments.

Securing Applications Using Keycloak's Helm Chart

Jan Lepsky — Tue, 10 Dec 2024 15:20:24 +0000

This article was originally posted on the mogenius blog by Rubaiat Hossain.

Keycloak, an open source identity and access management solution, offers single sign-on, user federation, and strong authentication for web applications and services.

Deploying Keycloak in a Kubernetes environment using Helm has several benefits:

Scalability and high availability: As users or requests increase, Kubernetes can automatically scale Keycloak across multiple nodes to handle the load efficiently, ensuring consistent performance.
Simplified deployment and management: Helm handles deployment configurations through a single customizable file, drastically reducing manual configuration mistakes and making it easier to manage upgrades, rollbacks, and scaling operations.
Integration with cloud-native apps: Keycloak on Kubernetes integrates easily with other microservices, providing security and access management across your entire stack.

This tutorial guides you through installing Keycloak on Kubernetes using Helm, configuring it for secure usage, and managing users and realms through Helm.

Prerequisites

Before starting the installation process, make sure you have the following prerequisites installed on your local machine.

Kubernetes cluster: You need a running Kubernetes cluster that supports persistent volumes. You can use a local cluster, like kind or Minikube, or a cloud-based solution, like GKE%20orEKS or EKS. The cluster should expose ports 80 (HTTP) and 443 (HTTPS) for external access. Persistent storage should be configured to retain Keycloak data (e.g., user credentials, sessions) across restarts.
Nginx Ingress: An ingress controller, such as Nginx Ingress, must be installed and configured to route external traffic to Keycloak. You can deploy Nginx Ingress by following its official documentation.
Cert manager: You'll need the Let's Encrypt certificate manager installed and configured to provide secrets for TLS configurations.
kubectl: The kubectl command line interface is required to issue commands to the Kubernetes APIs. You can install Kubectl by following the official installation guide. This tutorial is tested with kubectl version 1.27.
Helm: You'll use Helm to deploy Keycloak and perform management tasks, such as creating realms and managing password policies. You can install Helm by following the official installation guide. This tutorial is tested with Helm version 3.12.1.

Setting Up Keycloak Using Helm

To start, you'll deploy the Keycloak stack to your local Kubernetes cluster using a Helm chart.

Adding the Helm Repository

First, add the Bitnami Helm repository to access the Keycloak Helm chart:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

This step allows you to fetch and install the latest version of Keycloak from the Bitnami repository.

Configuring the Keycloak Values File

To customize your Keycloak deployment, you need to configure the values.yaml file for the Keycloak Helm chart. This file allows you to define environment variables, ingress settings, database configurations, and resource limits for your Keycloak deployment.

Create a values.yaml file and populate it with the following code snippet:

keycloak:
  extraEnvVars:
    - name: KEYCLOAK_LOG_LEVEL
      value: DEBUG
  persistence:
    enabled: true  
    existingClaim: keycloak-pvc

  ingress:
    enabled: true
    hostname: keycloak.example.com
    ingressClassName: nginx
    path: /
    annotations:
      nginx.ingress.kubernetes.io/rewrite-target: /
    tls: false
postgresql:
  enabled: true
  postgresqlUsername: keycloak
  postgresqlPassword: keycloakpassword
  postgresqlDatabase: keycloakdb
service:
  type: NodePort
  nodePorts:
    http: 30080
readinessProbe:
  httpGet:
    path: /realms/master
    port: http
  initialDelaySeconds: 60
  timeoutSeconds: 5
resources:
  requests:
    memory: "512Mi"
    cpu: "500m"
  limits:
    memory: "1Gi"
    cpu: "1"

The above configuration defines the environment variable KEYCLOAK_LOG_LEVEL and sets its value to DEBUG. This enables detailed logs for any issues you may encounter during deployment. You can use environment variables like this to configure Keycloak's runtime behavior and change settings like service ports or database credentials without directly modifying the application code.

It then sets up Ingress settings to expose your Keycloak service to external traffic and defines how ingress will route HTTP(S) requests to your Keycloak instance. The host name parameter specifies the domain name you'll use to access Keycloak. You can use any other domain name here, but make sure to map this URL to the external IP of your Kubernetes cluster in your DNS settings or local /etc/hosts file.

The following part of the code exposes your Keycloak instance to external traffic through the domain keycloak.example.com, with Nginx as the ingress controller managing the traffic:

ingress:
  enabled: true
  hostname: keycloak.example.com
  ingressClassName: nginx
  path: /
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  tls: false

Keycloak stores its user data, realms, and configurations in a PostgreSQL database. It uses the postgresql block in the values.yaml file to set up this database and its credentials. The following part of the configuration enables the default database and sets up its credentials. You can use any other values for the credential:

postgresql:
  enabled: true
  postgresqlUsername: keycloak
  postgresqlPassword: keycloakpassword
  postgresqlDatabase: keycloakdb

During the Helm chart installation, the enabled: true parameter enables this PostgreSQL database. You can set this to false if you have an external database or want to use a different service. In that case, you'll need to provide the connection details.

In the resources section of the code, you set resource requests and limits to control the CPU and memory usage of your Keycloak deployment. This ensures that Keycloak has enough resources to run while preventing it from consuming too much of your cluster's resources:

resources:
  requests:
    memory: "512Mi"
    cpu: "500m"
  limits:
    memory: "1Gi"
    cpu: "1"

When you access Keycloak for the first time, you'll need to log in to the admin console using the administrator account user and its password. You can set the admin password when deploying the Keycloak Helm chart or let Keycloak create an autogenerated password for you.

Use the second option for this tutorial to get a quick start.

Deploying Keycloak Using Helm

With the values.yaml file configured, you can now deploy Keycloak using Helm:

helm install keycloak bitnami/keycloak -f values.yaml

This command installs the Keycloak chart using the values.yaml file described above. It deploys Keycloak on your local Kubernetes cluster and creates all the necessary resources, including services, StatefulSets, and a PostgreSQL database.

To verify that Keycloak is running, check the status of the pods using the following kubectl command:

kubectl get pod

Make sure all Keycloak-related pods are ready (i.e., they have at least one replica) and in a running state.

Accessing the Keycloak Admin Console

Once Keycloak is successfully installed on your cluster, you need to obtain the URL of the deployed Keycloak instance by logging into the admin panel.

You can access Keycloak using the domain name set in the values.yaml file. However, the latest versions of the Keycloak Helm chart implement advanced SSL security checks that prevent users from logging in to the admin panel using HTTP requests to the URL.

You can bypass this by using the IP address of the backend Keycloak server instead. Use the following commands to obtain this IP address:

export HTTP_NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[?(@.name=='http')].nodePort}" services keycloak)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo "http://${NODE_IP}:${HTTP_NODE_PORT}/"

Use the URL provided in the output to access the Keycloak admin panel. The default admin username is user. To retrieve the autogenerated password from the main Keycloak pod, use the command below, which spawns a shell inside the main pod and locates the admin password using grep:

kubectl exec -it keycloak-0 -- printenv | grep KEYCLOAK_ADMIN_PASSWORD

Use these credentials to log in to the master realm, which is the default admin's realm from where you can create additional realms, security policies, users, and more.

Initial Security Measures

When you're logged in to the master realm as the admin user, you can set up some initial security measures from the GUI interface. These settings will apply to all other realms and their identities.

Enable SSL

Enabling SSL encrypts the data transferred between clients, such as browsers and applications, and the Keycloak server. This prevents attackers from intercepting sensitive data like usernames, passwords, and access tokens when users access Keycloak over the public internet.

Keycloak SSL configuration can be enabled by navigating to Configure > Realm settings > General. The Require SSL dropdown shows three options allowing you to configure SSL based on where the requests are coming from.

Set the Require SSL option to All requests to enforce HTTPS for all connections and protect sensitive data. Use External requests if only external IPs need encryption while allowing internal access without HTTPS; avoid using None in production for security.

Configure a Strong Password Policy

A strong password policy safeguards access to your business resources.

The Keycloak GUI makes it very easy to configure password policies. Navigate to Configure > Authentication > Policies > Password policy and click the Add policy dropdown action.

Keycloak password policy allows you to choose different attributes for user passwords, such as the minimum length, number of special characters and digits required, password expiry, and so on.

Enable Account Lockout After Failed Login Attempts

Setting up account lockout after a certain number of failed login attempts helps evade brute-force login attempts from malicious bots and thwarts DDoS attacks on the identity server.

You can enable account lockout in Keycloak by navigating to Configure > Realm settings > Security defenses > Brute force detection and setting the desired mode from the action menu.

Creating and Managing Realms with Helm

In Keycloak, a realm is a way to organize and isolate resources. It's a space where you manage users, roles, policies, and other settings specific to an application. Each realm is entirely isolated, meaning users and clients in one realm cannot access the resources of another realm. So realms allow you to isolate authentication and authorization settings for specific groups of users and applications.

Once in the master realm's admin console, you can create additional realms from the GUI interface.

Creating realms through the GUI, however, can be repetitive and time-consuming, especially in larger deployments.

Helm simplifies this process by allowing you to define realms and their settings in its values.yaml file using the "keycloakConfigCli" utility. This automates the deployment process and makes it reproducible and easy to manage. You can use the values.yaml file to define realms, users, and other Keycloak settings.

To create a new realm called "example-realm", add this configuration to your values.yaml file:

keycloakConfigCli:
  enabled: true
  configuration:
    example-realm.json: |
      {
        "realm": "example-realm",
        "enabled": true,
        "registrationAllowed": true
      }

Apply the changes directly to your Keycloak installation using the following helm command:

helm upgrade --install keycloak bitnami/keycloak -f values.yaml

If you log in to the Keycloak admin console again and navigate to the realm dropdown at the top left, you'll see the newly created example-realm.

User Management Basics

User management is one of Keycloak's core features. You can either create the various users for a realm through the Keycloak GUI interface or use Helm and "keycloakConfigCli" to automatically create users, groups, and roles in your realms during deployment.

By defining user configurations in the values.yaml file, you ensure that Keycloak automatically sets up your users and groups, making the process consistent and repeatable across environments.

For example, if you update the keycloakConfigCli section of your values.yaml with the following code, it will create the "example-realm" alongside a user called "basicuser" and a group named "basicgroup" inside that realm:

keycloakConfigCli:
  enabled: true
  configuration:
    example-realm.json: |
      {
        "realm": "example-realm",
        "enabled": true,
        "registrationAllowed": true,
        "users": [
          {
            "username": "basicuser",
            "email": "basicuser@example.com",
            "enabled": true,
            "firstName": "Basic",
            "lastName": "User",
            "credentials": [
              {
                "type": "password",
                "value": "basicpassword"
              }
            ]
          }
        ],
        "groups": [
          {
            "name": "basicgroup"
          }
        ]
      }

Update the Keycloak installation to see the changes:

helm upgrade --install keycloak bitnami/keycloak -f values.yaml

Creating users with helm provides a streamlined and automated approach to user and group management.

Conclusion

Congrats! You now know how to deploy and manage Keycloak on Kubernetes using Helm. You've learned how to configure Keycloak's values file, handle post-installation security measures, and automate realm and user management using the keycloakConfigCli utility and Helm.

While Keycloak's Helm chart helps you efficiently deploy and manage your identity and access management (IAM) needs, managing Kubernetes applications can still be challenging, especially at scale. mogenius enables you to take care of the entire lifecycle of your Kubernetes applications—from deployment to scaling and maintenance—so you can focus on development rather than infrastructure management.

Whether you use Keycloak or any other large-scale distributed application running on Kubernetes, mogenius abstracts Kubernetes complexity into easy-to-use workspaces, simplifies monitoring and maintenance, and provides seamless CI/CD pipelines to make application deployment a breeze. This reduces the need for DevOps support and increases productivity through developer self-service.

Introduction to Helm for Kubernetes

Jan Lepsky — Tue, 19 Nov 2024 12:32:59 +0000

This article was originally posted on the mogenius blog by Cameron Pavey.

Kubernetes is highly flexible in how it allows you to compose your applications, but this flexibility can mean that building and deploying applications with multiple components can be quite complex. Helm, a self-described “package manager for Kubernetes,” alleviates some of this complexity.‍

In this guide, you’ll learn all you need to know to get started with Helm, including its core components and features and what the Helm workflow looks like. Finally, you’ll see how Helm streamlines Kubernetes DevOps to equip your team for better efficiency.

What Is Helm?

Helm acts as a layer of abstraction on top of your typical Kubernetes workflow. Rather than manually applying manifests when you want to deploy or modify an application, you can use the Helm CLI client to create, find, install, modify, and uninstall charts. In Helm’s terminology, a chart is a collection of files that represents a Kubernetes application (called a release) by describing all the necessary related resources. The application that a chart describes can range from simple single-pod applications to more complex applications with multiple interconnected pods.‍

Charts are declared as files in a directory tree that can either be used as-is or subsequently packaged into versioned .tgz archives, ready for distribution. In a parent directory, a chart consists of several key files and directories:

Chart.yaml: A YAML file containing information about the chart.
values.yaml: The default configuration for the chart.
charts/: A directory containing any charts upon which this chart depends.
crds/: A directory containing any custom resource definitions.
templates/: A directory of templates that will be combined with values to generate valid Kubernetes manifests.

Noteworthy Helm Features

Helm has several features that help you manage your Kubernetes applications more efficiently.‍

Templates and Value Substitution

Often, you will need to create dynamic templates that can be configured depending on how the application will be used. Helm handles this by allowing you to define values in your values.yaml file, which can then be used in your templates through a special templating syntax.‍

For instance, consider a values.yaml file like this:

color: red

You could use this value in a template like so:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-configmap
data: 
  color: {{ .Values.color }}‍

Using templated values this way means you can control this value without directly modifying the manifest template.

Release Management and Versioning

As a package manager, one of Helm’s responsibilities is the release and version management of packages. While you will typically use the latest version when installing a chart, Helm package archives are versioned, so it is possible to specify a particular version of a package, much like other package managers you may be familiar with.‍

This means that if you maintain Helm charts, you can make incremental updates and publish new versioned archives as needed, while the old ones can remain available to have historical records or for legacy support.

Dependency Management

One of Helm’s benefits is its ability to help you install complex applications without manually setting everything up.‍

Complex applications often require a number of dependencies. If you were to set up an application from scratch, you would need to declare and manage the dependencies yourself.‍

Helm, however, has built-in dependency management: a chart declares its dependencies, and these will be set up and configured when the release is created.‍

Rollback Capabilities

If something goes wrong during a release, Helm makes it easy to revert to your previous working configuration through rollbacks.

Each time you install, upgrade, or roll back a release, your release’s revision number is incremented by 1. You can use this revision number to revert to a previous version of your release like so:

# helm rollback [RELEASE] [REVISION]
helm rollback my-release 1‍

To determine which revision to roll back to, you can view the revision history of a given release:‍

$ helm history my-release  
REVISION        UPDATED                         STATUS          CHART                   APP VERSION     DESCRIPTION  
1               Tue Sep 10 09:26:54 2024        deployed        wordpress-23.1.12       6.6.1           Install complete

How Does Helm Work?

Helm offers many options, and trying to understand them all at once can be overwhelming. However, you don’t need that to get started. You only have to know how Helm’s core features can be used together to form a workflow:‍

‍Chart creation and packaging: If you’re packaging your own application, the first step is to create the necessary files. This can be done via helm create NAME,which will generate some boilerplate files for you to modify. Once your application is configured, you can create a versioned archive if you want to distribute it.
‍Finding existing charts: Rather than packaging your own applications, you can also use Helm to find and download existing charts of other applications. You can use these charts to install and configure an application and its dependencies.
‍Template rendering process: As part of the installation process, Helm combines your chart, templates, and values to render your templates as Kubernetes manifests, ready to be applied to your cluster.
‍Interaction with the Kubernetes API: With the rendered manifests on hand, Helm interacts with the Kubernetes API server to apply the manifests and create or update any resources necessary for your release.‍

Getting Started with Helm

There are several ways to install Helm, including binary release, install scripts, and package managers. Refer to the Helm documentation to select the best method for your needs.‍

Once you have Helm installed and a Kubernetes cluster to use it with, you can try out some common workflows.‍

Basic Helm Commands

To use Helm, you’ll issue commands via the CLI client. Helm has quite a few commands, but the following are essential ones you should be aware of:‍

# Install a chart
helm install <name> <chart>

# Install a chart while setting values on the command line (can specify multiple or separate values with commas)
helm install <name> <chart> --set key1=val1,key2=val2
# Run a test installation to validate chart
helm install <name> <chart> --dry-run --debug
# Upgrade a release
helm upgrade <release> <chart>
# Upgrade a release, or install it if a corresponding release does not already exist
helm upgrade <release> <chart> --install
# Add a repository from the internet:
helm repo add <repo-name> <url>
# Update information of available charts locally from chart repositories
helm repo update
# Download a chart as a versioned archive
helm pull <chart>‍

The official documentation has a handy cheat sheet with more common commands and their purposes.

Creating Custom Charts

If you want to use Helm for packaging and distributing your own applications, you will need to create a custom chart. To do this, run helm create my-chart, which creates a new directory containing generated boilerplate files. The following are key files to be mindful of here:‍

Chart.yaml: This file contains metadata about your chart.
templates/: This directory contains manifest templates for each resource your chart will create. This is where you can define any Kubernetes resources you want your custom chart to include.
values.yaml: This file contains the configuration values that many of the default templates will use. Anything that should be configurable in your chart should be defined here.‍

Once you’ve modified your chart to suit your needs, you can install it directly from your my-chart/ directory by running helm install my-chart.It will run through the workflow above to render your templates before applying them to the Kubernetes cluster.‍

Finding and Using Existing Charts

You can use Helm to search for existing charts, either by searching the artifact hub or a repo.‍

For instance, if you wanted to install nginx you could search for it on the artifact hub using helm search hub nginx or by accessing the hub via the web. Viewing the entry on the web shows that you can install the chart with its default settings by running this command:‍

helm install my-nginx oci://registry-1.docker.io/bitnamicharts/nginx‍

If you want to view or modify the chart before installing it, use the helm pull command like so:‍

helm pull oci://registry-1.docker.io/bitnamicharts/nginx‍

This command downloads a versioned chart archive that you can view and modify as needed. You can also view the values supported by the chart to reveal any options you can modify and configure before installing:‍

helm show values oci://registry-1.docker.io/bitnamicharts/nginx‍

Specify any values you would like to change by defining them in a YAML file like my-values.yaml and pass it as an option when you install the chart, like so:‍

helm install my-nginx oci://registry-1.docker.io/bitnamicharts/nginx -f my-values.yaml‍

Deploying a Sample Application

Let’s now see how all of the above comes together when you install, upgrade, and roll back a simple application. A good example application to use for understanding Helm’s workflows is nginx; it needs no configuration, and it’s easy to tell whether it is working.‍

As mentioned, you can install the nginx chart directly from the artifact hub linked above, but adding the source repository and installing charts from there is closer to how you are likely to use Helm for internal application development. It is good to be familiar with both approaches, but the repository approach will be used here.‍

To add the Bitnami nginx repo, run this command:‍

helm repo add bitnami https://charts.bitnami.com/bitnami‍

You can search your repos for nginx charts like so:‍

$ helm search repo nginx
NAME                                    CHART VERSION   APP VERSION     DESCRIPTION  
bitnami/nginx                           18.1.11         1.27.1          NGINX Open Source is a web server that can be a...  
bitnami/nginx-ingress-controller        11.4.1          1.11.2          NGINX Ingress Controller is an Ingress controll...  
bitnami/nginx-intel                     2.1.15          0.4.9           DEPRECATED NGINX Open Source for Intel is a lig...‍

To install a basic nginx release, you can use one of the charts listed by your search:‍

helm install my-nginx bitnami/nginx --version 18.1.11 --namespace my-nginx --create-namespace‍

This will install version 18.1.11 of the chart, which corresponds to version 1.27.1 of nginx. The --namespace my-nginx specifies the Kubernetes namespace into which to install the chart, and the --create-namespace option creates the namespace if it does not already exist. You can omit these options if you prefer, but it's good to know them since they give you more control over how your applications are installed.‍

Once this command is done running, you can confirm whether the release has been successfully installed via kubectl:‍

$ kubectl get all -n my-nginx
NAME                            READY   STATUS    RESTARTS   AGE  
pod/my-nginx-546f4bccc7-pvmt2   1/1    Running   0          11m
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                   AGE  
service/kubernetes   ClusterIP      10.152.183.1    <none>       443/TCP                    2d11h  
service/my-nginx    LoadBalancer   10.152.183.65   <pending>   80:32399/TCP,443:31390/TCP   13m
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE  
deployment.apps/my-nginx   1/1    1            1           13m
NAME                                DESIRED   CURRENT   READY   AGE  
replicaset.apps/my-nginx-546f4bccc7   1         1       1       13m‍

Several resources have been created to support the release, even for this basic application. This is one of Helm’s key advantages — it manages all the resources and dependencies for you.‍

Suppose you want to change some of the values in the nginx chart. You can view a list of all the values available by running:

helm show values bitnami/nginx‍

There are many values you can configure, and one of them is replicaCount, which corresponds to how many pods will be running for this release. You can modify your existing release to set new values. For good measure, you can also use a different version of the chart (in this case, an earlier version if you were already on the latest):

helm upgrade my-nginx bitnami/nginx --version 18.1.0 --set replicaCount=2‍

Once this command is done, you can view the impact with kubectl again:‍

$ kubectl get all -n my-nginx
NAME                            READY   STATUS    RESTARTS   AGE  
pod/my-nginx-7f9647978d-5j8rn   1/1    Running   0          26s  
pod/my-nginx-7f9647978d-8hdqd   1/1    Running   0          15s
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                   AGE  
service/kubernetes   ClusterIP      10.152.183.1    <none>       443/TCP                    2d11h  
service/my-nginx    LoadBalancer   10.152.183.65   <pending>   80:32399/TCP,443:31390/TCP   18m
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE  
deployment.apps/my-nginx   2/2    2            2           18m
NAME                                DESIRED   CURRENT   READY   AGE  
replicaset.apps/my-nginx-546f4bccc7   0         0       0       18m  
replicaset.apps/my-nginx-7f9647978d   2         2       2       18m‍

Similarly, you can view the release history in Helm, which lists each revision your release has gone through:‍

$ helm history my-nginx
REVISION        UPDATED                         STATUS          CHART           APP VERSION     DESCRIPTION  
1               Tue Sep 10 22:10:05 2024        superseded      nginx-18.1.11   1.27.1          Install complete  
2               Tue Sep 10 22:11:02 2024        deployed        nginx-18.1.0    1.27.0          Upgrade complete‍

If you want to undo some changes you’ve made to a release, it’s as simple as running the following:‍

helm rollback my-nginx 1‍

This command reverts the release to the specified revision. You can confirm the revert through kubectl and helm history:‍

$ kubectl get all -n my-nginx
NAME                            READY   STATUS    RESTARTS   AGE  
pod/my-nginx-546f4bccc7-5bm62   1/1    Running   0          56s
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                   AGE  
service/kubernetes   ClusterIP      10.152.183.1    <none>       443/TCP                    2d11h  
service/my-nginx    LoadBalancer   10.152.183.65   <pending>   80:32399/TCP,443:31390/TCP   23m
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE  
deployment.apps/my-nginx   1/1    1            1           23m
NAME                                DESIRED   CURRENT   READY   AGE  
replicaset.apps/my-nginx-546f4bccc7   1         1       1       23m  
replicaset.apps/my-nginx-7f9647978d   0         0       0       22m‍

How Helm Streamlines Kubernetes DevOps

As you’ve seen, Helm offers powerful capabilities over your standard Kubernetes toolkit, making deploying and managing Kubernetes applications simpler.

When used effectively, Helm can streamline both the development and operation of your applications. Helm offers the following benefits for developers of Kubernetes applications:‍

Packaging: You can package an entire Kubernetes application into a single unit rather than a loose collection of manifests.
Version management: Helm provides easy, built-in version management functionality for your charts.
Templates: The combination of templates and values gives you an effective way to customize your application for deployment in different environments.
Dependencies: Helm streamlines the complexity of managing dependencies between different components of your application.
Reuse: You can create reusable charts for common application components and patterns you want to share between multiple applications.‍

Once applications have been developed and are ready to be deployed and operated, Helm also offers advantages for operations:‍

Consistency: Helm gives you a consistent, standard way to manage the deployment of your applications across many environments. It also provides a consistent mechanism for applying things like security best practices, environment configuration, and settings across deployments.
Upgrades and rollbacks: Helm’s upgrade and rollback functionality makes it easy and low-risk to upgrade applications, change values, and revert changes if something goes wrong. This reduces the effort and complexity involved in modifying running applications.
Management: Helm’s dependency management means applications are essentially plug-and-play, without needing to ensure all your dependencies are configured and ready to go in advance. This greatly simplifies the process of deploying new application instances.‍

Wrapping Up

Helm, a powerful package manager for Kubernetes applications, provides new functionality for managing larger-scale application deployments on top of the existing Kubernetes API server and kubectl functionality. Features like templates with value substitution, seamless upgrades and rollbacks, fully managed dependencies, and versioned package archives fill the gaps needed to take Kubernetes from a container orchestrator to an out-of-the-box application platform.‍

Even though tools like Kubernetes and Helm provide powerful capabilities for scalable cloud-native operations, complexity can be a burden for development teams. mogenius, as a self-service solution, bridges the gap and allows developers to safely deploy and manage applications without deep Kubernetes expertise. The internal developer platform comes with a built-in Helm manager that enables teams to manage Helm repositories, charts, and releases on clusters with a comprehensive interface. You can even sign up for free and deploy your first Helm chart effortlessly.

How Kubernetes YAML manifests are dragging down developer productivity

Jan Lepsky — Tue, 01 Oct 2024 08:40:48 +0000

In the previous blog post, we explored five best practices for writing Kubernetes YAML manifests. While YAML’s human-readable format and flexibility make it a popular choice for writing configuration files in the Kubernetes ecosystem, it presents significant challenges when used as an interface for developers and DevOps professionals.

In this blog post, we’ll first discuss some common issues associated with managing Kubernetes YAML manifests and their impact on developer productivity. Next, we’ll explore how Kubernetes self-service platforms help solve these challenges and why you should adopt them to streamline your workflow.

Let’s get started!

Challenges in managing Kubernetes YAML manifests

Below are some of the key challenges developers face when dealing with Kubernetes YAML manifests:

Complexity and sensitivity of YAML syntax
YAML relies heavily on indentation to define the structure and hierarchy of data. A minor mistake, such as mixing spaces and tabs or misplacing an indent, can lead to parsing errors. This sensitivity to whitespace can be a significant source of frustration, as even a small error can cause a deployment to fail or behave unexpectedly.

Moreover, YAML's syntax includes various features like anchors, aliases, and complex data types, which can add to the complexity. You need to be familiar with these features to use YAML effectively, which can increase the cognitive load and learning curve.

‍
Version inconsistencies
YAML has undergone several revisions, each introducing new features and subtle changes in behavior. This means that YAML documents can be parsed differently depending on the version being used. You must be aware of the specific YAML version you are working with to avoid compatibility issues, which adds another layer of complexity to managing YAML manifests. This requirement can be particularly challenging in environments where multiple tools or libraries with different YAML version dependencies are used.

Here are just a few examples of how version inconsistencies can impact YAML management:

1. Feature Deprecation: Older versions of YAML might support features that have been deprecated or changed in newer versions. For instance, if a YAML file uses an older syntax that has been replaced or removed in a newer version, this can lead to parsing errors or unexpected behavior. For example, YAML 1.1 has certain features like implicit typing that were revised in YAML 1.2.
2. Syntax Variations: Different YAML versions can have subtle differences in syntax rules. For example, YAML 1.1 allowed for more flexibility in how complex data structures were represented compared to YAML 1.2. A YAML file that works fine in one version might cause errors or fail to parse correctly in another if it relies on deprecated or changed syntax.
3. Library and Tool Compatibility: Various tools and libraries that handle YAML might be built to work with specific versions. If your project uses tools or libraries that expect a particular YAML version, you could encounter compatibility issues when integrating different tools or libraries. For example, if a CI/CD tool is designed to parse YAML 1.2 but your manifests are written using YAML 1.1 features, you might face integration issues.
4. Behavioral Changes: Subtle behavioral changes between YAML versions can lead to inconsistencies. For instance, the way certain data types or structures are interpreted might differ. This could lead to unexpected results if your YAML documents use features that have changed behavior between versions.

Cross-environmental management
Applications typically need to be deployed across multiple environments, such as development, testing, staging, and production. Each environment may require specific configuration settings, such as different resource limits or database connections. Managing these environment-specific configurations within YAML manifests can quickly become a maintenance nightmare. Developers often resort to duplicating and modifying manifests for each environment, leading to inconsistencies and configuration drift.

How Kubernetes YAML challenges affect developer productivity

The challenges associated with managing Kubernetes YAML manifests have a direct and profound impact on developer productivity. These issues consume valuable time and resources, hindering developers' ability to focus on building features that deliver value to users and the business. Here are some specific ways these challenges affect productivity:

Increased debugging time
The complexity and sensitivity of YAML syntax often result in errors that are difficult to diagnose and resolve. One common issue is indentation errors caused by using tabs instead of spaces. YAML is highly sensitive to indentation and expects consistent use of spaces for defining structure. Here’s an example of how this can lead to issues:

YAML Example with Tabs (Invalid)

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  labels:
    app: example
  spec:
    containers:
      - name: example-container
        image: nginx
        ports:
          - containerPort: 80

(Note: The above example uses tabs for indentation, but it should use spaces.)

YAML Example with Spaces (Valid)

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  labels:
    app: example
  spec:
    containers:
      - name: example-container
        image: nginx
        ports:
          - containerPort: 80

In the invalid YAML example, tabs are used for indentation instead of spaces. This error can be difficult to detect because YAML parsers often don’t provide clear error messages for this kind of issue. As a result, a deployment might fail or behave unexpectedly due to incorrect indentation, leading developers to spend considerable time debugging the issue.

Cognitive load and burnout
The need to master YAML's intricate features adds to the cognitive load on developers. This learning curve can be particularly steep for those new to YAML, leading to frustration and burnout. The constant attention required to manage these complexities detracts from creative problem-solving and innovation.

Inefficient collaboration and missed deadlines
The challenges of managing YAML manifests can impede collaboration between development and DevOps teams. The time and resources spent resolving YAML-related issues slow down the overall development process, leading to potential bottlenecks. This inefficiency can further impact project timelines, resulting in missed deadlines and a slower pace of product evolution. The reduced focus on feature development and innovation ultimately affects the competitiveness of the product in the market and hinders a team's ability to meet user demands and adapt to market changes.

Solving Kubernetes YAML challenges with Kubernetes self-service platforms

As discussed, Kubernetes YAML challenges can have a significant impact on developer productivity. Therefore, it's essential to explore solutions that can help overcome these obstacles. Kubernetes self-service platforms like mogenius offer a powerful means to streamline workflows and reduce the complexities associated with YAML management. Here’s how these platforms can make a difference:

Simplified configuration management
Self-service platforms provide user-friendly interfaces that abstract away the intricacies of YAML syntax, allowing developers to manage configurations without getting bogged down by syntax errors. This simplification reduces the time spent on debugging and allows developers to focus on core development tasks.

Automated version control
With built-in version control mechanisms, self-service platforms ensure that YAML manifests remain consistent across different environments. This automation mitigates the risk of version inconsistencies and configuration drift, leading to more reliable deployments and smoother operations.

Enhanced collaboration and communication
These platforms facilitate seamless collaboration between development and DevOps teams by providing centralized tools and repositories. By reducing the need for constant back-and-forth communication, teams can resolve issues more efficiently and maintain a steady development pace.

Environment-specific customization
Kubernetes self-service platforms enable developers to easily manage environment-specific configurations through templating and parameterization. This customization ensures that each environment's unique requirements are met without the need for redundant manual edits, enhancing consistency and reducing errors.

Reduced cognitive load
By abstracting complex features and automating routine tasks, these platforms significantly reduce the cognitive load on developers. With fewer distractions and technical hurdles, developers can dedicate more energy to innovation and delivering value to users.

Wrapping It Up

In this blog post, we first discussed some key challenges associated with managing Kubernetes YAML manifests. We then explored how these challenges significantly affect developer experience and decrease the productivity of teams. Finally, we examined how Kubernetes self-service platforms help address these challenges.

The message is clear: if you value efficient operations and want to make the most of your time, adopting self-service platforms is the best approach.

Best Practices for Writing Kubernetes YAML Manifests

Jan Lepsky — Mon, 09 Sep 2024 15:26:18 +0000

Kubernetes objects are deployed to Kubernetes clusters using configuration files written in YAML, often referred to as YAML manifests. You specify the "desired state" of objects in a manifest file and send the file to the Kubernetes API server. Kubernetes then automatically configures and manages the application based on your specifications. In this blog post, we'll outline five best practices you should remember while writing Kubernetes YAML manifests. Let's get started!

1) Use the latest stable API version

Kubernetes API versions typically go through three stages:‍

1. Alpha: The version names contain "alpha" (e.g., v1alpha1). These are experimental features that may be unstable and are disabled by default. Alpha APIs can change without notice and are not recommended for production use.‍
2. Beta: The version names contain "beta" (e.g., v2beta3). These are well-tested features, but are disabled by default. Beta features are considered safe to enable but are not recommended for production use as they may still undergo breaking changes.
3. Stable: The version names are simply "vX" where X is an integer (e.g., v1). These are production-ready features that are fully supported, enabled by default, and maintain backwards compatibility.

To leverage these versioned APIs, every Kubernetes object manifest must specify the API version in a field named apiVersion. This field tells Kubernetes which version of the API to use when processing the manifest.

The apiVersion field typically consists of two parts: the API group (such as apps or batch) and the actual version (such as v1). Note that for core Kubernetes objects, only the version is specified.

Here's an example manifest for a Deployment object:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.27

‍
In this example, apiVersion: apps/v1 indicates that this Deployment object uses the v1 version of the apps API group, which is the latest stable version for Deployments. When creating Kubernetes object manifests, you must always use the latest stable API version available for each object type.

Here’s why:

1. Reliability: Stable APIs are less likely to introduce breaking changes, ensuring that your applications remain functional over time.
2. Support: Stable versions receive regular updates and support from the Kubernetes community, making it easier to find help and resources.
3. Future-proofing: By adopting stable APIs, you position your applications to benefit from ongoing enhancements and avoid the risks associated with deprecated or unstable versions.

To find out the latest stable API version for an object on your current Kubernetes cluster, you can use the kubectl api-resources command. This command queries the Kubernetes API server you're connected to and lists all available resources and their API versions supported by that specific cluster.

2) Use labels that identify semantic attributes of your application

In Kubernetes, labels are arbitrary key/value pairs that you can attach to objects. They provide a flexible way to organize and categorize objects and manage them efficiently. Labels are particularly useful when combined with label selectors, which allow you to filter and operate on specific sets of objects.

Consider this example Deployment manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:v1

‍
While these labels are valid, they don't fully capture the semantic attributes of the application. In other words, these labels don't convey meaningful characteristics about the application. A better way to label the application would be to add more descriptive labels.

The Kubernetes official documentation recommends a set of common labels that you can apply to your object manifest. These labels, which begin with the prefix app.kubernetes.io/ followed by a separator (/), provide a standardized way to describe your application's components and improve interoperability with various Kubernetes tools and systems.

Here's what an improved version of the aforementioned Deployment manifest could look like, incorporating these recommended labels:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app.kubernetes.io/name: myapp
    app.kubernetes.io/version: "1.0.0"
    app.kubernetes.io/component: frontend
    app.kubernetes.io/part-of: web-application
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp
        template:
    metadata:
      labels:
        app.kubernetes.io/name: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:v1.0

‍
These semantic labels provide much more context about the application. They describe not just what the application is, but also its version and its role in the larger system.

The benefits of this semantic labelling approach include:

1. Improved organization: Resources are grouped in a more meaningful way.
2. Enhanced querying: You can easily find all frontend components or all resources related to a specific application.
3. Clearer communication: Team members can quickly understand the purpose and context of each resource.
4. Interoperability: The use of standardized app.kubernetes.io/ labels enables different Kubernetes tools to work together seamlessly, recognizing and utilizing the same information across various platforms and systems.
‍
By using semantic labels consistently across your Kubernetes objects, you create a more self-documenting system that is easier to understand and manage.‍

3) Put object descriptions in annotations for better introspection

In Kubernetes, annotations are key-value pairs that allow you to attach non-identifying metadata to objects. Typical examples of annotations include build information, release IDs, Git branch names, PR numbers, image hashes, registry information, or team contact details.

Annotations provide a way to examine and understand the objects in the cluster more deeply. This is what the phrase "better introspection" means in the context of Kubernetes annotations.

Here's an example of a Pod with three annotations for commit, author, and branch:

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  annotations:
    git.commit: "7a8b9c0d1e2f3g4h5i6j7k8l9m0n1o2p"
    git.author: "Sarah Chen <sarah.chen@example.com>"
    git.branch: "feature/custom-nginx-config"
spec:
  containers:
  - name: nginx
    image: nginx:1.27

In this example, annotations provide valuable context about the specific version of the NGINX configuration being deployed, which can be extremely useful for debugging, auditing, and managing your Kubernetes Deployments. While these annotations offer useful information for human readers, their power extends far beyond simple documentation.

In fact, annotations are primarily used to provide additional context or configuration information that can be utilized by external tools, automation systems, or client libraries interacting with the Kubernetes API.

For example:

A CI/CD tool might use annotations to store information about the build process.
A monitoring tool might use annotations to specify how to scrape metrics or which alerts to associate with a particular resource.
A custom deployment tool might use annotations to store information about rollout strategies or canary deployments. ‍ Therefore, always ensure to include descriptive and relevant annotations in your Kubernetes object definitions to enhance manageability and observability.‍

Key Differences in Allowed Characters

Labels: Keys are more restricted; both the prefix and the name must conform to DNS subdomain rules and specific character restrictions. Values must be relatively short (63 characters or less) and are limited to certain characters.
Annotations: Keys follow the same rules as labels, but values have no strict size limits and can contain any UTF-8 characters, allowing for much more flexibility in what they can store.

4) Don’t hardcode secret data

Secret data consists of sensitive information that should be protected from unauthorized access. This includes passwords, API keys, tokens, and other confidential data. A common mistake in Kubernetes deployments is hardcoding this sensitive information directly into manifest files.

To illustrate this issue, let's examine the following Deployment manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql-container
        image: mysql:8.0
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "my-secret-password"

In this manifest, we have hardcoded the value for the root password directly in the environment variable. This exposes the password in plain text, posing significant security risks.

A more secure approach is to store the root password in a Kubernetes Secret object and then reference it in the Deployment. Here's how you can create a Secret:

apiVersion: v1
kind: Secret
metadata:
  name: mysql-root-pass
type: Opaque
stringData:
  password: mysql_secret_password

‍
Once you've created the Secret, you can reference it in your Deployment manifest by mounting it as an environment variable:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql-container
        image: mysql:8.0
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-root-pass
              key: password

Using Secrets in this manner offers several key benefits:

1. Separation of concerns: Keeping sensitive data separate from application code adheres to best practices in configuration management.
2. Easier management: Secrets can be updated independently of the Deployment, allowing for easier rotation of credentials.
3. Environment-specific configuration: Different Secret objects can be created for various environments, facilitating consistent application behavior across different setups.
‍

5) Combine related Kubernetes objects into one file

When working with Kubernetes, you often need to create multiple related objects to deploy an application. While it's possible to define each object in a separate file, the following are some scenarios where combining related Kubernetes objects into a single file can be beneficial.

For small to medium-sized applications where all components are tightly coupled.
When you want to deploy an entire application stack with a single command.
For objects that are always deployed together and have a strong relationship.

Let's explore this concept with an example. Imagine you want to deploy a backend API server for a simple web application. Traditionally, you might have separate YAML files for this component: backend-deployment.yaml and backend-service.yaml.

backend-deployment.yaml

apiVersion: v1
kind: Pod
metadata:
 name: nginx-web-server
 labels:
   app.kubernetes.io/name: nginx-web-server
spec:
 containers:
   - name: nginx-web-server
     image: nginx:1.27
     ports:
       - containerPort: 80
         name: http-web-svc

‍
backend-service.yaml

apiVersion: v1
kind: Service
metadata:
 name: nginx-web-server-service
spec:
 selector:
   app.kubernetes.io/name: nginx-web-server
 ports:
   - name: http
     protocol: TCP
     port: 80
     targetPort: http-web-svc

‍
This approach results in two separate files that you need to manage and deploy individually.

Instead, you could combine the two objects into a single backend.yaml file. This file would contain the Deployment and the Service for your backend, separated by --- delimiters, like this:

backend.yaml
apiVersion: v1
kind: Pod
metadata:
 name: nginx-web-server
 labels:
   app.kubernetes.io/name: nginx-web-server
spec:
 containers:
   - name: nginx-web-server
     image: nginx:1.27
     ports:
       - containerPort: 80
         name: http-web-svc
---
apiVersion: v1
kind: Service
metadata:
 name: nginx-web-server-service
spec:
 selector:
   app.kubernetes.io/name: nginx-web-server
 ports:
   - name: http
     protocol: TCP
     port: 80
     targetPort: http-web-svc

‍
This combined approach offers several benefits:

1. Simplified deployment: You can deploy all objects with a single kubectl apply -f backend.yaml command.
2. Improved version control: Changes to related objects are tracked together in your version control system.
3. Better readability: It's easier to understand the full application structure when all components are in one file.
4. Easier troubleshooting: When all related objects are in one file, it's simpler to identify and fix issues.

However, it's important to note that this approach works best for smaller applications or tightly coupled components. For larger, more complex applications, you might still prefer to keep objects in separate files for better modularity and easier management of individual components.
‍

Conclusion

Adhering to best practices for writing Kubernetes YAML manifests is essential, particularly when collaborating within a team or managing complex applications. By consistently implementing these practices, you not only streamline your workflow but also enhance the clarity, security, and organization of your deployments. You contribute to a more efficient and collaborative development environment, which leads to improved productivity and a faster software development lifecycle.

Additionally, utilizing tools such as the YAML extension for Visual Studio Code can further improve your efficiency when working with YAML manifests. This extension provides features like auto-completion, error highlighting, and snippets, which can help you avoid syntax errors and speed up the file creation process, making it an invaluable tool for any Kubernetes user.

This article was first published on the mogenius blog. Check it out and discover a Kubernetes platform made for outstanding Developer Experience.

Monitoring Service Health in Kubernetes: A Simplified Approach

Jan Lepsky — Tue, 26 Mar 2024 14:42:28 +0000

Determining the health of a service is a crucial aspect of maintaining reliability and performance in modern software development, especially when working with container orchestration tools like Kubernetes.

The health of a service is a multifaceted concept that hinges on its ability to meet availability, performance, and correctness standards throughout its lifecycle. This is not a trivial task, given the complex interplay of factors that influence a service's health, from the build pipeline and deployment processes to the ongoing monitoring and management of the service on Kubernetes.

Let’s have a look at some critical areas.

Understanding Service Health: A Comprehensive Overview

Effective service status monitoring is crucial for real-time health insights across build, deployment, and operations, enabling developers to promptly identify and address failures.

1. Continuous integration and building: The Foundation of Service Health

The journey toward a healthy service begins with a robust CI pipeline. Key health indicators here include the speed and success of builds and tests, crucial for minimizing "Time to Feedback" for developers. Real-time feedback mechanisms, such as CI/CD dashboards and direct integration of checks into the pipeline (e.g., code coverage, linting), are essential for early detection and resolution of potential issues.

2. Deployment: Ready for Traffic

Once the build is finished it is key to observe the success of the deployment. Multiple building blocks of Kubernetes can cause failures, e.g. deployment configurations, resources, pod scheduling, etc. Furthermore, deployment practices must ensure that a service is fully prepared to handle traffic before it is exposed to users. Kubernetes Readiness Probes are critical in this phase, verifying that a service is ready to perform. Furthermore, the implementation of automatic rollback mechanisms based on metrics like error rates and latency ensures that any deployment that could degrade service health is promptly reverted.

To keep your service healthy, rigorously monitor CI/CD progress, deploy readiness checks, and proactively manage performance through real-time dashboards for uninterrupted service excellence. (source: grafana-dashboards)

3. Ongoing Service Monitoring: Keeping the Pulse

Once deployed, continuous monitoring of a service's health is crucial. There’s narrow features in Kubernetes like Liveness Probes that help determine the need for restarting instances, or broad solutions like Prometheus and Grafana that offer detailed monitoring and visualization of service health. Centralized log aggregation and distributed tracing tools enable deep analysis and troubleshooting.

Challenges of bringing service status monitoring to life

Each of the presented steps offers valuable tactics for achieving continuous service health, better visibility during the development process, and faster recovery in case of an incident. However, deploying them to a team and making use of the tools in the software development lifecycle is a different story. In Kubernetes environments, a failing pipeline or crashing pods can have multiple causes. Multiple tools are involved in the process and on Kubernetes-level there’s several workloads that can affect service health.

The challenge lies in the aggregation of service status data in order to deliver an actionable source of truth. With self-service in mind, this should be designed for software developers to enable them to independently check the status of their service and investigate each step in the pipeline. In an aggregated view, delivering near real-time data is key, in order to display what is happening at any given moment what is happening with build, deployment, pod, or health check. Additionally, a certain level of abstraction is required to allow fast interpretation of service status as well as reducing onboarding efforts of developers.

The goal: A single pane of glass

The recently emerging trend of Internal Developer Platforms (IDP) offers a solution to introducing self-service capabilities in development teams. Especially for Kubernetes environments, mogenius offers a Kubernetes Operations Platform that embodies the principles of platform engineering by offering a self-service solution dedicated to simplifying cloud-native development. This platform enables software developers to efficiently deploy and manage applications in the cloud, focusing on innovation rather than infrastructure management.

Within its suite of features, mogenius includes actionable service status monitoring, providing users with a transparent view of their service's health across the build, deployment, and operational phases. The service status delivers real-time data in an aggregated way, allowing developers to instantly identify the cause of failures independently.

Each component of the service indicates success or failure with associated logs to individually investigate the pipeline step.

Kubernetes Health Checks and Beyond

While competing solutions require specific Kubernetes onboarding and offer limited capabilities in adjusting and configuring resources, mogenius’s service status system allows for quick identification and resolution of issues, enhancing the developer experience by reducing the time spent on diagnosing problems by offering detailed logs and metrics at each step of the service lifecycle.

The recent build and deployment were successful and three pods are in running state.

Two of three pods are in an error state which can be investigated in the pod logs.

The last deployment failed, resulting in all three pods being unable to reach a running state.

mogenius goes further by introducing a fourth level of checks: Kubernetes Health Checks, enabling a comprehensive health overview up to the application layer, including scenarios where containers are running, but the application is unreachable, e.g. due to external dependencies like a database failure. With pre-configured health checks toggleable by users, mogenius leverages Kubernetes' Startup Probes, Liveness Probes, and Readiness Probes in a user-friendly manner, adhering to best practices while simplifying the user experience.

mogenius’ Advanced Kubernetes Health Monitoring

mogenius’s approach not only reduces the complexity involved in ensuring service health but also aligns with the shift-left ideology, empowering developers to handle more tasks earlier in the software development lifecycle. mogenius's new service status system exemplifies how technology can be leveraged to enhance visibility, reduce error diagnosis time, and improve overall service reliability and performance in a Kubernetes environment.

Consider

The health of a service is a critical aspect of software development, impacting not just the reliability and performance of the applications but also the efficiency of the development process itself. Navigating the complexities of maintaining service health in Kubernetes environments can be challenging. However, with the advent of internal developer platforms like mogenius, the landscape is changing. mogenius offers a compelling solution that simplifies cloud-native development, empowering developers and DevOps professionals to maintain the health of their services more effectively, allowing them to focus on what they do best: building great software.