Getting Started With Kustomize For Kubernetes

When you’re trying to figure out a folder hierarchy for your apps or how you’re going to pass in variables, parameters, and secrets, it can get tough. Especially if you have multiple applications and environments that you need to account for and manage throughout lifecycles.

With Kustomize, things get a little bit easier to manage.

In this blog post, you’ll learn about what Kustomize is, why you’d want to use it, and how to get started with it today.

What Is Kustomize

Every application that you have will have to run in multiple environments. Typically, you’ll see something like Development, Staging (or UAT), and Production. However, there can be plenty of other environments, like QA. If you have a multiple Kubernetes Manifests, that means you’ll have to create each Manifest for each environment. For example, If you have a Deployment.yaml and a Service.yaml Kubernetes Manifest, that means you’ll have to create them for each environment as things like volume mounts, replicas, names, etc. will be different. This is painful and a hassle because:

• Managing multiple manifests is a pain
• You’ll have to comb through multiple files for each change, like a container image version change
• A ton of tech debt can be accrued

with Kustomize, it’s a better different.

Kustomize allows you to have one file that contains values for every environment, and instead of having to have multiple Kubernetes Manifests, you simply point to the same Kubernetes Manifest for every environment. The only thing that changes are the values in each kustomization.yaml file.

Why Use Kustomize

There are a ton of different use-cases for Kustomize, but you may be wondering to yourself why you’d use it over, say, an environment variables (.env) file or something along those lines.

The first thing is having environment files and managing them for each environment as a separate entity can cause confusion for engineers down the line and ultimately result in tech debt. Kustomize is a Custom Resource that was created to work with the Kubernetes API, so the syntax for creating the files is the same as any other manifest. The CRD is kustomize.config.k8s.io/v1beta1

The next big thing is because Kustomize is part of the Kubernetes API, it works with kubectl, which means you don’t need to have another tool in your toolbelt. You can keep the configurations close to the Kubernetes API instead of looking at outside resources.

The third is when thinking about multiple environments, like Development, Staging, and Production, it’s great to have some segregation. Having the ability to split up environments for each application not only saves you time later when figuring out what goes where in your config, but it also allows you to easily use tools/platforms like Helm Charts for multi-environment configurations and packaging up Kubernetes manifests. When thinking about Helm Charts, you won’t need a config for each environment. Instead, you’d have one set of Helm Charts that’s reading from Kustomize configurations.

Hands-On: Set Up Kustomize

Now that you know about Kustomize and why you’d want to use it, let’s take a look at how to use it from a hands-on perspective.

First, let’s create two Kubernetes Manifests going off of the same file structure in the above screenshot from the What Is Kustomize section.

For the deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginxdeployment
  replicas: 2
  template:
    metadata:
      labels:
        app: nginxdeployment
    spec:
      containers:
      - name: nginxdeployment
        image: nginx:latest
        ports:
        - containerPort: 80

For the service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: nginxservice
spec:
  selector:
    app: nginxdeployment
  ports:
    - protocol: TCP
      port: 80
  type: LoadBalancer

Next, in the base directory, create a file called kustomization.yaml and add in the following:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml

The resources list tells Kustomize what the base configuration is.

Next, create a new directory structure outside of the base directory that goes overlays -->dev. It should look like the screenshot below.

Create a new file called kustomization.yaml (this is different from the kustomization.yaml file in the base directory) and add in the following Manifest:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base/

replicas:
- name: nginx-deployment
  count: 1

What the above Manifest is saying is:

• The base deployment.yaml and service.yaml is in the base directory. Look there for the Kubernetes Manifests.
• For the dev environment, ensure that only 1 replica is deployed.

Now that you have the proper directory structure and Manifest configurations, you can run the Kustomize setup.

To run a Kustomize setup, do it in the same way that you’d deploy a Kubernetes Manifest except:

• Instead of using -f, you would use -k
• Specify the directory that the kustomization.yaml file exists in instead of the actual file itself.

For example, the below command specifies the kustomization.yaml Manifest is in the current directory.

kubectl apply -k .

If you have the same folder structure that you saw in the What Is Kustomize section, you’d want to cd into the dev directory and run kubectl apply -k .

Running the Kustomize configuration from the dev directory, because you’re specifying that 1 replicas should exist, you’ll see five Pods running like in the screenshot below.

Wrapping Up

Although Kustomize definitely makes for far more file management, if you’re already using .env files or another form of variable/parameter files, Kustomize will be a breath of fresh air. You can set a kustomize.yaml for each environment, put in the parameters/variables/config data that you want for each environment, and get a deployment going for any stage in your development cycle.