Kyverno For Kubernetes

Security is arguably one of the largest issues in Kubernetes environments. Other than the fact that almost all engineers are still trying to figure out Kubernetes, once it’s figured out, the next concern is security.

Red Hat’s most recent report states that 70% of security issues are Kubernetes misconfigurations. Typically, misconfigurations are due to having poor policies or no policies in place.

That’s where a platform like Kyverno comes into play.

In this blog post, you’re going to learn about Kyverno, why it’s important, how it’s compared to Open Policy Agent (OPA), and how you can get started with it in production today.

What’s Kyverno?

When you deploy anything to Kubernetes, whether it’s a Pod, ConfigMap, or an entire containerized microservices, you’ll typically want some type of rules around them. For example, one of the best practices that every engineer should implement is to never use the latest tag of a container image inside of a Kubernetes Manifest in production. Why? Because the latest tag is typically a dev build that’s still being worked out. Instead, engineers should use a specific version of the containerized app.

Here’s the thing - how do you block engineers from using the latest tag?

That’s what a policy manager like Kyverno does.

Kyverno manages policies, whether they’re for security or general best practices, in your Kubernetes environment. You can set up policies for something like the latest tag example above, or something more security-centric like verifying container images for a software supply chain security. Kyverno works outside of standard Kubernetes Manifests as well. You can use overlays like Kustomize or Helm and implement Kyverno policies.

What About OPA?

Open Policy Agent (OPA) is gaining increased popularity in the policy management community, primarily because it’s vendor agnostic. Meaning, you can create policies for any environment, not just Kubernetes.

With Kyverno, it’s only for Kubernetes.

If you need policy management outside of Kubernetes, Kyverno won’t work and you may be better off going the OPA route.

What should be said for Kyverno, which is a major upside over OPA, is that Kyverno policies are created via Kubernetes Manifests. OPA policies are created with Rego, a configuration language that sort of looks like Go (golang). With that being said, if you don’t already know Rego, implementing OPA may be a burden. Where-as with Kyverno, you’re already used to Kubernetes Manifests, so it’s less of a lift.

How Do Policies Work?

A policy manager like Kyverno or OPA are Kubernetes Controllers that run inside of a Kubernetes cluster. Because of the underlying features of a Kubernetes Controller, like managing the state of deployed resources, it makes sense for a policy manager to be a Controller. The whole idea is to manage policies, and you can’t manage policies without proper state in place.

With policy managers, it’s all about webooks. There are policies in place inside of the Kubernetes cluster and whenever a request is made, like creating a Kubernetes Deployment resource, the webhook is received and makes a callback to the Kubernetes API server. It then checks to confirm that all of the policies in place are passed and if they are, the Kubernetes resource will be created. If they aren’t, the resource will be blocked.

Deploying Kyverno

When you deploy Kyverno in a Kubernetes cluster, a few Kubernetes resources are created:

• A Namespace where Kyverno will live
• The Kyverno CRD’s
• The Kyverno container image
• The Kyverno services
• Configmaps
• A service account with cluster roles and rolebindings. Please note that the service account does contain create and delete roles.

You can find the entire Kubernetes configuration for more detail here: https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml

Now that you know what the Kubernetes configuration consists of, let’s look at how to install and configure Kyverno.

First, run the Kubernetes Manifest installation.

kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml

If you’d prefer to use a Helm Chart, use the code below.

helm repo add kyverno https://kyverno.github.io/kyverno/

helm repo update

helm install kyverno kyverno/kyverno -n kyverno --create-namespace

For production purposes and ultimately what you want your environment to look like, take a look at the documentation found here.

After installing Kyverno, you should see the following output on your terminal.

Run the following command to see all of the resources in the Kyverno namespace.

kubectl get all -n kyverno

Now that Kyverno is installed, let’s take a look at policy configurations.

Luckily, Kyverno already has a ton of policies that are available. As mentioned in one of the previous sections, they’re all written in YAML as a Kubernetes Manifest. You can see a ton of the policies here.

For the purposes of this configuration, let’s use the policy called Disallow Latest Tag, which blocks a Kubernetes resource from being deployed with the latest container image.

Open up a new file and call it disallow.yaml, and paste in the following policy.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
  annotations:
    policies.kyverno.io/title: Disallow Latest Tag
    policies.kyverno.io/category: Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/description: >-
      The ':latest' tag is mutable and can lead to unexpected errors if the
      image changes. A best practice is to use an immutable tag that maps to
      a specific version of an application Pod. This policy validates that the image
      specifies a tag and that it is not called `latest`.      
spec:
  validationFailureAction: audit
  background: true
  rules:
  - name: validate-image-tag
    match:
      resources:
        kinds:
        - Pod
        - Deployment
    validate:
      message: "Using a mutable image tag e.g. 'latest' is not allowed."
      pattern:
        spec:
          containers:
          - image: "!*:latest"

If you looked at the policy via the link above for Disallow Latest Tag, you’ll notice that the kinds used to be Pod. The code was changed to Deployment and Pod instead of just Pod, which is why what’s on the Kyverno website looks different in the Kubernetes Manifest above.

Apply the Manifest to your Kubernetes cluster.

kubectl create -f Kyverno/disallow.yaml

clusterpolicy.kyverno.io/disallow-latest-tag created

Next, create a new YAML file called nginx.yaml and paste in the following configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginxdeployment
  replicas: 2
  template:
    metadata:
      labels:
        app: nginxdeployment
    spec:
      containers:
      - name: nginxdeployment
        image: nginx:latest
        ports:
        - containerPort: 80

Notice how it’s using the latest tag of the Nginx container image.

Apply the nginx.yaml Manifest and you should see an output similar to the below.

kubectl create -f nginx.yaml
Error from server: error when creating "nginx.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

policy Deployment/default/nginx-deployment for resource violation: 

require-labels:
  autogen-check-for-labels: 'validation error: label ''app.kubernetes.io/name'' is
    required. rule autogen-check-for-labels failed at path /spec/template/metadata/labels/app.kubernetes.io/name/'

Congrats! You have officially set up Kyverno policies for Kubernetes.