How Kyverno helps with policy management

Originally published on mccricardo.com.

The proliferation of the cloud and Kubernetes made it "easier" to provision new environments dynamically, democratizing the access to resources that can be used, for example, for testing. That comes with its own set of challenges; dynamically provisioning Roles and RoleBindings being some of them. If we think of a new "environment" as a namespace in Kubernetes, each of them will have its own Roles and RoleBindings and we would like for them to be managed dynamically.

We could attack this problem with several approaches but we'll leverage Kyverno and one of its advanced features to manage that for us. Kyverno "is a policy engine designed for Kubernetes. (...) Kyverno policies can validate, mutate, and generate Kubernetes resources". Kyverno can:

• validate - check resource configurations for policy compliance;
• mutate - modify resources during admission control;
• generate - create additional resources based on resource creation or updates.

We will leverage Kyverno's generate capabilities to create Roles and RoleBindings every time a new namespace is created.

Get the ball rolling

We will be spinning up a local Kubernetes cluster with k3d and we'll need the following tools to make it happen:

• kubectl
• k3d

We start by creating a local Kubernetes cluster with k3d:

k3d cluster create -a 2 kyverno
k3d kubeconfig merge kyverno --switch-context

We then deploy Kyverno into the cluster:

kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/definitions/release/install.yaml

Because we're generating resources we need to give Kyverno permissions to do so:

# kyverno.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: kyverno:generatecontroller
rules:
- apiGroups:
  - "*"
  resources:
  - namespaces
  - networkpolicies
  - secrets
  - configmaps
  - resourcequotas
  - limitranges
  - pods
  - roles
  - rolebindings
  verbs:
  - create 
  - get
  - update
  - delete
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: kyverno-admin-generate
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kyverno:generatecontroller
subjects:
- kind: ServiceAccount
  name: kyverno-service-account
  namespace: kyverno

There's a bit more than what we need but we can go ahead and apply these permissions:

kubectl apply -f kyverno.yaml

Now that Kyverno is equipped with the necessary permissions to manage our Roles and RoleBindings we need to create the necessary policies that handle that:

# role-policy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: role
spec:
  rules:
  - name: role
    match:
      resources:
        kinds:
        - Namespace
    exclude:
      resources:
        namespaces:
        - "kube-system"
        - "default"
        - "kube-public"
        - "kyverno"
    generate:
      kind: Role
      name: pod-reader
      namespace: "{{request.object.metadata.name}}"
      data:  
        rules:
          - apiGroups: [""] # "" indicates the core API group
            resources: ["pods"]
            verbs: ["get", "watch", "list"]
---
# rolebinding-policy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: rolebinding
spec:
  rules:
  - name: rolebinding
    match:
      resources:
        kinds:
        - Namespace
    exclude:
      resources:
        namespaces:
        - "kube-system"
        - "default"
        - "kube-public"
        - "kyverno"
    generate:
      kind: RoleBinding
      name: read-pods
      namespace: "{{request.object.metadata.name}}"
      data:  
        subjects:
          - kind: User
            name: "{{request.object.metadata.labels.user}}"
            apiGroup: rbac.authorization.k8s.io
        roleRef:
          kind: Role
          name: pod-reader
          apiGroup: rbac.authorization.k8s.io

To achieve our goal we create two policies, one to create a Role and another for a RoleBinding where Kyverno will be "listening" for changes to Namespace resources (excluding the ones it doesn't care about). The {{request.object}} contains the fields of the Namespace resource and Kyverno makes use of them to identify the namespace and decide to whom the permission is given, which in our case is a specific user:

kubectl apply -f role-policy.yaml
kubectl apply -f rolebinding-policy.yaml

If we now create a namespace, with the proper fields, Kyverno will do it's magic:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    user: jane
  name: test-namespace

➜ kubectl -n test-namespace get role                   
NAME         CREATED AT
pod-reader   2020-12-27T20:09:49Z
➜ kubectl -n test-namespace get rolebinding 
NAME       ROLE              AGE
read-pods   Role/pod-reader   11s

How did Kyverno do this?

Kubernetes has the concept of Admission Controller which is software that can intercept requests to the API server, prior to its creation, and do something with it. It also possesses the concept of Dynamic Admission Control which are HTTP callbacks that provide similar functionality, allowing us to code the behavior we desired. While we could have developed our own Dynamic Admission Control, Kyverno already has the capabilities we seek and makes leveraging it a no-brainer.