Debugging Kubernetes: Execute kubectl commands with a Service Account

As you already know, Kubernetes is not an easy technology. It's a powerful Cloud technology but it can be time-consuming and painful to debug or troubleshoot a problem and to know how to do in several solutions.

It's for that reason I created a lot of technical sketchnotes about Kubernetes that you can see in "Understanding Kubernetes in a visual way", I created also a new serie of videos that mix sketchnote and audio content, and now my new idea is to publish an article focused on one problem/one need.

Need

In this first article, we will focus on one need:
Usually when I'm connected to a cluster I have all the rights (cluster-admin) so how can I connect to a Kubernetes cluster and execute kubectl commands as an user (as a ServiceAccount)?

Why?

You can have this need in many situations.
Imagine you have a Kubernetes cluster (or severals clusters) with an isolation per namespace (per team project for example), and users in your clusters have rights depending on ClusterRole.

When you want to add or edit user's rights and test the behavior, you'll need to test as an user.

What?

In this article, our need is to execute kubectl commands, in a Kubernetes cluster, as an user who have rights listed in a ClusterRole. The user should only can have read-only rights for secrets in a namespace.

How?

In this use case, you already have Roles and ClusterRoles in your cluster, because you want to test them ^^, but in case you don't have already a ClusterRole and you want to test this RBAC (Role-Based Access Control) settings, the following step allows you to create a ClusterRole:

• 0. Create a ClusterRole that grants read secret access in all namespaces in the cluster

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

Here the list of steps that will allow you to create a namespace, deploy Kubernetes resources that allows you to add some rights in this namespace, generates the kubeconfig file and execute commands in Kubernetes cluster as a ServiceAccount (who have the rights you want to test).

• 1. Create a namespace "test-ns"

$ kubectl create ns test-ns

• 2. Create a ServiceAccount "my-sa-test" in this namespace

$ kubectl create serviceaccount my-sa-test -n test-ns

• 3. Create a RoleBinding that grant secret-reader to my-sa-test

$ kubectl create rolebinding read-pods -n test-ns --clusterrole=secret-reader --serviceaccount=test-ns.my-sa-test

• 4. Create a kubeconfig file for the created ServiceAccount "my-sa-test"

$ export SECRET_NAME_SA=`kubectl get sa my-sa-test -n test-ns -ojsonpath="{ .secrets[0].name }"`
$ export TOKEN_SA=`kubectl get secret $SECRET_NAME_SA -n test-ns -ojsonpath='{.data.token}' | base64 -d`
$ kubectl config view --raw --minify > kubeconfig.txt
$ kubectl config unset users --kubeconfig=kubeconfig.txt
$ kubectl config set-credentials ${SECRET_NAME_SA} --kubeconfig=kubeconfig.txt --token=${TOKEN_SA}
$ kubectl config set-context --current --kubeconfig=kubeconfig.txt --user=${SECRET_NAME_SA}

• 5. Execute kubectl commands in the cluster as the ServiceAccount

$ kubectl --kubeconfig=kubeconfig.txt get secrets -n test-ns

OR you can execute kubectl commands directly with the ServiceAccount token (no need to create another kubeconfig file):

$ export NAMESPACE_SA=test-ns
$ export TEAM_SA=my-sa-test

$ export TOKEN=$(kubectl get $(kubectl get secret -o name -n ${NAMESPACE_SA} |grep  ${TEAM_SA} ) -o jsonpath='{.data.token}' -n ${NAMESPACE_SA} | base64 -d)

$ kubectl --token=${TOKEN} get ns

Cool!
Now I can have only the rights as my user so I can simulate my users behaviors! :-)

Conclusion

I hope this new serie of article, with concrete examples and uses cases will helps you in your Kubernetes understanding journey.

Comments

[Indrit Fejza]

with 1.24 I believe there is no longer a token automatically created from creating a service account.