Connect with your Kubernetes cluster Manually

Howdy. I hope everyone going well recently and my city is in curfew so I utilize this period to make up some new skills. Kubernetes(K8s) is one of my favourite deployment platform. I spent my weekend to connect the cluster on the CI pipeline.

Prerequisites

Before we get our hands on, we'll need to setup some tools like register in GKE. Many other tools out there could do the same things as long as they support K8s. Feel free to use other options but make sure that adjust your setting while you're following with this post.

Compose Kube config file

Let's begin with setting up K8s cluster. Could skip this one if you already got yours.

gcloud container clusters create cluster-1 --zone europe-west3-a

Create a temporary folder to store kube config file and any other stuffs.

# environment variables
SERVICE_ACCOUNT_NAME=deployer
NAMESPACE=default
TARGET_FOLDER="/tmp/kube"
KUBECFG_FILE_NAME="${TARGET_FOLDER}/k8s-${SERVICE_ACCOUNT_NAME}-${NAMESPACE}-conf"

# create temp folder
mkdir -p "/tmp/kube"

Create service account and name it as deployer and better to restrict the permission which only need through.

kubectl create sa deployer --namespace default
kubectl create clusterrolebinding deployer --clusterrole cluster-admin --serviceaccount default:deployer

We want the secret which belongs to the service account and the secret bring us to acquire a certificate and a token. There commands help to do us a favour

SECRET_NAME=$(kubectl get sa deployer --namespace=default -o json | jq -r .secrets[].name)

# put the certificate into temp folder
kubectl get secret --namespace default "${SECRET_NAME}" -o json | jq \
    -r '.data["ca.crt"]' | base64 --decode  > "tmp/kube/ca.crt"

# extract and declare user token
USER_TOKEN=$(kubectl get secret --namespace default "${SECRET_NAME}" -o json | jq -r '.data["token"]' | base64 --decode)

Then we can set kube config file with the information above likes certificate, token etc. Eventually we should have a kube config file with corresponding value to allow our agent docker container connect to K8s.

CONTEXT=$(kubectl config current-context)

CLUSTER_NAME=$(kubectl config get-contexts "${CONTEXT}" | awk '{print $3}' | tail -n 1)

ENDPOINT=$(kubectl config view \
    -o jsonpath="{.clusters[?(@.name == \"${CLUSTER_NAME}\")].cluster.server}")

# set cluster in kube config
kubectl config set-cluster "${CLUSTER_NAME}" \
    --kubeconfig="${KUBECFG_FILE_NAME}" \
    --server="${ENDPOINT}" \
    --certificate-authority="${TARGET_FOLDER}/ca.crt" \
    --embed-certs=true

# set token credentials in kube config 
kubectl config set-credentials \
    deployer-default-${CLUSTER_NAME}" \
    --kubeconfig=/tmp/kube/kube-conf \
    --token="${USER_TOKEN}"

# set context in kube config
kubectl config set-context \
    "deployer-default-${CLUSTER_NAME}" \
    --kubeconfig=/tmp/kube/kube-conf \
    --cluster="${CLUSTER_NAME}" \
    --user="deployer-default-${CLUSTER_NAME}" \
    --namespace=default

# use context with kube config
kubectl config use-context "deployer-default-${CLUSTER_NAME}" \
    --kubeconfig=/tmp/kube/kube-conf

Now the bullets are loaded and ready to roll. We can find the kube config file in this path: /tmp/kube/.

# sample of kube config file
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: BASE64_CA_CERT
    server: https://YOUR_API_IP
  name: k8s
contexts:
- context:
    cluster: k8s
    user: k8s-deployer
  name: k8s
current-context: k8s
kind: Config
preferences: {}
users:
- name: k8s-deployer
  user:
    token: BASE64_TOKEN

Last but definitely not least, we have to grant the role based access control permission for service account and create the permissions-template.yaml

# permissions-template.yaml
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: my_account-clusterrolebinding
subjects:
  - kind: ServiceAccount
    name: my_account
    namespace: my_namespace
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: ""

Time to apply the permission

# adjust the template and create the permission file for service account
sed -e "s|my_account|deployer|g" -e "s|my_namespace|default|g" \
    permissions-template.yaml > permissions_deployer.yaml     

# apply permission
kubectl apply -f permissions_deployer.yaml

Now we can simply use this kube config file on whichever instance it is and connect to our cluster. You could test the connectivity with this command

KUBECONFIG=/tmp/kube/kube-conf kubectl get pods

Conclusion

Done. It's time to plug and play with our Kubernetes cluster. Stay connected. 😁