Django on K8s (Part VI: Advanced Topics)

This part will be a collection of advanced topics that we need to make use of Kubernetes and it is the last part of this tutorial which I hope simplified the technology and gave you a good start, so let's start

For the sake of simplicity, we will use the SQLite version (master branch) of the application, but the MySQL branch will work as well.

Readiness and Liveness checks

They are similar in terms of configuration, but different when each gets invoked, that is why most guides state them together.

Rediness Check: When true the container is ready to accept traffic, can be useful if the container needs some time to prepare the environment and start receiving traffic.
Liveness Check: It is a check that Kubernetes will run periodically to make sure that the container is alive and kicking, in case it fails, the container will be considered dead and it will be terminated and a new container will be started.

Note: We said 'containers' and not 'pods' are the checks are per container.

Types of the Checks

The checks can be one of the following

1. command: that returns 0 if succeed, error otherwise
2. HttpGet: which is a success if the code is between 200 and 399.
3. tcpSocket: if Kubernetes can open the connection to the TCP port then it is a success else a failure.

Configure Probes:

As stated in the documentation, there are several configurations for the probes

• initialDelaySeconds: the number of seconds before the check is initiated (after container starts), default is 0
• periodSeconds: The interval between the checks, default 10 seconds
• failureThreadhold: Try this number of times before considering the probe failed (which means restarting the container), default to 3

Example

Note: In our case (web-app), both readiness and liveness checks shall be 'httpGET', but the sake of the tutorial, we will configure the rediness check as a command and liveness to be httpGET

in the web-app container section, add the following YAML which is available in django-example_checks.yaml

     containers:
      - name: web-app
        image: django-example:v1.0
        ports:
        - containerPort: 80
        readinessProbe:                 #<-- Start here
          periodSeconds: 5
          initialDelaySeconds: 5
          failureThreshold: 100
          exec:
            command:
            - cat
            - /tmp/ready
        livenessProbe:
           httpGet:
             path: /
             port: 80
           initialDelaySeconds: 2
           periodSeconds: 3              #<-- End here
        volumeMounts:
            - name: sqlite-volume
              mountPath: /app/db.sqlite3

So the container are ready, if '/tmp/ready' exists, the container is alive it can response to the HTTP request on port 80.

Note: For the sake of the demo, we set the failureThreshold very high that should NOT be the case on a real environment.

Testing it

django-on-k8s$ kubectl apply -f django-example_checks.yaml

Wait for few seconds and then run

kubectl get po -l app=django-example

You will see that the po is not ready, so to know why it is ready run describe as shown

kubectl describe po -l app=django-example

and check at the end. You will see that the pod is unheathly as rediness probe is failing as shown

To solve the issue, connect to the pod to touch the final, this is describe in the next section 'Executing Commands in Pods'

Executing Commands in Pods

Sometimes, we need to run commands in the containers like we need to run a management command, here if it we demo, how to connect to the pod we created and touch the '/tmp/ready' so the the container becomes ready.

kubectl exec django-example-5c79c94df6-9vkr7 -c web-app -it -- /bin/bash

Notes
1. 'django-example-5c79c94df6-9vkr7' is the pod name
2. -c takes a container name, useful if there are multiple containers in the pod
3. What is after '--' is the command to pass to container, so here we started a bash session
4. '-it' is a flag to state that this is an interactive session.

Then once you get the terminal to the container, enter the following command

touch /tmp/ready

Then press CTRL+D to exit, and after check the status of the pod and you will find it ready as shown below.

Rolling Updates

Writing a complete bug-free application doesn't exist so we need to make an update to our web application now.

Rather greeting the user by the username, we will greet him by the first name.

Note: You can move to update branch if you dont want to make the change manually.

To do the update open 'django_app/accounts/templates/home.html' and Go to line 18 and change it to

  <h1>Welcome {{ request.user.first_name }}!</h1>

Next, you need to build a new docker image, to save time, build it against 'minikube docker'

django-on-k8s$ eval $(minikube docker-env)
django-on-k8s$ docker build -t django-example:v1.1 .

Then edit the deployment on Kubernetes by running

kubectl edit deployments.apps django-example

In the file, search for the image and change it to the new image and also, remove the readiness check as shown below

Now run

kubectl get po

See the old pod are terminating and a new one is starting.

Lets try the new application

Oops!, we made an issue, we didn't handle the case when there is no first name, like with the account created by 'create superuser', so first let's rollback the production to the old image, till we fix the issue.

Rollback

When we have an issue like this, don't freak-out, just ask Kubernetes to rollback like this.

First, let's get the history

kubectl rollout history  deployment django-example 

This will show the change history as shown below

Note: I have multiple changes as I didn't remove the readiness check in the 2nd change.

So we want to go to 1 so we run

kubectl rollout  undo  deployment django-example --to-revision=1

Why is it ready, because of the ready state, so let's touch the file as we did above.

Let's refresh the website.

Wrap-Up

So in the last part of this tutorial, we showed readiness and liveness probes, executing commands in the container and finally how to roll and rollback an update.

There are some interesting topics we didn't cover, like
ConfigMaps and secrets, which allow to store information in Kubernetes (like the mysql password) instead on saving them on file, also Ingress and Ingress Controllers which may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting.

Finally, I hope this tutorial was useful and you learnt something.

Thank you.