DEV Community: Xavier Geerinck

Automating PyTorch ARM Builds with Docker BuildX for Nvidia CUDA and Python > 3.6

Xavier Geerinck — Thu, 25 Nov 2021 11:19:12 +0000

My Workflow

For a use case I wanted to utilize the Nvidia Jetson for edge inference. One of the bottlenecks here was that my software required a Python version that is greater than 3.6. When looking at the Nvidia Jetson packages for PyTorch it was seen that this was only created for version 3.6.

Searching around, I could find quite a lot of people struggling with this (@todo: PROOF) which made me to look into a solution that would help me automate the building of ARM Wheel of PyTorch that can run on Nvidia Devices (e.g. Nvidia Jetson Nano) and thus support CUDA.

The entire process above took me around ~11 full days, starting of with figuring out how to build the Dockerfile and finally automating the CI process.

Note: I also decided to utilize this article as an entry for a running Hackathon by Dev.to for GitHub Actions

You would wonder why we want to automate this? Well when compiling this initially on my Nvidia Jetson Nano, I couldn't get it to compile past ~80% when using 16Gb of Swap Space (the device only has 2 Gb). After building it on my personal PC, this took ~6h to get it to compile completely. Making it large and long enough to automate it.

The final source code can be found on GitHub

Submission Category:

DIY Deployments, Interesting IoT

Contributions

In any kind of project of this size, there are specific contributions that were made. In my project I believe to have made the following:

Install CUDA on non-GPU devices
Compile PyTorch with CUDA enabled on non-GPU devices
Compile PyTorch for Python > 3.6
Build for ARM with CI through Docker Buildx

Project Outline

As a best practice, I always love to include the project outline of how I tackled the issue above (to share my thought process):

Create a Dockerfile that builds the Wheel
- How do I build for CUDA? (Hardest part)
- How do I cross build?
Create GitHub Action
- How do I cross build? Can I run for ARM specifically?

Dockerfile Creation - Building PyTorch for ARM and Python > 3.6

The hardest part BY FAR is how to compile PyTorch for ARM and Python > 3.6 with CUDA enabled. Because we are going to run on a non-GPU device, thus CUDA is not available on there. I splitted up the Dockerfile into 3 specific sections that can be ran in parallel:

Set up CUDA
Set up PyTorch (the cloning takes a while)
Set up Python 3.9
Compile PyTorch
- I have Nvidia Jetson optimisations included here, thanks QEngineering!
Dockerfile Result

Below you can find the explanation of the respective steps. For the final Dockerfile optimizations where made to decrease the docker layer sizes (by grouping RUN in 1 command)

Setting up CUDA

For CUDA, we do not have the CUDA libraries, nor do we have access to them! There is however a trick that allows us to get CUDA loaded in. We copy over the public key of NVIDIA Jetson and we authorize ourselves to the repository. Then we use our package manager to install them:

V_CUDA_DASH=10-2

# Add the public key
echo "[Builder] Adding the Jetson Public Key"
curl https://repo.download.nvidia.com/jetson/jetson-ota-public.asc > /etc/apt/trusted.gpg.d/jetson-ota-public.asc
echo "deb https://repo.download.nvidia.com/jetson/common ${L4T} main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
echo "deb https://repo.download.nvidia.com/jetson/t186 ${L4T} main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list

# Install the CUDA Libraries
echo "[Builder] Installing CUDA System"
apt-get update
apt-get install -y --no-install-recommends \
    cuda-libraries-$V_CUDA_DASH \
    cuda-libraries-dev-$V_CUDA_DASH \
    cuda-nvtx-$V_CUDA_DASH \
    cuda-minimal-build-$V_CUDA_DASH \
    cuda-license-$V_CUDA_DASH \
    cuda-command-line-tools-$V_CUDA_DASH \
    libnvvpi1 vpi1-dev

# Link CUDA to /usr/local/cuda
ln -s /usr/local/cuda-$CUDA /usr/local/cuda

When we eventually start compiling, we will see CUDA enabled in our CMAKE output 🥳

#   USE_CUDA              : ON
#     Split CUDA          : OFF
#     CUDA static link    : OFF
#     USE_CUDNN           : OFF
#     USE_EXPERIMENTAL_CUDNN_V8_API: OFF
#     CUDA version        : 10.2
#     CUDA root directory : /usr/local/cuda
#     CUDA library        : /usr/local/cuda/lib64/stubs/libcuda.so
#     cudart library      : /usr/local/cuda/lib64/libcudart.so
#     cublas library      : /usr/local/cuda/lib64/libcublas.so
#     cufft library       : /usr/local/cuda/lib64/libcufft.so
#     curand library      : /usr/local/cuda/lib64/libcurand.so
#     nvrtc               : /usr/local/cuda/lib64/libnvrtc.so
#     CUDA include path   : /usr/local/cuda/include
#     NVCC executable     : /usr/local/cuda/bin/nvcc
#     NVCC flags          : <CUT>
#     CUDA host compiler  : /usr/bin/clang
#     NVCC --device-c     : OFF
#     USE_TENSORRT        : OFF

Setting up PyTorch

In a separate docker step, we set up PyTorch and clone it to the working repository (in our case /build/pytorch)

V_PYTORCH=v1.10.0

# Downloads PyTorch to /build/pytorch
git clone --recursive --branch ${V_PYTORCH} http://github.com/pytorch/pytorch /build/pytorch

Setting up Python 3.9

We configure our Python version through the deadsnakes ppa and link it as the default one.

Best practice we should have a venv but since I am running it in a Docker container this should suffice.

# Setting up Python 3.9
RUN add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y python${V_PYTHON} python${V_PYTHON}-dev python${V_PYTHON}-venv python${V_PYTHON_MAJOR}-tk \
    && rm /usr/bin/python \
    && rm /usr/bin/python3 \
    && ln -s $(which python${V_PYTHON}) /usr/bin/python \
    && ln -s $(which python${V_PYTHON}) /usr/bin/python${V_PYTHON_MAJOR} \
    && curl --silent --show-error https://bootstrap.pypa.io/get-pip.py | python

Compiling PyTorch

The last step in the Dockerfile is to compile PyTorch. For this we set the correct environment variabels to enable CUDA and to optimise the building process by disabling some other parts (e.g. MKLDNN, NNPACK, XNNPACK, ... to be turned off).

We also configure it to use clang as the Nvidia Jetson has NEON registers and clang supports those (GCC doesn't).

For our source, we utilize the other layer we created and just copy it from there.

COPY --from=downloader-pytorch /build/pytorch /build/pytorch
WORKDIR /build/pytorch

# PyTorch - Build - Prerequisites
# Set clang as compiler
# clang supports the ARM NEON registers
# GNU GCC will give "no expression error"
ARG CC=clang
ARG CXX=clang++

# Build
rm build/CMakeCache.txt || : \
sed -i -e "/^if(DEFINED GLIBCXX_USE_CXX11_ABI)/i set(GLIBCXX_USE_CXX11_ABI 1)" CMakeLists.txt \
pip install -r requirements.txt
python setup.py bdist_wheel

Copying the result as an Artifact

Docker Buildx is amazing in the sense that we can utilize the --output type-local,dest=. command to output files to our local filesystem, making it such that docker builds and we can export the build result as an artifact.

To achieve this, we pull from the scratch image and copy over our result to it from the other docker layer. Our / path will then contain all the build wheels of PyTorch (e.g. torch-1.10.0a0+git36449ea-cp39-cp39-linux_aarch64.whl)

FROM scratch as artifact
COPY --from=builder /pytorch/dist/* /

Dockerfile Result

Finally, the full Dockerfile will look like this:

# ##################################################################################
# Setup Nvidia CUDA for Jetson
# ##################################################################################
FROM ubuntu:18.04 as cuda-devel

# Configuration Arguments
ARG V_CUDA_MAJOR=10
ARG V_CUDA_MINOR=2
ARG V_L4T_MAJOR=32
ARG V_L4T_MINOR=6

ENV V_CUDA=${V_CUDA_MAJOR}.${V_CUDA_MINOR}
ENV V_CUDA_DASH=${V_CUDA_MAJOR}-${V_CUDA_MINOR}
ENV V_L4T=r${V_L4T_MAJOR}.${V_L4T_MINOR}

# Expose environment variables everywhere
ENV CUDA=${V_CUDA_MAJOR}.${V_CUDA_MINOR}

# Accept default answers for everything
ENV DEBIAN_FRONTEND=noninteractive

# Fix CUDA info
ARG DPKG_STATUS

# Add NVIDIA repo/public key and install VPI libraries
RUN echo "$DPKG_STATUS" >> /var/lib/dpkg/status \
    && echo "[Builder] Installing Prerequisites" \
    && apt-get update \
    && apt-get install -y --no-install-recommends ca-certificates software-properties-common curl gnupg2 apt-utils \
    && echo "[Builder] Installing CUDA Repository" \
    && curl https://repo.download.nvidia.com/jetson/jetson-ota-public.asc > /etc/apt/trusted.gpg.d/jetson-ota-public.asc \
    && echo "deb https://repo.download.nvidia.com/jetson/common ${V_L4T} main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list \
    && echo "deb https://repo.download.nvidia.com/jetson/t186 ${V_L4T} main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list \
    && echo "[Builder] Installing CUDA System" \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
    cuda-libraries-${V_CUDA_DASH} \
    cuda-libraries-dev-${V_CUDA_DASH} \
    cuda-nvtx-${V_CUDA_DASH} \
    cuda-minimal-build-${V_CUDA_DASH} \
    cuda-license-${V_CUDA_DASH} \
    cuda-command-line-tools-${V_CUDA_DASH} \
    libnvvpi1 vpi1-dev \
    && ln -s /usr/local/cuda-${V_CUDA} /usr/local/cuda \
    && rm -rf /var/lib/apt/lists/*

# Update environment
ENV LIBRARY_PATH=/usr/local/cuda/lib64/stubs

RUN ln -fs /usr/share/zoneinfo/Europe/Brussels /etc/localtime

# ##################################################################################
# Create PyTorch Docker Layer
# We do this seperately since else we need to keep rebuilding
# ##################################################################################
FROM --platform=$BUILDPLATFORM ubuntu:18.04 as downloader-pytorch

# Configuration Arguments
# https://github.com/pytorch/pytorch
ARG V_PYTORCH=v1.10.0
# https://github.com/pytorch/vision
ARG V_PYTORCHVISION=v0.11.1
# https://github.com/pytorch/audio
ARG V_PYTORCHAUDIO=v0.10.0

# Install Git Tools
RUN apt-get update \
    && apt-get install -y --no-install-recommends software-properties-common apt-utils git \
    && rm -rf /var/lib/apt/lists/* \
    && apt-get clean

# Accept default answers for everything
ENV DEBIAN_FRONTEND=noninteractive

# Clone Source
RUN git clone --recursive --branch ${V_PYTORCH} http://github.com/pytorch/pytorch

# ##################################################################################
# Build PyTorch for Jetson (with CUDA)
# ##################################################################################
FROM cuda-devel as builder

# Configuration Arguments
ARG V_PYTHON_MAJOR=3
ARG V_PYTHON_MINOR=9

ENV V_PYTHON=${V_PYTHON_MAJOR}.${V_PYTHON_MINOR}

# Accept default answers for everything
ENV DEBIAN_FRONTEND=noninteractive

# Download Common Software
RUN apt-get update \
    && apt-get install -y clang build-essential bash ca-certificates git wget cmake curl software-properties-common ffmpeg libsm6 libxext6 libffi-dev libssl-dev xz-utils zlib1g-dev liblzma-dev

# Setting up Python 3.9
WORKDIR /install

RUN add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y python${V_PYTHON} python${V_PYTHON}-dev python${V_PYTHON}-venv python${V_PYTHON_MAJOR}-tk \
    && rm /usr/bin/python \
    && rm /usr/bin/python${V_PYTHON_MAJOR} \
    && ln -s $(which python${V_PYTHON}) /usr/bin/python \
    && ln -s $(which python${V_PYTHON}) /usr/bin/python${V_PYTHON_MAJOR} \
    && curl --silent --show-error https://bootstrap.pypa.io/get-pip.py | python

# PyTorch - Build - Source Code Setup 
# copy everything from the downloader-pytorch layer /torch to /torch on this one
COPY --from=downloader-pytorch /pytorch /pytorch
WORKDIR /pytorch

# PyTorch - Build - Prerequisites
# Set clang as compiler
# clang supports the ARM NEON registers
# GNU GCC will give "no expression error"
ARG CC=clang
ARG CXX=clang++

# Set path to ccache
ARG PATH=/usr/lib/ccache:$PATH

# Other arguments
ARG USE_CUDA=ON
ARG USE_CUDNN=ON
ARG BUILD_CAFFE2_OPS=0
ARG USE_FBGEMM=0
ARG USE_FAKELOWP=0
ARG BUILD_TEST=0
ARG USE_MKLDNN=0
ARG USE_NNPACK=0
ARG USE_XNNPACK=0
ARG USE_QNNPACK=0
ARG USE_PYTORCH_QNNPACK=0
ARG TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
ARG USE_NCCL=0
ARG USE_SYSTEM_NCCL=0
ARG USE_OPENCV=0
ARG USE_DISTRIBUTED=0

# Build
RUN cd /pytorch \
    && rm build/CMakeCache.txt || : \
    && sed -i -e "/^if(DEFINED GLIBCXX_USE_CXX11_ABI)/i set(GLIBCXX_USE_CXX11_ABI 1)" CMakeLists.txt \
    && pip install -r requirements.txt \
    && python setup.py bdist_wheel \
    && cd ..

# ##################################################################################
# Prepare Artifact
# ##################################################################################
FROM scratch as artifact
COPY --from=builder /pytorch/dist/* /

GitHub Action Creation

Since PyTorch is now finally compiling! It is time to start automating this and publishing them to an Artifact in GitHub (this way we can always trigger it ourselves and kick of the building process). I want to start building automatically, as soon as a Release is published! So for our action, we have the following outline:

Workflow Outline

When a release is created trigger the action
Clone the repository
Setup Docker with Buildx
Run our container
Copy over the Built Wheel to an artifact on GitHub

Used Actions

As for actions, the following actions could be reused:

docker/setup-buildx-action
- I cross-compile for ARM on AMD64 machines in the pipeline
docker/setup-qemu-action
- Configure QEMU to be able to compile for ARM and install the QEMU static binaries
actions/checkout
- Check out a repo
actions/cache
- Allow us to cache the docker layers
actions/upload-artifact
- Upload the output of a directory to GitHub artifacts

Result

Finally resulting in the following GitHub action:

name: ci

# https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows
on:
  push:
    branches: [ main ]
  release:
    types: [ created ]

jobs:
  build_wheels:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout Code
      uses: actions/checkout@v2

    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1

    - name: Set up Docker Buildx
      id: buildx
      uses: docker/setup-buildx-action@v1

    - name: Cache Docker layers
      uses: actions/cache@v2
      with:
      path: /tmp/.buildx-cache
      key: ${{ runner.os }}-buildx-${{ github.sha }}
      restore-keys: |
        ${{ runner.os }}-buildx-

    - name: Build Docker Image
      run: |
      docker buildx build \
          --platform=linux/arm64 \
          --progress=plain \
          --output type=local,dest=./wheels \
          --file Dockerfile.jetson .

    - name: Upload Artifacts
      uses: actions/upload-artifact@v2
      with:
      name: wheels
      path: |
        wheels/*.whl

    # This ugly bit is necessary if you don't want your cache to grow forever
    # till it hits GitHub's limit of 5GB.
    # Temp fix
    # https://github.com/docker/build-push-action/issues/252
    # https://github.com/moby/buildkit/issues/1896
    - name: Move cache
      run: |
      rm -rf /tmp/.buildx-cache
      mv /tmp/.buildx-cache-new /tmp/.buildx-cache

Yaml File or Link to Code

The source code can be found on GitHub with a build of the resulting wheel and including GitHub Action Workflow.

Future Work

Extra Optimizations to be made

Some support could still be added for the Nvidia Jetson Nano by adapting the source code, but this is currently out of scope of this project. These optimisations can be found in QEngineering their post

GitHub Actions Improvements

Currently, Build Arguments are included but not yet used. In theory, the following can be added to the docker buildx command to build for other PyThon versions:

Python 3.8

docker buildx build \
    --platform=linux/arm64 \
    --progress=plain \
    --build-arg PYTHON_MAJOR=3 \
    --build-arg PYTHON_MINOR=8 \
    --output type=local,dest=./wheels \
    --file Dockerfile.jetson .

Conclusion

This project was definitely not easy, taking a long time between builds, figuring out where to build, automating it, ... by sharing this I hope to help the community utilize GPUs more easily with the latest versions.

In a next article, I hope to show you how to run an actual AI model with CUDA enabled on the Nvidia Jetson nano and Python > 3.6 😉

References

All of the above was possible by some contributions of others:

https://github.com/soerensen3/buildx-pytorch-jetson helped me with some of the Dockerfile code (but didn't had CUDA support)

Monitoring the Kubernetes Nginx Ingress Controller with Prometheus and Grafana

Xavier Geerinck — Sun, 20 Sep 2020 19:22:06 +0000

In a previous article I explained how we can set-up an Nginx Kubernetes Ingress Controller, but how can we now monitor this? This is what I would like to tackle in this article, on how we are able to utilize Prometheus and Grafana to start visualizing what is happening on our Ingress Controller.

Prerequisites

After following my previous article you will now have the following components running:

An Nginx Ingress controller deployed in the ingress-nginx namespace
A demo application that we can reach through a ingress route

If you do not have these 2 components running, I would recommend you to go check it out!

Setting up Prometheus

When we open the reference of ingress-nginx online we can see that it should be quite straightforward to install prometheus.

Simply run the following to deploy and configure the Prometheus Server:

kubectl apply --kustomize github.com/kubernetes/ingress-nginx/deploy/prometheus/

Once this is done (~2 min) you will see some output of everything that was created:

serviceaccount/prometheus-server created
role.rbac.authorization.k8s.io/prometheus-server created
rolebinding.rbac.authorization.k8s.io/prometheus-server created
configmap/prometheus-configuration-hct76d4c56 created
service/prometheus-server created
deployment.apps/prometheus-server created

Now on to the "hard" part - reconfiguring our Nginx.

Note: at the time of writing (20-SEP-2020) the documentation was incomplete, making me spend 5h on the issue of reconfiguring nginx...

Reconfiguring Nginx

Reconfiguring Nginx to allow metrics to be sent to Prometheus might sound as an easy task, but actually it isn't... It appears that the official documentation is lacking on this point and that a recent pull request requires the ServiceMonitor to be used.

Installing the ServiceMonitor is a good option, but when checking this on the official repository it shows that this is still in "Beta" so it's not a really a good option to configure this one.

Luckily for us we can always check the values defined in our helm chart, which shows that we can still utilize the prometheus annotations.

Therefor we upgrade our Helm chart through the following command:

helm upgrade ingress-controller ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.metrics.enabled=true \
  --set-string controller.podAnnotations."prometheus\.io/scrape"="true" \
  --set-string controller.podAnnotations."prometheus\.io/port"="10254"

When we validate this through the following command

helm get values ingress-controller --namespace ingress-nginx

We can see that our values are now set as they should:

controller:
  metrics:
    enabled: true
    service:
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"

When I now open the Prometheus dashboard (remember: kubectl get nodes and get the external IP address, and open this with the prometheus port shown through kubectl get svc -A) and start typing ng it shows our metrics!

Installing Grafana

From here on, it's a smooth ride to the finish line! We can simply follow the official documentation and install grafana with:

kubectl apply --kustomize github.com/kubernetes/ingress-nginx/deploy/grafana/

Once this is done, we do the same as for prometheus and open the grafana dashboard in our browser (kubectl get nodes; kubectl get svc -A) where we can use admin:admin as our credentials and run the following steps:

Click "Add data source"
Select "Prometheus"
Enter the details (note: I used http://CLUSTER_IP_PROMETHEUS_SVC:9090)
Left menu (hover over +) -> Dashboard
Click "Import"
Enter the copy pasted json from https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/grafana/dashboards/nginx.json
Click Import JSON
Select the Prometheus data source
Click "Import"

Conclusion

In this article we learned how we can now start monitoring our earlier application its ingress controller. The goal of this is to in a next post be able to auto-scale the created infrastructure based on the incoming requests!

Note: feel free to check the original post at my blog https://xaviergeerinck.com

Creating a Kubernetes Nginx Ingress Controller and create a rule to a sample application

Xavier Geerinck — Sun, 20 Sep 2020 19:16:06 +0000

Whenever you are creating an application that you want to expose to the outside world, it's always smart to control the flow towards the application behind it. That's why Kubernetes has something called Kubernetes Ingress. But what is it?

Kubernetes Ingress

Kubernetes Ingress allows you to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. The traffic routing is then controlled by rules defined in the ingress sources.

For this article, I will explain how you can get started on creating your own Nginx Ingress Controller. Of course this is not the only possibility, so feel free to check other ingress controllers such as Istio, HAProxy, Traefik, ...

Some advantages of using an ingress controller:

Rate limiting, Timeouts, ...
Authentication
Content based routing

Sample Hello World application

Before we create our controller, let's get started on creating a simple demo application. The only thing our application will do is process the HTTP request, wait a couple of seconds and return a "Hello World" response.

Creating our sample app

I decided to create this application in Node.js. So if you have npm and node installed, run the following commands:

npm init -y
npm i express --save

Whereafter you can create an index.js file with the following content:

const express = require('express')
const app = express()
const port = 3000

app.get('/', async (req, res) => {
  console.log('Got request, waiting a bit');
  await delay(10 * 1000);
  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

const delay = async (timeout = 1000) => {
  return new Promise((resolve, reject) => setTimeout(resolve, 1000));
}

Packaging it as a container

Since everything is created in terms of application code, we can package it all up into a Docker container by creating a Dockerfile:

Dockerfile

FROM node:latest

WORKDIR /usr/src/app

# Install deps
RUN apt-get update

# Create Certificate
RUN apt-get install ca-certificates

# Install Package.json dependendencies
COPY package.json .
RUN npm install

# Copy Source Code
ADD . /usr/src/app

CMD [ "npm", "run", "start" ]
EXPOSE 3000

That we can build with (choose one for your use-case):

# Local build (for local use)
# Note: when using minikube, make sure to run `eval $(minikube docker-env)` to build images in minikube context
docker build -t local/node-sample-helloworld .

# Remote build (to push to docker repository)
docker build -t thebillkidy/node-sample-helloworld .
docker push thebillkidy/node-sample-helloworld

Running it on Kubernetes

Once it is build, we can now run it on our Kubernetes cluster. For that we create a Deployment YAML file:

kubernetes.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: d-node-sample-helloworld
spec:
  selector:
    matchLabels:
      app: node-sample-helloworld
  replicas: 1
  template:
    metadata:
      labels:
        app: node-sample-helloworld
    spec:
      containers:
      - name: main
        image: thebillkidy/node-sample-helloworld:latest # if local, utilize local/node-sample-helloworld
        imagePullPolicy: Always # if local, utilize Never
        ports:
        - containerPort: 3000

That we can apply with kubectl apply -f kubernetes.yaml and should now show the following after running kubectl get deployments -A:

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
d-node-sample-helloworld   1/1     1            1           37s

Kubernetes is getting more popular everyday and it's no wonder why! When you are running applications on-premise or in-cloud, the possibility of having the applications in a portable way is a strong one! Removing the friction for scaling-out your application when you are ready for it, or even bursting scenarios.

Nginx Ingress Controller

We now have a simple Hello World application running, but it's only available internally! What we could do now is expose it through Kubernetes and a LoadBalancer, but let's actually utilize our Ingress Controller here! So let's get started creating this Ingress Controller.

Installation

The first step that we should do is create the NGINX Ingress controller. For this we can follow these steps:

Note: before the stable/nginx-ingress chart was utilized. But this is now deprecated!

# ref: https://github.com/kubernetes/ingress-nginx (repo)
# ref: https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx (chart)

# 1. Create namespace
kubectl create namespace ingress-nginx

# 2. Add the repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

# 3. Update the repo
helm repo update

# 4. Install nginx-ingress through Helm
helm install ingress-controller ingress-nginx/ingress-nginx --namespace ingress-nginx

Once we ran the above, we should now be able to access the ingress controller by loading the external IP (kubectl -n ingress-nginx get svc).

Note: when working with Minikube or others, we can utilize kubectl port-forward svc/ingress-controller-ingress-nginx-controller --namespace ingress-nginx --address 0.0.0.0 8000:80 and access it on http://localhost:8000

We are now ready to expose our application!

Exposing our Application

Once an ingress controller is created, we need to expose our application internally:

kubectl expose deployment d-node-sample-helloworld --name svc-node-sample-helloworld

and configure our Ingress controller to route traffic to it as defined in the Kubernetes Ingress API. By creating a YAML file:

ingress-node-sample-helloworld.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-node-sample-helloworld
  annotations:
    # Target URI where the traffic must be redirected
    # More info: https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/rewrite/README.md
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    # Uncomment the below to only allow traffic from this domain and route based on it
    # - host: my-host # your domain name with A record pointing to the nginx-ingress-controller IP
    - http:
        paths:
        - path: / # Everything on this path will be redirected to the rewrite-target
          backend:
            serviceName: svc-node-sample-helloworld # the exposed svc name and port
            servicePort: 3000

Which we apply with kubectl apply -f ingress-node-sample-helloworld.yaml

Now once this is applied, we should be able to execute a cURL request to access our application! So let's try this:

# Execute a GET request with the specified host and IP
# Note: the Host should match what is written in spec.rules.host
curl -k -X "GET" -H "Host: my-host" http://YOUR_IP

Or we can also open it in our browser and navigate to http://YOUR_IP

Note: If this is not working, check kubectl describe ing and make sure that the configuration is correct

Conclusion

In this article a demonstration was made on how you can set up your own ingress controller for Kubernetes. This is of course a small step in the entire chain of use cases, where most often you want to do more such as rate limiting, or even monitoring it.

The next article will explain more in-depth how you are able to start monitoring what we have just set-up through Prometheus and visualize all of it in Grafana.

Note: feel free to check the original post at my blog https://xaviergeerinck.com