K3s on Raspberry Pi - Ingress

In this post we’ll look at how ingress works in a K3s cluster. For background, I recommend reading the Networking Section of the K3s documentation.

Ingress Overview

K3s automatically deploys the Traefik Ingress Controller and provides a service load balancer called Klipper. To see everything deployed in the kube-system namespace, run the following command:

kubectl get all --namespace kube-system

NOTE: I have my default context set to rpi-k3s so I don’t have to specify --context on every command.

This shows the following resources related to Traefik:

pod/traefik-97b44b794-dbmz2  
service/traefik
deployment.apps/traefik
replicaset.apps/traefik-97b44b794

And the following resource related to the Klipper load balancer:

pod/svclb-traefik-fc57n
pod/svclb-traefik-mj4md
pod/svclb-traefik-4qnbh
daemonset.apps/svclb-traefik

The traefik deployment contains the specification for a pod with one container using the image rancher/library-traefik:2.4.8 and having container ports 8000 (web) and 8443 (websecure).

The traefik service specifies a LoadBalancer for the traffic pod, and maps port 80 of the service to port 8000 on the traefik container, and port 443 of the service to port 8443 on the traefik container.

Klipper then creates a DaemonSet called svclb-traefik, which creates a pod on each node to act as a proxy to the service. Each of these pods is accessible from the node’s external IP address, and exposes ports 80 and 443, which map to the respective ports on the service.

The overall setup looks something like this:

Next we can deploy an example application and expose it through Traefik.

Ingress Test

This example is adapted from the Traefik documentation.

Create a namespace called whoami:

kind: Namespace
apiVersion: v1
metadata:
  name: whoami

Create a deployment for running two pods with the whoami container:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: traefiklabs
    name: whoami

spec:
  replicas: 2
  selector:
    matchLabels:
      app: traefiklabs
      task: whoami
  template:
    metadata:
      labels:
        app: traefiklabs
        task: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami
          ports:
            - containerPort: 80

Create a service for the whoami deployment:

apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: traefiklabs
    task: whoami

Create an ingress to link the whoami service to Traefik:

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: whoami
  namespace: whoami
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web

spec:
  rules:
    - http:
        paths:
          - path: /bar
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
          - path: /foo
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80

With the above ingress we should now be able to access the paths /foo or /bar on port 80, using the external IP address of any node. The overall setup now looks like the following:

If we open a browser and navigate to http://192.168.1.244/bar, we get the following output:

Hostname: whoami-7d666f84d8-4fs7c
IP: 127.0.0.1
IP: ::1
IP: 10.42.1.4
IP: fe80::c414:76ff:fe4c:75cc
RemoteAddr: 10.42.2.3:39256
GET /bar HTTP/1.1
Host: 192.168.1.244
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:88.0) Gecko/20100101 Firefox/88.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Dnt: 1
Sec-Gpc: 1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.42.0.0
X-Forwarded-Host: 192.168.1.244
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-97b44b794-dbmz2
X-Real-Ip: 10.42.0.0

This shows the request reached one of the whoami containers at 10.42.1.4.

If we refresh the page, we now see the response came from the other whoami container at 10.42.2.5.

Since Traefik is performing round-robing load balancing, the requests will continue to alternate between the two whoami containers.