Deploying a K3s cluster with SSL

In this post I’ll explain how to deploy a ready to prod k8s cluster (with only 1 node) using k3s and exposing a service using https with an auto-provisioned certificate using Let’s Encrypt

As a plus we’ll install Apisix, an OpenSource ApiGateway, to allow grow up our stack with more nodes+applications

Requirement

• A Linux server with (min) 2Gb

• A registered DNS, for example api.jorge.io

• kubectl installed

• helm installed

Install k3s

K3s is a light implementation of kubernetes ready to production. You can find more information at https://docs.k3s.io/installation

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--tls-san api.jorge.io" sh -

Install Apisix

Apisix is an ApiGateway. You can use it as standalone, with docker or integrate into your cluster. It has a lot of plugins and features ready to be used in production

We’ll use a namespace for Apisix:

kubectl create namespace ingress-apisix

We’ll install a basic/typical implementation:

helm repo add apisix https://charts.apiseven.com
helm update
KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm install apisix apisix/apisix --set gateway.tls.enabled=true --set ingress-controller.enabled=true --namespace ingress-apisix

Install CertManager

CertManager is the agent able to create and provision certificates. It allows to create diferent kinds of certificates and talk with different CAs to create and install them as secrets into our cluster. For our we’ll use it to install free SSL certificates from Let’s Encrypt

helm repo add jetstack https://charts.jetstack.io --force-update
helm update
KUBECONFIG=/etc/rancher/k3s/k3s.yaml helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.14.4

Checking

We’ll check if Apisix is ready

kubectl get all -n ingress-apisix

Pods and services are up and running

The funny part

First thing (can be also last, of course, it doesnt matter) will be to redirect all plain http requests to https

traefik-https-redirect-middleware.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
    name: redirect-https
spec:
    redirectScheme:
        scheme: https
        permanent: true

kubectl apply -f traefik-https-redirect-middleware.yaml

Next, we’ll create the issuer:

cluster-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
    name: letsencrypt
    namespace: ingress-apisix
spec:
    acme:
        server: https://acme-v02.api.letsencrypt.org/directory
        email: [e-mail]
        privateKeySecretRef:
            name: letsencrypt
        solvers:
        - selector: {}
          http01:
            ingress:
            class: traefik

kubectl apply -f cluster-issuer.yaml

We’ve created a ClusterIssuer (so all nodes and all namespaces in the cluster will be able to use it)

It will negociate with Lets Encrypt using the http01 method via traefik and will create a secret into the cluster named letsencrypt (or whatever you specify)

Now, we’ll send all trafic to Apisix

ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
    annotations:
        spec.ingressClassName: traefik
        cert-manager.io/cluster-issuer: letsencrypt
        traefik.ingress.kubernetes.io/router.middlewares: default-redirect-https@kubernetescrd
    name: api-jorge-ingress
    namespace: ingress-apisix
spec:
    rules:
    - host: api.jorge.io
      http:
        paths:
        - pathType: Prefix
            path: /
            backend:
                service:
                    name: apisix-gateway
                    port:
                        number: 80
    tls:
    - hosts:
        - api.jorge.io
      secretName: letsencrypt-cert

kubectl apply -f ingress.yml

Pay attention to the annotations, this is the "trick":

• we will use the traekif ingress

• we’re instructing to the certificate manager to use the previous issuer created

• we’re instructing to traefil to redirect all plain http to https

• we’re instructing we want to use a certificate from a secret (created by cert-manager)

Now all traffic will be served from our k3s cluster using https with a certificate created by Lets’Encrypt

Obviously we’ll have a 404 as Apisix doesn’t know what to do with the request

Deploying a service

We’ll deploy a typical service (httpbin)

kubectl run httpbin --image kennethreitz/httpbin --namespace ingress-apisix

(We are creating a deploying httpbin using the public image kennethreitz/httpbin)

and exposing it

kubectl expose pod httpbin -n ingress-apisix --port 80

It only remains to "link" Apisix with the new httpbin service:

routes.yaml

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
    name: httpbin
spec:
    http:
    - name: httpbin
      match:
        paths:
        - /*
        hosts:
            - api.jorge.io
      backends:
        - serviceName: httpbin
          servicePort: 8080

We are creating an ApisixRoute to route all requests to api.jorge.io to the httpbin service.

And this is all. Right now a request from outside will follow:

• traefik redirecting http to https

• cert-manager negociate, creates and store the certificate

• traefik redirect all requests to Apisix

• Apisix determine which service to use (httpbin in our case)

• httpbin response to the requests