K3d & K3s for local Kubernetes environments

Hi, readers this time I want to explain how you can set up a local Kubernetes environment using K3s a lightweight distribution. For this we are going to use K3d to install K3s in you local machine. Specially if you are a developer would be nice to install this local and lightweight Kubernetes installed in your laptop.

This blog post will focus on:
Run a local Kubernetes environment using K3s

What you will learn

Install K3d on a Linux environment
Create a cluster using K3s with K3d
Install Gateway API for the new cluster
Create a Gateway and HttpRoute to expose a deployment
Delete cluster using K3d

These are the topics that you will learn in this blog post.

Requirements

Linux running on your device (Laptop, PC, etc)

Install K3s on a Linux environment

Let's asume that you are using a Linux environment, for example Ubuntu. Now we are going to install k3d. For this follow the next steps:
1. Install K3d, the tool that we are going to use to run K3s using containers in your environment. For this run the following command:

curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

2. Run the following command to return the K3d version installed:

k3d version

Create a cluster using K3s with K3d

With everything installed let's create a cluster with K3d which uses K3s internally to create a Kubernetes cluster using containers. For these follow the next steps:
1. Create a cluster with K3d without installing Traefik and the load balancer default service in K3s:

k3d cluster create mycluster \
--no-lb --network host \
--k3s-arg '--disable=traefik@server:*' --k3s-arg '--disable=servicelb@server:*'

This will show an output like this:

INFO[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default
INFO[0000] Prep: Network
INFO[0000] Re-using existing network 'host' (....)
INFO[0000] Created image volume k3d-mycluster-images
INFO[0000] Starting new tools node...
INFO[0000] Starting node 'k3d-mycluster-tools'
INFO[0001] Creating node 'k3d-mycluster-server-0'
INFO[0001] Using the k3d-tools node to gather environment information
INFO[0001] Starting cluster 'mycluster'
INFO[0001] Starting servers...
INFO[0001] Starting node 'k3d-mycluster-server-0'
INFO[0008] All agents already running.
INFO[0008] All helpers already running.
INFO[0008] Cluster 'mycluster' created successfully!
INFO[0008] You can now use it like this:
kubectl cluster-info

2. Troubleshoot the cluster with the following commands:

kubectl get pods -A #All the pods should appear with STATUS Running
kubectl get nodes #Show all the nodes STATUS as Ready

3. Install MetalLB to have a load balancer provisioner for your cluster:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.4/config/manifests/metallb-native.yaml

4. Found a range of IP Addresses to use for your load balancer for this, you can check the with the route command your available routes. Here is an example:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         sergioarmgpl.ms 0.0.0.0         UG    0      0        0 eth0
10.42.0.0       0.0.0.0         255.255.255.0   U     0      0        0 cni0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.29.192.0    0.0.0.0         255.255.240.0   U     0      0        0 eth0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

In this case we are going to use a cluster that use the internal network of the host that use the eth0 interface. You can also use an AI Tool(Like Gemini or Chatgpt to get a valid range to set for your IPAddressPool on MetalLB, in this case for the output
"172.29.192.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0"
You can ask a valid range of 30 IP addresses. In this case we got the following range:
172.29.206.120-172.29.206.150
With that value we can create a IP address pool for MetalLB to have a range to assign for Load Balancer services in your cluster.
5. Create a IPAddress Pool with the previous range:

cat <<EOF | kubectl create -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: ip-pool
  namespace: metallb-system
spec:
  addresses:
  - 172.29.206.120-172.29.206.150
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2advertisement
  namespace: metallb-system
spec:
  ipAddressPools:
  - ip-pool
EOF

Install Gateway API for the new cluster

Now Let's install the Gateway API that is a common software used in the latest Kubernetes. For this follow the next steps:
1. Install the Gateway API CRDs for the NGINX Gateway Fabric, which is one of the Gateways available to expose application sharing a gateway for your applications:

kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v2.4.2" | kubectl apply -f -

2. Install the NGINX Gateway Fabric:

helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway

Create a Gateway and HttpRoute to expose a deployment

Now with everything installed let's create a Gateway and a HttpRoute to expose a webserver deployment. For this follow the next steps:
1. Create a deployment running NGINX called myapp, with a service using the port 80. The following commands create a namespace, a deployment, a service and a gateway:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: mynamespace
spec: {}
status: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: myapp
  name: myapp
  namespace: mynamespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: myapp
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}
---
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: myapp
  name: myapp
  namespace: mynamespace
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: myapp
  type: ClusterIP
status:
  loadBalancer: {}
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: mygateway
  namespace: mynamespace
spec:
  gatewayClassName: nginx
  listeners:
  - name: http
    port: 80
    protocol: HTTP
EOF

2. Now create a HTTPRoute using the Load Balancer IP Created by the Service. For this run the following commands:

export GW=$(kubectl get svc mygateway-nginx -n mynamespace -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: approute
  namespace: mynamespace
spec:
  parentRefs:
  - name: mygateway
  hostnames:
  - "$GW.nip.io"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: myapp
      port: 80
EOF

Note: The previous commands set the IP Address created by the service in the variable GW, and use the value to create an HttpRoute, which uses nip.io free service to create a free URL as IP Address.nip.io which resolves to the IP Address.
3. Now access the NGINX deployment by running:

curl http://$GW.nip.io

For a more advanced Gateway API configuration check the following link: https://dev.to/sergioarmgpl/goodbye-ingress-hello-gateway-api-migrating-the-right-way-20k9

Delete cluster using K3d

To delete the cluster follow the next command:

k3d cluster delete mycluster

Note: For more commands about K3d, you can run k3d in your CLI to get more commands.

Conclusion about K3d

K3d is a nice tool that helps your to create quick and functional lightweight Kubernetes clusters using K3s. This tutorial covers the basics to create your how cluster. To customize it more, you need some basic of networking to play a little bit, but with this information you can do more than the basics. Also after writing this tutorial I discover that Talos Linux has a similar tool but doesn't include the metrics service in the cluster by default. Also not a big fan of MiniKube because runs slowly than K3s with K3d. Kind is also another good alternative but K3s feels more production ready. If you are looking to don't get your environment dirty with a lot of installation you can still have a clean system by using K3d, I can recommend it to use it for that quick dev environments using Kubernetes without using a lot of ran. Try to run K3s with K3d today, I am using this with my students in the university with really good results. Last to say, see you in my next blog post.