DEV Community

Cover image for Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04
Vultr profile image Sanskriti Harmukh
Sanskriti Harmukh for Vultr

Posted on with Aashish Chaurasiya Originally published at docs.vultr.com

Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04

#kubernetes #nvidia #gpu #k3s

K3s is a lightweight, fully compliant Kubernetes distribution designed for simplified deployment and operation in resource-constrained environments. With a small memory footprint and a single-binary architecture, K3s is optimized for edge computing, IoT devices, and environments where a traditional Kubernetes setup would be too resource-intensive. It strips out non-essential features and dependencies, making it faster and easier to install while retaining all core Kubernetes functionality. This guide installs and configures K3s on a GPU-enabled Ubuntu 22.04 server, configures Kubernetes with Helm, sets up firewall rules for external cluster access, and installs the NVIDIA GPU Operator to manage GPU resources within K3s for optimized GPU workloads. By the end, you'll have a lightweight Kubernetes cluster with GPU scheduling ready for production workloads.

Prerequisites: a GPU-enabled server (with an NVIDIA GPU and its driver already installed) running Ubuntu 22.04, accessed over SSH as a non-root user with sudo privileges.

Install K3s and the NVIDIA GPU Operator

1. Disable the Docker system service:

$ sudo systemctl disable docker
Enter fullscreen mode Exit fullscreen mode

2. Stop the Docker system service:

$ sudo systemctl stop docker
Enter fullscreen mode Exit fullscreen mode

3. View the Docker service status and verify it's inactive:

$ sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

Output:

○ docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
     Active: inactive (dead) since Wed 2024-11-06 20:45:17 UTC; 5s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
    Process: 1088 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=0/SUCCESS)
   Main PID: 1088 (code=exited, status=0/SUCCESS)
        CPU: 404ms
Enter fullscreen mode Exit fullscreen mode

4. Install K3s:

$ curl -sfL https://get.k3s.io | sh -
Enter fullscreen mode Exit fullscreen mode

Output:

[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Enter fullscreen mode Exit fullscreen mode

5. Create a .kube directory in your home directory (replace linuxuser with your actual user):

$ mkdir -p /home/linuxuser/.kube
Enter fullscreen mode Exit fullscreen mode

6. Symlink the K3s config as the default kubeconfig so kubectl and other CLI tools can find it:

$ ln -s /etc/rancher/k3s/k3s.yaml /home/linuxuser/.kube/config
Enter fullscreen mode Exit fullscreen mode

7. Change the config file's permissions to 755 so Helm can read it:

$ sudo chmod 755 /home/linuxuser/.kube/config
Enter fullscreen mode Exit fullscreen mode

8. Install Helm to manage Kubernetes applications:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
Enter fullscreen mode Exit fullscreen mode

9. Add the NVIDIA Helm repository:

$ helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update
Enter fullscreen mode Exit fullscreen mode

This adds the NVIDIA Helm chart repository (which contains the GPU Operator and other NVIDIA tools for Kubernetes) and refreshes Helm's local repository cache.

10. Install the NVIDIA GPU Operator:

$ helm install --wait gpu-operator nvidia/gpu-operator --create-namespace -n gpu-operator --set driver.enabled=false
Enter fullscreen mode Exit fullscreen mode

This installs the GPU Operator into a new gpu-operator namespace and skips the bundled NVIDIA driver installation (--set driver.enabled=false) since the driver is already installed on the host.

11. Add the required firewall rules:

$ sudo ufw allow 6443/tcp && sudo ufw allow 30000:32767/tcp && sudo ufw allow 30000:32767/udp
Enter fullscreen mode Exit fullscreen mode

12. Enable the K3s system service so it starts on boot:

$ sudo systemctl enable k3s
Enter fullscreen mode Exit fullscreen mode

13. View the K3s service status and verify it's running:

$ sudo systemctl status k3s
Enter fullscreen mode Exit fullscreen mode

Output:

● k3s.service - Lightweight Kubernetes
 Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
 Active: active (running) since Wed 2024-11-06 20:45:47 UTC; 7min ago
   Docs: https://k3s.io
Main PID: 2883 (k3s-server)
  Tasks: 206
 Memory: 3.9G
    CPU: 1min 19.382s
 CGroup: /system.slice/k3s.service
 ...................................
Enter fullscreen mode Exit fullscreen mode

Next Steps

  • Deploy a CUDA-enabled workload or sample pod to confirm the GPU Operator schedules pods onto the GPU correctly
  • Add a StorageClass for persistent workloads — run kubectl get storageclass and substitute your cloud provider's actual StorageClass name
  • If you expose services externally with a LoadBalancer, note that some cloud providers require a provider-specific LoadBalancer annotation
  • Install kubectl and NVIDIA's DCGM exporter to monitor GPU utilization across the cluster

For the full guide with additional tips, visit the original article on Vultr Docs.

Top comments (0)