DEV Community

Cover image for Installing K3s with ROCm GPU Operator on Ubuntu 24.04
Vultr profile image Sanskriti Harmukh
Sanskriti Harmukh for Vultr

Posted on with Aashish Chaurasiya Originally published at docs.vultr.com

Installing K3s with ROCm GPU Operator on Ubuntu 24.04

#kubernetes #k3s #rocm #gpu

K3s is a lightweight, certified Kubernetes distribution designed for resource-constrained environments such as edge computing and IoT devices. It simplifies deploying and managing Kubernetes clusters while staying compatible with standard Kubernetes tools and APIs. Paired with the ROCm GPU Operator, K3s can efficiently manage AMD GPUs within the cluster, automating GPU resource management and monitoring so you can deploy and scale GPU-accelerated workloads without manual configuration. This guide assumes a GPU-enabled Ubuntu 24.04 server with an AMD ROCm-compatible GPU and drivers already set up, and walks through installing K3s and Helm, then deploying cert-manager (for automated TLS certificate management) and the ROCm GPU Operator to enable AMD GPU management across the cluster. By the end, you'll have a K3s cluster with GPU resource management running.

1. Install and Configure K3s

1. Install K3s:

$ curl -sfL https://get.k3s.io | sh -
Enter fullscreen mode Exit fullscreen mode

2. Create a .kube directory in your user home directory:

$ mkdir -p $HOME/.kube
Enter fullscreen mode Exit fullscreen mode

3. Symlink the K3s config file as the default kubeconfig:

$ ln -s /etc/rancher/k3s/k3s.yaml $HOME/.kube/config
Enter fullscreen mode Exit fullscreen mode

4. Change the config file permissions so Helm can load it:

$ sudo chmod 755 $HOME/.kube/config
Enter fullscreen mode Exit fullscreen mode

5. Install Helm:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
Enter fullscreen mode Exit fullscreen mode

6. Check the K3s service status:

$ sudo systemctl status k3s
Enter fullscreen mode Exit fullscreen mode

7. Enable the K3s service to start on boot:

$ sudo systemctl enable k3s
Enter fullscreen mode Exit fullscreen mode

2. Install the ROCm GPU Operator

This section installs cert-manager, a dependency for managing TLS certificates in Kubernetes, and deploys the ROCm GPU Operator, which handles GPU management in the cluster.

1. Add the Helm repository for cert-manager:

$ helm repo add jetstack https://charts.jetstack.io --force-update
Enter fullscreen mode Exit fullscreen mode

2. Install cert-manager:

$ helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.15.1 --set crds.enabled=true
Enter fullscreen mode Exit fullscreen mode

This installs cert-manager v1.15.1 — check the official documentation for the latest version before installing.

3. Install the ROCm GPU Operator:

$ helm install amd-gpu-operator --namespace kube-amd-gpu --create-namespace https://github.com/ROCm/gpu-operator/releases/download/v1.0.0/gpu-operator-charts-v1.0.0.tgz
Enter fullscreen mode Exit fullscreen mode

This deploys the AMD GPU Operator into the kube-amd-gpu namespace, enabling management and utilization of AMD GPUs in the cluster.

4. List the deployed Kubernetes resources:

$ kubectl get all --namespace kube-amd-gpu
Enter fullscreen mode Exit fullscreen mode

Output:

NAME                                                                  READY   STATUS    RESTARTS   AGE
pod/amd-gpu-operator-gpu-operator-charts-controller-manager-6bmkfwr   1/1     Running   0          6m
pod/amd-gpu-operator-kmm-controller-5f8d79b46-j4sbw                   1/1     Running   0          6m
pod/amd-gpu-operator-kmm-webhook-server-5d5fc8bdd6-p7qfs              1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-gc-78989c896-lffs2        1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-master-b8bffc48b-dtw7x    1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-worker-m6jfz              1/1     Running   0          6m
.........
Enter fullscreen mode Exit fullscreen mode

Pods can take 5–10 minutes to transition to a Running status.

Next Steps

  • Deploy a ROCm-accelerated workload (PyTorch, TensorFlow, or a custom container) onto the cluster to confirm GPU scheduling works end to end.
  • Look into running Docker with ROCm GPU support if you also need GPU access outside of Kubernetes.
  • Monitor GPU utilization and health through the metrics the GPU Operator exposes.
  • Review the ROCm GPU Operator usage docs for advanced scheduling and node-labeling options.

For the full guide with additional tips, visit the original article on Vultr Docs.

Top comments (0)