Access Nvidia gpu inside docker container in Arch linux

Why

If you need gpu acceleration in a container or just have too much free time on your hands, you can run all kinds of things. For example Large Language Models or GUI apps.
You can even create your own gpu accelerated docker image by using Nvidia ubuntu images

How

We have to install the NVIDIA Container Toolkit
The package can be installed from the Extras repository
```
sudo pacman -S nvidia-container-toolkit
```

Configure the container runtime by using the nvidia-ctk command. The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
```
sudo nvidia-ctk runtime configure --runtime=docker
```
For the change to take effect we have to restart the docker daemon.
```
sudo systemctl restart docker
```

You can add the gpu using docker compose:

service_name:
  image: ....
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: 1
            capabilities: [gpu]

If you use the docker run command, you can do:

sudo docker run --runtime=nvidia --gpus all ....

Resources

Hands-on debugging session: instrument, monitor, and fix

Join Lazar for a hands-on session where you’ll build it, break it, debug it, and fix it. You’ll set up Sentry, track errors, use Session Replay and Tracing, and leverage some good ol’ AI to find and fix issues fast.

RSVP here →