Access Nvidia gpu inside docker container in Arch linux

Why

• If you need gpu acceleration in a container or just have too much free time on your hands, you can run all kinds of things. For example Large Language Models or GUI apps.
• You can even create your own gpu accelerated docker image by using Nvidia ubuntu images

How

• We have to install the NVIDIA Container Toolkit

• The package can be installed from the Extras repository
  

  sudo pacman -S nvidia-container-toolkit
  

1. Configure the container runtime by using the nvidia-ctk command. The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
   

   sudo nvidia-ctk runtime configure --runtime=docker
   

2. For the change to take effect we have to restart the docker daemon.
   

   sudo systemctl restart docker
   

3. You can add the gpu using docker compose:
   

   service_name:
     image: ....
     deploy:
       resources:
         reservations:
           devices:
             - driver: nvidia
               count: 1
               capabilities: [gpu]
   

   If you use the docker run command, you can do:
   

   sudo docker run --runtime=nvidia --gpus all ....
   

Resources

• NVIDIA Container Toolkit documentation
• Toolkit Arch package page