How to install CUDA on WSL2

Hello again!
This time I was trying to get my Ollama setup running on the GPU because I noticed that, by default, it was only using my CPU cores. I suspected that my CUDA drivers weren't set up correctly on WSL2.

Here are the steps I followed to install it on my machine:

First, I searched on google and found Microsoft's documentation, which you can access here: Enable NVIDIA CUDA on WSL

The firsts steps envolve ensuring you have a WSL instance running with a glibc-based distribution, such as Ubuntu or Debian.

Just a side note: By default, when you install the CUDA drivers on a windows machine, they also include a fully supported driver for WSL2. Therefore, running precompiled CUDA applications shouldn't be a problem. However, if you need to compile a CUDA application targeting a Linux or WSL2 backend, you must follow these steps to install the latest CUDA Toolkit for x86 Linux.

Now, let's get started.

Get started with NVIDIA CUDA

Updated instructions can be found in the NVIDIA CUDA on WSL User Guide, but I will summarize the steps in this article.

Step 1 - Check for WSL updates

Open a PowerShell instance and run the following command:

wsl.exe --update

This ensures that you have the latest WSL kernel available.

Step 2 - Remove the old GPG Key

sudo apt-key del 7fa2af80

Step 3 - Install the Linux x86 CUDA Toolkit Using WSL-Ubuntu Package

Detailed instructions can be found at CUDA Toolkit 12.8 Update 1. However, for convenience, I will replicated them here. I still recommend checking the URL above to ensure you're using the latest available instructions.

Step 4 - Download and Organize the Files

First, download the base installer:

$ wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin

Move it to the correct location:

$ sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600

Next, download the .deb package:

$ wget https://developer.download.nvidia.com/compute/cuda/12.8.1/local_installers/cuda-repo-wsl-ubuntu-12-8-local_12.8.1-1_amd64.deb

Step 5 - Install the .deb Package and Copy the Key

Use dpkg -i to install the .deb package:

$ sudo dpkg -i cuda-repo-wsl-ubuntu-12-8-local_12.8.1-1_amd64.deb

Copy the generated key (which will be prompted in your shell—pay attention!) and replace * in the following command:

$ sudo cp /var/cuda-repo-wsl-ubuntu-12-8-local/cuda-*-keyring.gpg /usr/share/keyrings/

Step 6 - Run apt update and Install CUDA

Now, lets run apt-get update to reload package information and ensure we have the lastest versions:

$ sudo apt-get update

Finally, install the CUDA Toolkit:

$ sudo sudo apt-get -y install cuda-toolkit-12-8

Additional installation options are detailed here

That's it! You're all set.

Comments

[Scott]

Have you tested performance against running CUDA in native Linux? If so, do you notice any slowdown running CUDA in WSL in comparison?

[Gabriel Lavoura]

Hi Scott, thanks for your questions.

I haven't personally run any benchmarks or performance tests, but in my experience, it works well. I came across some research from a few years ago suggesting that WSL2 (being virtualized) achieves CUDA performance comparable to native Linux, with differences within 1% in certain benchmarks. However, other research indicate that native Linux can be up to 33% faster in specific scenarios. I’d say this depends on the library implementation, as PyTorch, for example, has some performance issues. That said, for daily work, I haven't noticed any significant issues.

References:
Developer Nvidia blog
Hacker News discussion
Video Games AI

[Scott]

That's cool - I may have to check this out. Thanks!