DEV Community: Gabriel Lavoura

How to install CUDA on WSL2

Gabriel Lavoura — Fri, 21 Mar 2025 02:20:16 +0000

Hello again!
This time I was trying to get my Ollama setup running on the GPU because I noticed that, by default, it was only using my CPU cores. I suspected that my CUDA drivers weren't set up correctly on WSL2.

Here are the steps I followed to install it on my machine:

First, I searched on google and found Microsoft's documentation, which you can access here: Enable NVIDIA CUDA on WSL

The firsts steps envolve ensuring you have a WSL instance running with a glibc-based distribution, such as Ubuntu or Debian.

Just a side note: By default, when you install the CUDA drivers on a windows machine, they also include a fully supported driver for WSL2. Therefore, running precompiled CUDA applications shouldn't be a problem. However, if you need to compile a CUDA application targeting a Linux or WSL2 backend, you must follow these steps to install the latest CUDA Toolkit for x86 Linux.

Now, let's get started.

Get started with NVIDIA CUDA

Updated instructions can be found in the NVIDIA CUDA on WSL User Guide, but I will summarize the steps in this article.

Step 1 - Check for WSL updates

Open a PowerShell instance and run the following command:

wsl.exe --update

This ensures that you have the latest WSL kernel available.

Step 2 - Remove the old GPG Key

sudo apt-key del 7fa2af80

Step 3 - Install the Linux x86 CUDA Toolkit Using WSL-Ubuntu Package

Detailed instructions can be found at CUDA Toolkit 12.8 Update 1. However, for convenience, I will replicated them here. I still recommend checking the URL above to ensure you're using the latest available instructions.

Step 4 - Download and Organize the Files

First, download the base installer:

$ wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin

Move it to the correct location:

$ sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600

Next, download the .deb package:

$ wget https://developer.download.nvidia.com/compute/cuda/12.8.1/local_installers/cuda-repo-wsl-ubuntu-12-8-local_12.8.1-1_amd64.deb

Step 5 - Install the `.deb` Package and Copy the Key

Use dpkg -i to install the .deb package:

$ sudo dpkg -i cuda-repo-wsl-ubuntu-12-8-local_12.8.1-1_amd64.deb

Copy the generated key (which will be prompted in your shell—pay attention!) and replace * in the following command:

$ sudo cp /var/cuda-repo-wsl-ubuntu-12-8-local/cuda-*-keyring.gpg /usr/share/keyrings/

Step 6 - Run `apt update` and Install CUDA

Now, lets run apt-get update to reload package information and ensure we have the lastest versions:

$ sudo apt-get update

Finally, install the CUDA Toolkit:

$ sudo sudo apt-get -y install cuda-toolkit-12-8

Additional installation options are detailed here

That's it! You're all set.

Running Ollama on Docker: A Quick Guide

Gabriel Lavoura — Thu, 13 Mar 2025 04:04:28 +0000

Hi it's me again! Over the past few days, I've been testing multiples ways to work with LLMs locally, and so far, Ollama was the best tool (ignoring UI and other QoL aspects) for setting up a fast environment to test code and features.
I've tried GPT4ALL and other tools before, but they seem overly bloated when the goal is simply to set up a running model to connect with a LangChain API (on Windows with WSL).

Ollama provides an extremely straightforward experience. Because of this, today I decided to install and use it via Docker containers — and it's surprisingly easy and powerful..

With just five commands, we can set up the environment. Let's take a look.

Step 1 - Pull the latest Ollama Docker image

docker pull ollama/ollama

If you want to download an older version, you can specify the corresponding tag after the container name. By default, the :latest tag is downloaded. You can check a list of available Ollama tags here.

Step 2 - Create a Docker network

Since we'll typically use and connect multiple containers, we need to specify a shared communication channel. To achieve this, it's a good practice create a Docker network.

docker network create <network-name>

You can check a list of created Docker networks by running the following command:

docker network list

Step 3 - Run the Ollama container

In this tutorial, we're going to run Ollama with CPU only. If you need to use GPU, the official documentation provide a step-by-step guide.

The command to run the container is also listed in the documentation, but we need to specify which network it should connect to, so we must add the --network parameter.

docker run -d --netowk <network-name> -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Step 4 - Run commands inside the Ollama container

To download Ollama models, we need to run ollama pull command.

To do this, we simply execute the command below, which enables the execution inside the container by enabling the interative mode (-it parameter).
Then, we run ollama pull to download the llama3.2:latest (3B), quantized model:

docker exec -it ollama ollama pull llama3.2

Visit the Ollama website to check the list of available models. Now, wait for the download to finish.

You'll get this:

Step 5 - Check the downloaded models

To list the locally available models, just run:

ollama list

You should get this output:

So you're done! Now you have Ollama running (using only the CPU), with the llama3.2:latest model available locally. To run it with a GPU, check the documentation link in Step 3.

I'll share more short notes on working with Ollama and LangChain in the next few days. Stay tuned!

How to Setup Ollama on Google Colab

Gabriel Lavoura — Tue, 11 Mar 2025 04:32:17 +0000

Ok, this is just another "how-to on Google Colab" tutorial... but the purpose of this post is more like a note for my future self.

Step 1 - Install colab-xterm

First, we need to have access to a terminal within the Google Colab code cell. This way, we can install Ollama using the shell script approach.

!pip install colab-xterm  
%load_ext colabxterm

This script will install the colab-xterm package (Perhaps an awesome tool! Thanks @popcornylu, you're awesome!), which allows us to use a terminal within our Colab notebook, even on the free tier.

Step 2 - Open a terminal and install Ollama

To open a terminal, just run the following command in a new cell:

%xterm
#curl -fsSL https://ollama.com/install.sh | sh
#ollama serve & ollama pull llama3.2

Then, paste the curl and ollama commands into the terminal and wait. Once it finishes, you're done!

Step 3 - Test the environment

Create a new cell and run:

!ollama list

If everything is good, it should return the name, ID, size, and last modified date of the model, as shown in the image below.

At this point, you should have a running Ollama server with the Llama3.2:latest model (with 3B parameters) ready to be used.

Step 4 - Clear

Once your session is over, you might want to remove Ollama models to free up space, so create a new cell and run:

!rm -rf /usr/local/bin/ollama

And it's done!

PS: A note from the past: you're welcome, future Gabriel! :)

Como usar Matplotlib com WSL2

Gabriel Lavoura — Sun, 31 Jan 2021 04:59:11 +0000

Nos últimos meses tenho migrado meu ambiente de desenvolvimento para WSL2 (Windows Subsystem for Linux 2), sempre fui um usuário assíduo de dual boot, mas tenho trabalhado em diversos projetos em paralelo, sendo necessário mudar de sistema varias vezes ao dia, seja devido ao Home Office ou por reuniões.
Neste caso Windows torna-se um sistema amigável e com "Batteries Included", porém sinto falta da liberdade e agilidade que o pinguim me oferece.
WSL2 veio para suprir essa demanda, atualmente tenho utilizado como sistema principal de desenvolvimento, a experiência tem sido muito positiva, algumas horas para configurar tudo de primeira, mas para a próxima vez que for necessário reinstalar já está tudo automatizado, utilizando shell script para configurar o ambiente.
Hoje precisei rodar um script Python direto pelo WSL2, um plot simples de uma distribuição gaussiana e me deparei com um problema clássico:

    "UserWarning: Matplotlib is currently using agg, which is a non-GUI.

Para resolver é bem simples, bora lá.

Escolha um X-server do seu gosto!

Eu testei com o Xming e o VcXsrv, ambos funcionaram que é uma beleza, vou seguir com o Xming neste post.

Baixe a última versão disponível do Xming para Windows e basta instalar com next, next, next, install. Aconselho liberar a ferramenta no firewall do Windows para não dar dor de cabeça.

Agora deve-se ter um pouco de atenção, segue os passos ilustrados:

Abra o XLauch, escolha Multiple Windows.
Nesta tela deixe como está.
Marque a caixa chamada "No Access Control".
Agora é só next, finish e está feito!

Vamos para o WSL2

Neste caso eu estou utilizando a imagem do Ubuntu 20.04 disponível na Microsoft Store.

Instale o python-tk:

$ sudo apt-get install python3-tk

Coloque o ip no arquivo resolv.conf, aconselho colocar no seu arquivo .zshrc / .bashrc, para nao precisar inserir novamente, pois o ip do wsl2 não é fixo.

$ export DISPLAY="`grep nameserver /etc/resolv.conf | sed 's/nameserver //'`:0"

E está feito, relativamente simples e eficiente, a seguir uma imagem do resultado final:

Claro que não podia faltar o código que usei, peguei um dos exemplos do matplotlib para ilustrar este post:

  import matplotlib
  import matplotlib.pyplot as plt
  import numpy as np

  # Data for plotting
  t = np.arange(0.0, 2.0, 0.01)
  s = 1 + np.sin(2 * np.pi * t)

  fig, ax = plt.subplots()
  ax.plot(t, s)

  ax.set(xlabel='time (s)', ylabel='voltage (mV)',
         title='About as simple as it gets, folks')
  ax.grid()

  fig.savefig("test.png")
  plt.show()

Então é isso, espero ter ajudado e poupado um pouco do seu tempo! :)