DEV Community

Robert
Robert

Posted on

How to use libvmaf_cuda on Windows: an easy-to-follow guide (WSL2 + Docker + NVIDIA)

#ffmpeg #nvidia #docker #tutorial

TL;DR: Running GPU-accelerated VMAF on Windows is surprisingly painful. After spending hours fighting broken builds and undocumented errors, the only reliable path I found is WSL2 + Docker Desktop + NVIDIA Container Toolkit + a CUDA-enabled FFmpeg. This guide walks you through the whole setup using the easyVmaf project, so you can skip the trial-and-error I went through.

This guide uses easyVmaf, a project that ships a Dockerfile.cuda specifically built for this purpose. We'll run it inside WSL2, with Docker Desktop and the NVIDIA Container Toolkit handling the GPU passthrough.

The result: VMAF running at ~15x real-time speed on an RTX 3060 Mobile. A 46-minute video gets analyzed in about 3 minutes. On CPU, the same task would take like an hour.

⚠️ AMD and Intel GPUs will not work with this guide. libvmaf_cuda is NVIDIA-exclusive.

Table of contents

The Problem

If you've ever tried to calculate VMAF on Windows, you already know that:

  • The standard libvmaf filter works, but it runs on the CPU (slow).
  • libvmaf_cuda does not exist in any prebuilt Windows binary (not in gyan.dev, not in BtbN, not anywhere that I know).
  • Most information online is scattered, outdated, or simply missing.

Prerequisites

Before starting, make sure you have:

Component Minimum requirement
Windows Windows 10 (version 2004+) or Windows 11
NVIDIA GPU Any CUDA-capable GPU (I'm using an RTX 3060 Mobile)
NVIDIA drivers Version 525+ (for CUDA 12.x) — download here
Disk space ~30 GB free (WSL + Docker + images)
RAM 16 GB recommended (WSL2 is memory-hungry)

⚠️ Do not install NVIDIA drivers inside WSL. WSL automatically uses the drivers from Windows. Installing Linux drivers inside WSL will break GPU passthrough.

Part 1 — Setting up the environment

This part covers everything needed to get a working Linux + Docker + GPU stack: WSL2, Docker Desktop, and the NVIDIA Container Toolkit.

1.1 Install WSL2

Open PowerShell as Administrator and run

wsl --install
Enter fullscreen mode Exit fullscreen mode

This command will:

  • download and install WSL2 kernel
  • install Ubuntu distro by default

Restart Windows when prompted.

After the restart, open PowerShell again and verify:

wsl --list --verbose
Enter fullscreen mode Exit fullscreen mode

Expected output:

  NAME      STATE           VERSION
* Ubuntu    Running         2
Enter fullscreen mode Exit fullscreen mode

Make sure VERSION is 2. If it says 1, upgrade with:

wsl --set-version Ubuntu 2
wsl --set-default-version 2
Enter fullscreen mode Exit fullscreen mode

Ubuntu update

Open your Ubuntu terminal either by typing in PowerShell

ubuntu
Enter fullscreen mode Exit fullscreen mode

or

wsl
Enter fullscreen mode Exit fullscreen mode

Once in Ubuntu terminal enter:

sudo apt update && sudo apt upgrade -y
Enter fullscreen mode Exit fullscreen mode

1.2 Accessing Windows drives from WSL

If you're new to Linux, one of the first things to understand is that WSL doesn't use C:\, D:\, etc. Instead, it mounts every Windows drive under /mnt/.

Windows path WSL path
C:\Users\YourName\Videos /mnt/c/Users/YourName/Videos
D:\Movies /mnt/d/Movies
E:\Backups\2026 /mnt/e/Backups/2026

1.3 Verify GPU access from WSL

if you already have installed NVIDIA drivers on Windows, run:

nvidia-smi
Enter fullscreen mode Exit fullscreen mode

You should see the nvidia-smi table with your GPU listed. If it doesn't work, your Windows drivers are outdated or WSL isn't configured properly.

Wed Sep 16 09:30:35 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 615.71.08              KMD Version: 616.92        CUDA UMD Version: 13.4     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    On  |   00000000:01:00.0  On |                  N/A |
| N/A   55C    P8             14W /  115W |    1085MiB /   6144MiB |      6%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

1.4 Install Docker Desktop

Download Docker Desktop from: https://www.docker.com/products/docker-desktop/

During installation, make sure to check "Use WSL 2 instead of Hyper-V".

Configure WSL2 integration

Once installed, open Docker Desktop and go to Settings

docker_desktop_settings

General tab:

  • Check Use the WSL 2 based engine
  • Click Apply & Restart

docker_desktop_WSL_integration_backend

Resources → WSL Integration:

  • Check Enable integration with my default WSL distro
  • Turn on the Ubuntu toggle

docker_desktop_WSL_integration

Resources → Advanced (optional):

  • Uncheck "Enable Resource Saver"

docker_desktop_settings_resources

If "Resource Saver" is enabled, Docker will suspend WSL2 after inactivity, and you'll have to restart it manually.

Verify Docker works in WSL

In your Ubuntu terminal:

docker --version
Enter fullscreen mode Exit fullscreen mode

Expected output:

Docker version 27.3.1, build ce12230
Enter fullscreen mode Exit fullscreen mode

⚠️ If you get var/run/docker.sock: connect: permission denied. jump to the Troubleshooting section.

1.5 Install the NVIDIA Container Toolkit

This is the component that allows Docker containers to access the GPU.

Install

In your WSL Ubuntu terminal:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update
sudo apt install -y nvidia-container-toolkit
Enter fullscreen mode Exit fullscreen mode

Configure the Docker runtime 

sudo nvidia-ctk runtime configure --runtime=docker
Enter fullscreen mode Exit fullscreen mode

Expected output:

INFO[0000] Config file does not exist; using empty config
INFO[0000] Wrote updated config to /etc/docker/daemon.json
INFO[0000] It is recommended that docker daemon be restarted.
Enter fullscreen mode Exit fullscreen mode

Restart Docker

Restart Docker Desktop from Windows (either via the restart icon or by quitting and reopening it).

Verify Docker can see the GPU

Once Docker Desktop is running again, execute in your WSL terminal:

docker run --rm --gpus all nvidia/cuda:12.3.2-base-ubuntu22.04 nvidia-smi
Enter fullscreen mode Exit fullscreen mode

You should see the nvidia-smi table with your GPU listed.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 615.71.08              KMD Version: 616.92        CUDA UMD Version: 13.4     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    On  |   00000000:01:00.0  On |                  N/A |
| N/A   55C    P8             14W /  115W |    1085MiB /   6144MiB |      6%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

Part 2 — Building the easyVmaf image

We'll use the easyVmaf project, which ships a Dockerfile.cuda configured for GPU-accelerated VMAF.

2.1 Clone the repository 

cd ~
git clone --depth 1 https://github.com/gdavila/easyVmaf.git
cd easyVmaf
Enter fullscreen mode Exit fullscreen mode

2.2 Critical fix: pin nv-codec-headers

💡 This step is undocumented anywhere else. I figured it out after spending hours debugging the compilation error with the help of AI. Skip it and your build will fail.

The original Dockerfile.cuda clones the latest version of nv-codec-headers, which is incompatible with FFmpeg 8.1. You'll get this error during the build:

libavcodec/nvenc.c:2529:42: error: 'NV_ENC_CLOCK_TIMESTAMP_SET' 
has no member named 'countingType'; did you mean 'countingTypeLSB'?
Enter fullscreen mode Exit fullscreen mode

In recent versions, NVIDIA renamed countingType to countingTypeLSB/countingTypeMSB, but FFmpeg 8.1 still uses countingType.

The fix: Pin nv-codec-headers to version n12.1.14.0, which still uses countingType and already includes the modern CUDA functions (cuStreamCreateWithPriority, cuMemHostAlloc, etc.) that libvmaf_cuda needs.

Edit the Dockerfile:

nano Dockerfile.cuda
Enter fullscreen mode Exit fullscreen mode
  • Press CTRL+W, type nv-codec-headers, press ENTER. You'll land on this line: 
RUN git clone --depth 1 https://git.videolan.org/git/ffmpeg/nv-codec-headers.git && \
    cd nv-codec-headers && \
    make install
Enter fullscreen mode Exit fullscreen mode

Change it by adding n12.1.14.0 before --depth:

RUN git clone --branch n12.1.14.0 --depth 1 https://git.videolan.org/git/ffmpeg/nv-codec-headers.git && \
    cd nv-codec-headers && \
    make install
Enter fullscreen mode Exit fullscreen mode

Save with CTRL+X, then Y, then ENTER.

2.3 Build the image 

docker build -f Dockerfile.cuda -t easyvmaf:cuda .
Enter fullscreen mode Exit fullscreen mode

2.4 Verify the libvmaf_cuda filter

⚠️ The easyvmaf:cuda image defines easyVmaf (its own CLI) as the ENTRYPOINT. To run raw ffmpeg, you must override the entrypoint.

docker run --rm --gpus all --entrypoint ffmpeg easyvmaf:cuda -filters | grep -E "libvmaf|scale_cuda"
Enter fullscreen mode Exit fullscreen mode

Expected output:

.. libvmaf           VV->V      Calculate the VMAF between two video streams.
.. libvmaf_cuda      VV->V      Calculate the VMAF between two video streams.
.. scale_cuda        V->V       GPU accelerated video resizer
Enter fullscreen mode Exit fullscreen mode

If you see libvmaf_cuda, you're done with the setup.

Part 3 — Running VMAF with GPU acceleration

3.1 capabilities=video

By default, --gpus all alone only grants the compute and utility capabilities. It does not mount the video decode/encode libraries libnvcuvid.so.1 (NVDEC) and libnvidia-encode.so.1 (NVENC). Since we tell FFmpeg to decode with -hwaccel cuda, it needs those libraries. Without them, FFmpeg fails with:

Cannot load libnvcuvid.so.1
Failed loading nvcuvid.
Failed setup for format cuda: hwaccel initialisation returned error.
Enter fullscreen mode Exit fullscreen mode

The fix: explicitly request the video capability:

--gpus all,capabilities=video
Enter fullscreen mode Exit fullscreen mode

Now Docker mounts libcuda.so.1 (CUDA compute), libnvcuvid.so.1 (NVDEC), and libnvidia-encode.so.1 (NVENC). FFmpeg can then decode, filter, and analyze entirely on the GPU.

3.2 Full analysis command 

docker run --gpus all,capabilities=video --rm --entrypoint ffmpeg \
  -v "/path/to/your/videos":/videos \
  easyvmaf:cuda \
  -hwaccel cuda -hwaccel_output_format cuda \
  -i "/videos/distorted.mkv" \
  -hwaccel cuda -hwaccel_output_format cuda \
  -i "/videos/reference.mkv" \
  -filter_complex "[0:v]scale_cuda=format=yuv420p[dis];[1:v]scale_cuda=format=yuv420p[ref];[dis][ref]libvmaf_cuda=log_fmt=json:log_path=/videos/vmaf_full.json" \
  -f null -
Enter fullscreen mode Exit fullscreen mode

Real-world result

Here's the output from a test on a 46-minute video file:

[Parsed_libvmaf_cuda_2 @ 0x760b44004f80] VMAF score: 93.558906
speed=14.9x elapsed=0:03:05.05
[out#0/null @ 0x5ef225e22140] video:27438KiB audio:2072848KiB subtitle:0KiB
frame=66265 fps=350 q=-0.0 Lsize=N/A time=00:46:03.79 bitrate=N/A speed=14.6x elapsed=0:03:09.43
Enter fullscreen mode Exit fullscreen mode

3 minutes for a 46-minute video at ~15x real-time speed. That's the whole point of using libvmaf_cuda.

Troubleshooting

var/run/docker.sock: connect: permission denied 

docker: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Head "http://%2Fvar%2Frun%2Fdocker.sock/_ping": dial unix /var/run/docker.sock: connect: permission denied.
Enter fullscreen mode Exit fullscreen mode

Your user doesn't belong to the docker group, which owns /var/run/docker.sock.

fix:

enter this command in your WSL Ubuntu terminal

sudo usermod -aG docker $USER
Enter fullscreen mode Exit fullscreen mode

This adds your user to the docker group. Then close and reopen WSL (group changes only apply to new sessions), and verify:

groups
Enter fullscreen mode Exit fullscreen mode

Expected output:

youruser adm cdrom sudo dip plugdev users docker
Enter fullscreen mode Exit fullscreen mode

Cannot load libnvcuvid.so.1

Missing ,capabilities=video in the --gpus flag. See section 3.1.

NV_ENC_CLOCK_TIMESTAMP_SET has no member named 'countingType'

You didn't pin nv-codec-headers to n12.1.14.0. See section 2.2.

WSL2 hangs after inactivity

Edit C:\Users\YOUR_USER\.wslconfig:

[wsl2]
vmIdleTimeout=-1
Enter fullscreen mode Exit fullscreen mode

Then run wsl --shutdown in PowerShell. Also disable "Resource Saver" in Docker Desktop (see section 1.3).

Conclusion

Setting up libvmaf_cuda on Windows was a long journey. At the start, I couldn't find much information about it — most guides either stop at "use libvmaf on CPU" or assume you're on Linux. Even though this isn't 100% native to Windows (it runs through WSL2 + Docker), it's a solid alternative that is absolutely worth the effort.

Once it's working, you get VMAF analysis at 15x real-time speed, which completely changes what's practical for video quality workflows.

References

Let's Connect

If you found this post useful and you're looking for a Full Stack Developer or a technical writer, feel free to reach out!

Thanks for reading! 🙌

Top comments (0)