DEV Community

Cover image for Can You Run vLLM on Windows? Yes — via WSL2
Mr Say Nothing
Mr Say Nothing

Posted on Originally published at mrsaynothing.dev

Can You Run vLLM on Windows? Yes — via WSL2

#ai #vllm #windows #wsl2

Originally published on mrsaynothing.dev — the local-LLM cluster lives there.

93,632 GitHub stars since February 2023, and the install requirements still open with one line: "OS: Linux. Python 3.10–3.13." Two lines lower, the note that sends everyone searching: "vLLM does not support Windows natively. To run vLLM on Windows, you can use the Windows Subsystem for Linux (WSL) with a compatible Linux distribution, or use some community-maintained forks." That note is the whole post. The supported road is WSL2; the native road is a fork with your name on the bug reports.

TL;DR

  • No native Windows build exists. The install docs list one OS: Linux — their words, not mine.
  • Yes, you can run it: WSL2. Ubuntu inside Windows, the NVIDIA driver on the Windows side, and the same pip install vllm as any Linux box.
  • Skipping WSL costs the fork tax — community forks only, and you are the QA. Solo user? Ollama runs Windows natively and is probably the right answer anyway.

How do you run vLLM on Windows?

Through WSL2, and it stops feeling like Windows fast.

WSL2 is Microsoft officially admitting the best feature in Windows is Linux. It ships with the OS, installs in one line, and once you are inside Ubuntu, every vLLM instruction on the internet applies to you unchanged:

# PowerShell (Admin) — then reboot when it asks:
wsl --install -d Ubuntu

# inside Ubuntu — the normal Linux install, nothing Windows-specific:
uv venv --python 3.12 && source .venv/bin/activate
uv pip install vllm --torch-backend=auto

# serve something small and prove the GPU passed through:
vllm serve Qwen/Qwen3-0.6B --port 8000
Enter fullscreen mode Exit fullscreen mode

Verify: curl http://localhost:8000/v1/models from Windows — WSL2 forwards localhost, your model comes back in JSON.

The one detail people miss: the GPU driver lives on the Windows side. NVIDIA's CUDA-on-WSL design passes the GPU through from the Windows driver into any distro, so you install the driver once, in Windows, and never inside Ubuntu. Install a driver inside WSL anyway and you can spend an evening unsticking a stack that was never broken.

Can you run vLLM on Windows without WSL?

Only through a community fork — and the fork tax is real: you trade the installer for a compiler.

The docs name the scene themselves: "some community-maintained forks". I read the record on all of them, this week:

  • SystemPanic/vllm-windows — the living one. 663 stars, pushed the day before this post. Native Windows wheels, CUDA 13, Blackwell support, even multi-GPU via NCCL on Windows. This is a serious project doing what the core refuses to.
  • devnen/vllm-windows — 28 stars, last push May 2026. Five months quiet in a project that moves this fast is a retirement notice.
  • dexteri-ai/vllm-windows — a GUI wrapper, created January 2026. Look past the UI and it drives vLLM inside WSL2 — the road it pretends to skip.

Even on the healthy fork you are building CUDA kernels with MSVC patches, losing FlashAttention-3, and re-testing after every upstream release. That is the fork tax. Pay it when a corporate policy blocks WSL and a server is out of reach — not because the one-line install felt too easy.

Why is there no native build at all? vLLM compiles CUDA kernels and leans on NCCL — a toolchain that only fully exists on Linux. The docs say it plainly: "vLLM can fully run only on Linux." A native port means reworking the kernel build for MSVC, which is exactly the maintenance the project chose not to take on, and the forks prove how much work it is by how few survive it.

Does vLLM work on Windows for GGUF models?

Yes — through WSL2, on an NVIDIA or AMD GPU. GGUF in vLLM is GPU-only, so Windows stacks the OS requirement on top of the hardware one. On a CPU-only Windows box, run the same file in Ollama or llama.cpp instead — both are native.

Two locks, in order. The hardware lock first: can vLLM run GGUF — yes, through the official plugin, GPU only, x86 and Arm CPUs refused. The OS lock second, this post: the GPU-only path must run somewhere, and on Windows that somewhere is WSL2 with CUDA pass-through. Both locks open together or not at all. No GPU anywhere in the picture? The file still runs — how to run GGUF models locally covers the native tools.

Should you run vLLM on Windows — or just use Ollama?

Most Windows questions about vLLM are really questions about concurrency. vLLM exists to serve many users from big GPUs with continuous batching. If the user count is "me, at my desk", the honest answer is the bottom row:

Path vLLM? Setup Best for
vLLM inside WSL2 full engine one line + reboot serving a team from one GPU
SystemPanic native fork near-full, you test it compile, chase CUDA WSL blocked by policy, GPU attached
vLLM on a Linux box, Windows as client full engine, elsewhere any OpenAI-compatible client the GPU lives in another room anyway
Ollama, native no — different engine one installer solo use, GGUF, port 11434
llama.cpp, native no — different engine Windows binaries one user, widest quant coverage

The docs' support table says Linux. WSL2 says: fine. And if the question underneath was never about vLLM — just "run a model on my Windows box" — then nothing here was ever locked: Ollama installs natively, serves 127.0.0.1:11434, and when it refuses to start, we have the failure autopsied too.

Top comments (1)

Collapse
 
mrsaynothing profile image
Mr Say Nothing •

The number that made this post: 663 stars on SystemPanic/vllm-windows, pushed the day before I published — a native Windows vLLM exists, and it runs on one team nobody pays. Honest question for the Windows crowd: has anyone here actually put the native fork under real load, or is everyone quietly living in WSL2?