DEV Community

Esseba-dev
Esseba-dev

Posted on

Native vLLM + ROCm 7.15 Runtime for RX 6000 (RDNA2) on Windows 11 — 26 TFLOPS FP16, 62 tok/s, One-Click Install, No WSL2 [RX 6750 XT gfx1031 Verified]

#hardware #llm #machinelearning #performance

I built a native vLLM + ROCm 7.15 runtime for AMD RX 6000 Series on Windows 11 — now with a one-click installer.

AMD lists RX 6750 XT / 6700 XT / 6600 XT as "Runtime only" on Windows with HIP SDK excluded. I built rocBLAS binaries for gfx1031 via ROCm/TheRock to close that gap.

This runs native Windows HIP and ROCm directly, no WSL2 wrapper, no compiler needed.

Tested on AMD Radeon RX 6750 XT 12GB (gfx1031) — Windows 11 Native — August 2026

Verification — Real terminal logs

  1. Environment:

    torch 2.12.0+rocm7.15.0 | cuda_avail True | dev AMD Radeon RX 6750 XT
    vLLM 0.19.1 | plugin vllm_windows_rocm activated | TRITON_ATTN | enforce_eager=True

  2. rocBLAS Benchmark — 26 TFLOPS FP16:

    Device ID 0 : AMD Radeon RX 6750 XT gfx1031 with 12.9 GB memory
    rocBLAS version: 5.7.0.67811f1ee52
    transA,transB,M,N,K,alpha,lda,beta,ldb,ldc,cold_iters,hot_iters,rocblas-Gflops,us
    N,N,4096,4096,4096,1,4096,0,4096,4096, 2, 10, 25977.3, 5290.73
    => 25.97 TFLOPS in 5.29ms

  3. vLLM Inference (Qwen3.5-4B, web chat UI):

    Output: 59-62 tok/s | Init ~1s | hidden thinking (spinner) then answer
    Model auto-detected via /v1/models

Full logs in benchmarks/ and screenshots in assets/ on GitHub.

How it works

  1. TheRock builds clr (HIP) and rocBLAS with Tensile kernels for gfx1031
  2. HSA_OVERRIDE_GFX_VERSION=10.3.0 forces HIP to recognize RX 6750 XT
  3. PyTorch 2.12.0+rocm7.15 links against TheRock runtime => torch.cuda.is_available() True
  4. vLLM plugin vllm_windows_rocm bypasses vllm._C dependency and registers WinRocmAwqGemvKernel with TRITON_ATTN
  5. vLLM engine loads with enforce_eager=True and runs native

Quick Start — One click

Prerequisites: Windows 11 23H2+, RX 6600-6750 XT (gfx1030/gfx1031/gfx1032), Adrenalin 24.x+, ~8GB free disk

  1. Download the repo (or git clone) and run INSTALL.bat as Administrator
  2. The installer downloads the release archives automatically (multi-part join included, no manual steps) and sets up C:\TheRock\ + Python 3.11 + venv
  3. Run CHAT.bat — the browser opens the chat UI. Expected: 59-62 tok/s

VERIFY.bat runs a 3-in-1 check: GPU detection + rocBLAS benchmark + vLLM smoke test.

What's new in v2.0

  • One-click installer: INSTALL.bat auto-downloads and joins split archives (no browser upload/download hassle)
  • Chat web UI (OpenAI-compatible API): hidden thinking with spinner, then the answer streams — like NVIDIA ChatRTX
  • Fat-binary gemv kernels for all RDNA2 (gfx1030/1031/1032) — no per-card rebuild
  • ROCm 7.15 native, vLLM 0.19.1, torch 2.12.0+rocm7.15
  • Faster: 59-62 tok/s output (was 54.2)
  • Repo cleaned: INSTALL.bat + CHAT.bat + VERIFY.bat + MANIFEST.json + docs, archives on GitHub Releases

Known Issues — Please note this may have bugs

This is an experimental reference implementation. It works on my RX 6750 XT but could present bugs on other hardware.

  • Other RDNA2 cards (6600/6600 XT/6700 XT) not yet tested — may need 10.3.0 vs 10.3.1 or rocBLAS rebuild for gfx1030
  • enforce_eager=True required — torch.compile disabled on RDNA2 Windows
  • FP8 / AWQ not tested yet, multi-GPU not tested
  • If you test it on your RDNA2 card, please open an Issue with GPU model and logs — contributions welcome

GitHub Repo:

https://github.com/sebastianmechno-sys/vllm-rocm-windows-rdna2

Includes INSTALL.bat, CHAT.bat, VERIFY.bat, scripts/serve.py (OpenAI-compatible API), chat.html, assets with screenshots, docs/BUILD_ROCBLAS.md, MANIFEST.json with release checksums

Let me know if you test it on other RDNA2 cards.

Built on ROCm/TheRock, PyTorch ROCm, vLLM. Not affiliated with AMD. License Apache 2.0

Top comments (0)