DEV Community: AIVisionsLab

Running Local AI on an AMD RX 580 in 2026 — The Complete Vulkan Guide

AIVisionsLab — Thu, 11 Jun 2026 23:35:39 +0000

Running Local AI on an AMD RX 580 in 2026 — The Complete Vulkan Guide
"Your RX 580 can't run AI. Buy a new GPU."

That was the consensus in 2026. AMD dropped ROCm support for Polaris/GCN4 architecture in v5.x. DirectML crashes with OpaqueTensorImpl. OpenVINO fails silently on Forge. Every mainstream AI stack gave up on this card.

We didn't.

This is the complete technical record of how we built a full local AI production stack on an AMD RX 580 8GB — running LLMs at 17 tok/s, generating images in 72 seconds, transcribing audio 150× faster than CPU, and even cloning voices. All offline. All free. All on hardware that cost under $50.

The Hardware
Component Spec
GPU AMD RX 580 2048SP 8GB GDDR5 (Polaris / GCN4)
CPU Intel Xeon E5-2690 v3 — 12c/24t · 3.5GHz (2014)
RAM 32GB DDR4 REG ECC Quad Channel
Storage NVMe 1TB — 1.7–3.5 GB/s
OS Windows 10 Pro + WSL2 / Ubuntu 26.04 LTS
The RX 580 2048SP is the mining-variant with 2048 shader processors instead of the original 2304SP. It's everywhere on the used market for under $50. It performs identically through Vulkan.

One thing nobody talks about: storage matters as much as the GPU. Moving from HDD to NVMe reduced FLUX.1 model load time from 25 minutes to 30 seconds. The bottleneck was never the GPU.

Why Vulkan?
The entire mainstream AI stack runs on either CUDA (Nvidia-only) or ROCm (AMD dropped Polaris in v5.x). That leaves legacy AMD GPUs with no official path.

But there's a third option: Vulkan — a universal graphics/compute API that works on any modern GPU, including the RX 580, which has supported Vulkan 1.x since its 2017 drivers.

The ggml project (the engine behind llama.cpp and stable-diffusion.cpp) implements Vulkan compute backends in pure C++. This means you can compile directly against the Vulkan API and completely bypass the ROCm/CUDA ecosystem. No driver packages. No compatibility layers. Just the GPU doing math.

What We Tried Before Vulkan (And Why It All Failed)
Before finding the working path, we hit every dead end:

DirectML + ComfyUI — The GPU gets detected as privateuseone0, but then:

NotImplementedError: Cannot access storage of OpaqueTensorImpl
DirectML wraps tensor data in opaque objects that ComfyUI's attention backends literally cannot read. Also: Microsoft hasn't updated it since September 2024. It's abandoned.

ROCm on Polaris — AMD officially dropped GCN4/Polaris in ROCm v5.x. Compatibility layers via WSL2 generate kernel panics under inference load. There is no Windows support. Dead end by design.

OpenVINO + Stable Diffusion Forge — Intel's extension was built for the old Automatic1111 architecture. Forge restructured everything. Result:

ModuleNotFoundError: No module named 'ldm'
ModuleNotFoundError: No module named 'sgm'
Error build_unet: Invalid backend: 'openvino'
CPU-only + HDD — Our baseline before any optimization: 85-second startup, ~19 minutes per 512×512 image. The mechanical HDD competing with memory paging made it completely unusable.

The pattern: every "AMD-compatible" option either targets newer hardware, is abandoned, or is simply incompatible with modern pipelines. Vulkan is the only path that actually works.

The Architecture: Dual-Path Stack
The core insight of this project is that not every workload fits in 8GB of VRAM. The solution is intelligent routing between GPU and CPU:

OpenWebUI :3000 (Docker)
│
├──► llama-server :8081 ──► RX 580 Vulkan [llama.cpp]
│ └── Ollama :11434 ──► CPU fallback
│
└──► sd-server :7860 ──► RX 580 Vulkan [stable-diffusion.cpp]
├── SD 1.5 GGUF ──► 72s / image
└── FLUX hybrid ──► ~14 min / image

└──► ComfyUI       :8188  ──►  Xeon CPU WSL2

Path 1 — GPU Vulkan: LLM inference + SD 1.5 image generation. Fast, responsive, daily driver.

Path 2 — CPU Xeon: FLUX.1 16GB models, AnimateDiff video pipelines. The 32GB ECC RAM acts as "virtual VRAM" for models that don't fit on the card.

Building llama.cpp with Vulkan
Run in Developer PowerShell for Visual Studio:

powershell
cd E:\
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j20
Validate GPU detection:

powershell
cd build\bin\Release
.\llama-cli.exe --list-devices

Expected: Vulkan0: AMD Radeon RX 580 2048SP ✅

Start the LLM server:

powershell
.\llama-server.exe -m "E:\models\Mistral-7B-Q4_K_M.gguf" `
--host 0.0.0.0 --port 8081 --device Vulkan0
How to verify it's actually using the GPU:

ggml_vulkan: Found 1 Vulkan device(s)
ggml_vulkan: 0 = AMD Radeon RX 580 2048SP | VRAM: 8192MB
17.77 t/s ← RX 580 Vulkan ✅
If you see 3–5 t/s with no ggml_vulkan line — it's running on CPU. The --device Vulkan0 flag is mandatory.

Building stable-diffusion.cpp with Vulkan
powershell
git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp
mkdir build && cd build
cmake .. -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release -j20
Start the image server:

powershell
E:
cd "E:\stable-diffusion.cpp\build\bin\Release"
.\sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 `
-m "E:\models\dreamshaper8.gguf"
FLUX.1 Schnell: Running a 16GB Model on 8GB VRAM
FLUX.1 Schnell is a 12B parameter SOTA model that nominally requires 16GB. Here's how we run it on 8GB:

The strategy is memory segmentation — put the diffusion model on VRAM, offload everything else to RAM:

Component File Where
Diffusion Model flux1-schnell-q4_k.gguf GPU VRAM (~6.5GB)
VAE ae.safetensors CPU RAM (~160MB)
CLIP L clip_l.safetensors GPU VRAM (~235MB)
T5XXL t5xxl_fp16.safetensors CPU RAM (~9.3GB)
batch
sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 ^
--diffusion-model "E:\models\flux1-schnell-q4_k.gguf" ^
--vae "E:\models\ae.safetensors" ^
--clip_l "E:\models\clip_l.safetensors" ^
--t5xxl "E:\models\t5xxl_fp16.safetensors" ^
--cfg-scale 1.0 --steps 4 --clip-on-cpu --vae-on-cpu --vae-tiling
⚠️ --vae-tiling is not optional. Without it, VAE decode causes OOM and crashes the server.

Timing per 1024×1024 image:

Stage Time
T5XXL conditioning 11.49s
Sampling (4 steps) ~838s
VAE decode (9 tiles) 40.45s
Total ~14 min
Critical: Two GGUF formats for FLUX

This trips up almost everyone. There are two different GGUF distributions for FLUX:

Source Compatible with
city96 (HuggingFace) ComfyUI + ComfyUI-GGUF node only
leejet (HuggingFace) stable-diffusion.cpp ✅
Using a city96 GGUF in sd-server returns:

[ERROR] main.cpp:92 - new_sd_ctx_t failed
Always download from: huggingface.co/leejet/FLUX.1-schnell-gguf

whisper.cpp: Audio Transcription on the RX 580
This is where the numbers get absurd.

Build whisper.cpp with Vulkan:

powershell
git clone https://github.com/ggml-org/whisper.cpp
cd whisper.cpp
cmake -B build -DGGML_VULKAN=ON -DGGML_HIPBLAS=OFF -DGGML_HIP=OFF -DGGML_CUDA=OFF
cmake --build build --config Release -j4
Transcribe a video (MP4 → TXT):

powershell

Extract audio first (Whisper requires WAV on Windows)

ffmpeg -i "video.mp4" -ar 16000 -ac 1 -c:a pcm_s16le "audio.wav"

Transcribe

.\build\bin\Release\whisper-cli.exe -m models\ggml-large-v3-turbo.bin
-f "audio.wav" -l pt --output-txt
Performance on a 15-minute video (Windows):

Stage Time
Model load 4s
Mel spectrogram 1.2s
GPU encode 73s
Decode + batch 168s
Total 307s
VRAM used: only 2.6GB of 8GB. CPU stays at ~5%.

On Linux (Ubuntu 26.04, Mesa RADV), same hardware, same model:

Metric Windows Linux
Time (106s audio) 307s 23.58s
VRAM used 2.6GB 1.6GB
A 13× speedup on the same GPU. Mesa RADV's Vulkan compute path is dramatically more efficient for this workload than the Windows AMD driver.

Windows vs Linux: Full Benchmark Comparison
Workload Windows 10 Ubuntu 26.04 (Mesa RADV) Winner
LLM Qwen3 4B @ 99 layers ~15–17 tok/s ~35 tok/s 🏆 Linux (2×)
LLM Qwen3.6 35B @ max layers 7.62 tok/s (max 10 ngl) 5.18 tok/s (max 20 ngl) ⚖️ Tie
SD 1.5 DreamShaper (50 steps) ~72s ~85s 🏆 Windows
FLUX Schnell (4 steps, 512×512) ~84s ~52s 🏆 Linux
Whisper large-v3-turbo 307s · 2.6GB 23.58s · 1.6GB 🏆 Linux
Why Linux is faster for LLM: Mesa RADV allows up to 20 GPU layers for large models where Windows AMD drivers cap at 10. RADV's memory management is simply more aggressive and efficient.

Why Windows wins SD 1.5: The proprietary AMD driver has more stable direct rendering for this specific workload. Consistent 1.44s/it vs 1.65s/it on Linux.

Voice Cloning: Applio RVC on AMD Windows
We also built a full voice cloning pipeline:

Text → Balabolka (TTS) → WAV → Applio RVC → Cloned Voice
The key insight: instead of using a generative TTS model (which sounds robotic), we use a real voice actor (Antônio Neural, a Microsoft Neural voice) for prosody and emotion, then apply RVC to convert the identity to our target voice (Yuri). Result: 80–95% naturalness vs 60–70% for pure TTS.

AMD-specific critical findings:

DirectML is effectively dead for RVC — torch-directml is locked to torch==2.4.1 while Applio requires torch==2.7.1. Irreconcilable conflict.

Use CPU mode. On Xeon E5-2690 v3 (24 threads): ~6 min/epoch, ~20 hours for 200 epochs. Inference after training: 2 hours of audio → ~30 minutes processing.

The silent failure trap:

powershell

NEVER set these — they silently break feature extraction

set CUDA_VISIBLE_DEVICES=-1

set ROCM_VISIBLE_DEVICES=-1

Training will print "Model trained successfully" but produce nothing

Always verify logs/project/extracted/ contains .npy files before starting training.

The Community Timeline
This project didn't happen in isolation. Three independent researchers, same GPU, same conclusion:

Date Author Contribution
Jan 2025 艾米心 Amihart First LLM via Vulkan on RX 580 — 24.56 tok/s on Debian
Dec 2025 DH / DadHacks First SD via Vulkan — stable-diffusion.cpp breakthrough
2026 AIVisionsLab Full Windows + Linux production stack, voice cloning, transcription
The shared foundation: ggml by Georgi Gerganov. Vulkan compute backends in pure C++ that bypass the entire proprietary driver ecosystem.

Real Benchmarks Summary
Workload Model Backend Result
LLM inference Mistral 7B Q4_K_M RX 580 Vulkan (Win) 17–18 tok/s
LLM inference Qwen3 4B Q4_K_M RX 580 Vulkan (Linux) ~35 tok/s
LLM baseline Mistral 7B Q4_K_M Xeon CPU pure 3–5 tok/s
Image gen DreamShaper 8 SD1.5 RX 580 Vulkan ~72s / 512×512
Image gen flux1-schnell-q4_k GPU+CPU hybrid ~14 min @ 1024×1024
Audio transcription Whisper large-v3-turbo RX 580 Vulkan (Linux) 23.58s / 106s audio
Video frames AnimateDiff Xeon WSL2 CPU ~141s/frame
Voice inference Applio RVC Xeon CPU ~30 min / 2h audio
Troubleshooting: The Most Common Failures
generate_image returned no results / frozen terminal Bug in sd-server with Seed: -1. Fix: set a fixed integer seed (42, 1337) in OpenWebUI.

new_sd_ctx_t failed with FLUX You're using a city96 GGUF. Download from leejet instead.

Docker can't reach sd-server Windows Defender blocks the Docker subnet (172.x.x.x). Run as Administrator:

powershell
New-NetFirewallRule -DisplayName "sd-server AIVisionsLab" `
-Direction Inbound -Protocol TCP -LocalPort 7860 -Action Allow
--override-tensor exps=CPU slows down Vulkan This flag is optimized for CUDA/PCIe on Nvidia. Under Vulkan, the CPU↔GPU transfer overhead destroys any gains. Don't apply CUDA-optimized flags to Vulkan backends.

Full Documentation
This post covers the core architecture. Full guides for each component:

📖 Master documentation (PT/EN): setup-ia-local-rx580-vulkan.web.app
💻 GitHub repository: github.com/aivisionslab-studios/rx580-local-ai-guide
🎥 YouTube: @aivisionslab-hub
Conclusion
The narrative that legacy AMD GPUs can't run AI is a software problem, not a hardware limitation. The RX 580 has supported Vulkan since 2017. The compute capability was always there.

What changed is that ggml and its ecosystem built Vulkan backends that bypass the entire proprietary driver stack. The result is a GPU from 2017 running SOTA models from 2026 — locally, privately, for free.

RX 580 (2017) + Xeon (2014) + Vulkan + ggml = SOTA AI in 2026
The problem was never the GPU.

AIVisionsLab — Documenting local AI on legacy hardware. São Paulo, Brazil 🇧🇷

Description
How we built a full local AI stack (LLMs, Stable Diffusion, FLUX, Whisper, Voice Cloning) on a 2017 GPU using Vulkan. No CUDA. No ROCm. No cloud. Real benchmarks, real failures, real solutions.

Title
Running Local AI on an AMD RX 580 in 2026 — The Complete Vulkan Guide

Published
true

Tags
ai, amd, vulkan, opensource

Cover_image
https://setup-ia-local-rx580-vulkan.web.app/og-image.png

Canonical_url
https://setup-ia-local-rx580-vulkan.web.app

Three researchers. One GPU. Two years. How the RX 580 became an AI platform.

AIVisionsLab — Sun, 24 May 2026 13:20:37 +0000

All images in this article were generated on the RX 580 8GB — the same GPU everyone said couldn't run AI.

This is collective knowledge

Three independent researchers. No coordination. Same GPU. Same conclusion.

January 2025 — 艾米心 Amihart

Platform: Debian Linux
Published: Medium

Amihart was the first to document LLM inference via Vulkan on the RX 580.

Compiled llama.cpp with -DGGML_VULKAN=on on Debian, connected a Celeron G6900 CPU setup, and measured:

CPU only: 5.45 tok/s
RX 580 via Vulkan: 24.56 tok/s

A 4.5× uplift on hardware that officially "doesn't support AI."

But then came this line — honest, and correct for the time:

"Sadly, even though Vulkan seems to do a pretty good job with the RX580, I am unaware of any way to get Vulkan to work with Stable Diffusion. If you want to use Stable Diffusion, you will need ROCm."

That sentence opened a question that the next researcher answered.

December 2025 — DH / DadHacks

Platform: Linux/Debian
Published: dadhacks.org

DadHacks refuted Amihart's limitation — not as a criticism, but as proof that the software evolved.

stable-diffusion.cpp had matured. With -DSD_VULKAN=ON (equivalent to -DGGML_VULKAN=ON in newer versions), image generation via Vulkan on the RX 580 worked.

Including FLUX.1 Schnell in Q4 quantization, with CPU offloading for components that exceeded VRAM.

The barrier Amihart correctly identified in January had fallen by December.

2026 — AIVisionsLab

Platform: Windows 10 Pro + WSL2
Published: setup-ia-local-rx580-vulkan.web.app

The third step was integration.

Both previous projects ran on Linux. Neither connected everything into a unified daily-use system on Windows. Neither documented the failures (DirectML, ROCm, OpenVINO). Neither built automation scripts. Neither integrated OpenWebUI.

AIVisionsLab filled those gaps:

Full Windows stack with .bat automation
OpenWebUI integration via Docker with firewall notes
Dual architecture: GPU Vulkan for fast models, Xeon CPU WSL2 for FLUX 16GB
Documented every failure with root cause analysis
Discovered the critical GGUF incompatibility: city96 vs leejet formats

The question each project answered

Project	Question	Answer
Amihart	Can LLMs run on Vulkan RX 580?	Yes. 24.56 tok/s
DadHacks	Can Stable Diffusion run on Vulkan RX 580?	Yes. sd.cpp works
AIVisionsLab	Can all this run integrated on Windows daily?	Yes. Full stack documented

The common denominator

All three converge on the same engine:

ggml (Georgi Gerganov)
  ├── llama.cpp    → LLMs via Vulkan
  └── stable-diffusion.cpp (leejet) → Images via Vulkan

ggml ported deep learning tensor operations to C and exposed Vulkan hooks. That single decision freed legacy AMD hardware from the CUDA/ROCm dependency trap.

Three philosophies, same conclusion

Amihart:

"Despite how ancient this card is, it is technically possible to use it for AI."

DadHacks:

"This setup provides an accessible pathway for leveraging existing hardware investments without requiring expensive upgrades or specialized software stacks like ROCm."

AIVisionsLab:

"Commercial planned obsolescence is a market choice, not an engineering barrier. Legacy hardware doesn't die — it's liberated by the right software."

Full documentation

📖 setup-ia-local-rx580-vulkan.web.app — complete guide in PT/EN/ES/FR/AR
📦 github.com/aivisionslab-studios/rx580-local-ai-guide
🤗 huggingface.co/aivisionslab/ai-local-rx580-stack

Running FLUX.1 Schnell on an RX 580 8GB — GPU/CPU hybrid architecture

AIVisionsLab — Sun, 24 May 2026 13:18:29 +0000

Image above: generated by FLUX.1 Schnell running on the hybrid architecture described in this post.

The problem

FLUX.1 Schnell is a 12B parameter model. Full precision needs more VRAM than the RX 580 has.

The solution: split the components between GPU and CPU RAM.

Memory map

Component	File	Where	Size
Diffusion model	flux1-schnell-q4_k.gguf	GPU VRAM	~6.5GB
VAE	ae.safetensors	CPU RAM	~160MB
CLIP L	clip_l.safetensors	GPU VRAM	~235MB
T5XXL	t5xxl_fp16.safetensors	CPU RAM	~9.3GB

Total VRAM used: ~6.7GB / 8GB available
Total RAM used: ~9.5GB

The T5XXL encoder dominates RAM usage. If you're tight on RAM, t5xxl_fp8.safetensors reduces it to ~5GB.

⚠️ Critical: use leejet GGUF, not city96

Two different GGUF formats exist for FLUX. They have similar names but are NOT interchangeable:

Source	For
city96 on HuggingFace	ComfyUI + ComfyUI-GGUF node
leejet on HuggingFace	stable-diffusion.cpp ✅

Using city96 GGUF with sd-server returns:

[ERROR] stable-diffusion.cpp:355 - get sd version from file failed
[ERROR] main.cpp:92 - new_sd_ctx_t failed

Download from: https://huggingface.co/leejet/FLUX.1-schnell-gguf

The command

sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 ^
  --diffusion-model "E:\models\flux1-schnell-q4_k.gguf" ^
  --vae "E:\models\ae.safetensors" ^
  --clip_l "E:\models\clip_l.safetensors" ^
  --t5xxl "E:\models\t5xxl_fp16.safetensors" ^
  --cfg-scale 1.0 --steps 4 ^
  --clip-on-cpu --vae-on-cpu --vae-tiling

Flag breakdown:

Flag	Why
`--clip-on-cpu`	Frees ~235MB VRAM
`--vae-on-cpu`	Frees ~160MB VRAM
`--vae-tiling`	Prevents OOM at high resolution
`--cfg-scale 1.0`	Required for FLUX — higher values distort
`--steps 4`	Schnell converges in 4 steps by design

Real benchmark

Stage	Time
T5XXL conditioning	11.49s
Sampling (4 steps @ 1024×1024)	~838s (~14 min)
VAE decode (9 tiles)	40.45s
Total	~14 min

Terminal status at generation:

Listening on http://0.0.0.0:7860
VRAM: 7.6/8.0 GB | RAM: ~9.5 GB | Temp: 66°C

Windows Firewall fix

If OpenWebUI can't reach the server even with --listen-ip 0.0.0.0:

# Run as Administrator
New-NetFirewallRule -DisplayName "sd-server AIVisionsLab" `
  -Direction Inbound -Protocol TCP -LocalPort 7860 -Action Allow

Docker runs in an isolated WSL2 network — 127.0.0.1 won't work. Use your machine's actual local IP.

Full documentation

📖 setup-ia-local-rx580-vulkan.web.app
📦 github.com/aivisionslab-studios/rx580-local-ai-guide

Everything that failed before Vulkan saved our RX 580 AI setup

AIVisionsLab — Sun, 24 May 2026 13:14:16 +0000

All images in this article were generated locally on the RX 580 8GB — after we fixed everything described below.

The graveyard

Before Vulkan worked, we tried everything. This is the technical autopsy.

1. DirectML — Microsoft's promise that crashed

The attempt: torch-directml with --directml flag in ComfyUI.

The GPU was detected as privateuseone0. Looked promising.

Then this appeared on every run:

WARNING: torch-directml barely works, is very slow,
has not been updated in over 1 year and might be
removed soon, please don't use it.

NotImplementedError: Cannot access storage of OpaqueTensorImpl

Root cause: DirectML wraps tensor data in opaque objects called OpaqueTensorImpl. When ComfyUI's modern attention backends try to read the raw memory contents, the Microsoft layer blocks access entirely.

The project hasn't been updated in over a year. It's effectively abandoned.

Manual fix attempt: Downgrade to the May 2024 dev build:

pip uninstall torch torch-directml torchaudio
pip install torch==2.3.1+cpu --index-url https://download.pytorch.org/whl/cpu
pip install torch-directml==0.2.1.dev240521 --no-deps

This stops the crash but the performance is so slow it's unusable.

2. ROCm — officially dead for GCN4

The attempt: AMD's official GPGPU framework.

The reality: AMD dropped official support for Polaris/GCN4 architecture in ROCm v5.x. Permanently. There is no workaround.

On Windows: no native ROCm support at all.
On WSL2 with compatibility layers: kernel panics under heavy inference load.

The only working ROCm path for the RX 580 is via Docker containers that emulate gfx803 — which is what Amihart documented in January 2025. It works for Stable Diffusion, but requires Docker overhead and doesn't support modern FLUX architecture.

3. OpenVINO + Stable Diffusion Forge

The attempt: Intel's sd-webui-openvino extension inside Forge.

ModuleNotFoundError: No module named 'ldm'
ModuleNotFoundError: No module named 'sgm'
Error build_unet: Invalid backend: 'openvino'

Root cause: The extension was designed for the old AUTOMATIC1111 architecture. Forge completely restructured the codebase and replaced the native ldm and sgm modules. The OpenVINO injection fails at the foundation level.

4. CPU + HDD — the baseline disaster

Before any GPU acceleration:

Boot time: 85 seconds
LLM response: 3–5 tok/s
Image generation: ~19 minutes per 512×512 image
FLUX 16GB model load: 25 minutes from HDD

The mechanical drive was as much of a bottleneck as the missing GPU acceleration.

What actually worked

After all of this: Vulkan.

The ggml engine in llama.cpp and stable-diffusion.cpp uses Vulkan as a native GPU backend. The RX 580 has supported Vulkan 1.x since 2017 drivers. No special installation. No compatibility layers. Just compile with -DGGML_VULKAN=ON.

Results after switching:

LLM: 15–16 tok/s (from 3–5)
Image: ~72s (from ~19 min)
FLUX load: 30 seconds (from 25 min, after NVMe migration)

The lesson

The hardware was never the problem. Every failure above was a software problem:

DirectML: abandoned by Microsoft
ROCm: architecture policy decision by AMD
OpenVINO: extension not maintained for modern frontends
HDD: wrong storage choice

The RX 580 was waiting for ggml + Vulkan.

Full documentation

📖 setup-ia-local-rx580-vulkan.web.app
📦 github.com/aivisionslab-studios/rx580-local-ai-guide

Rodei Flux Schnell + LLM numa GPU de R$300. Sem CUDA. Sem cloud. Sem ROCm.

AIVisionsLab — Sun, 24 May 2026 13:08:33 +0000

Todas as imagens deste artigo foram geradas localmente na RX 580 8GB descrita abaixo.

A narrativa era clara

Em 2026, todo guia diz a mesma coisa:

"Sua AMD RX 580 não roda IA. Compra uma GPU nova."

A AMD removeu suporte ROCm para Polaris/GCN4 na v5.x.
DirectML travava com erros de OpaqueTensorImpl.
OpenVINO falhava silenciosamente.

GPU de 8GB parada em 0% de uso enquanto o CPU respondia LLMs a 3 tokens por segundo.

A gente recusou comprar uma GPU nova.

A solução: Vulkan

O projeto ggml — engine base do llama.cpp e stable-diffusion.cpp — suporta Vulkan como backend de GPU. Vulkan é um padrão aberto que ainda suporta a RX 580 nativamente desde os drivers de 2017.

Sem CUDA. Sem ROCm. Sem DirectML. Só Vulkan.

Resultados reais (logs do terminal, não benchmarks sintéticos)

Workload	Modelo	Velocidade
LLM	Mistral 7B Q4	15–16 tok/s
Geração de imagem	DreamShaper 8 GGUF	~72s/imagem
FLUX.1 Schnell	flux1-schnell-q4_k híbrido	~14 min @ 1024×1024

CPU sem GPU: 3–5 tok/s.
Ganho com Vulkan: 3–4× numa GPU que "não suporta IA".

Hardware

GPU:     AMD RX 580 2048SP — 8GB GDDR5 (Polaris / GCN4)
CPU:     Intel Xeon E5-2690 v3 — 12c/24t (2014)
RAM:     32GB DDR4 REG ECC
Storage: NVMe 1TB — 1.7–3.5 GB/s
OS:      Windows 10 Pro + WSL2 Ubuntu 22.04

O NVMe sozinho reduziu o carregamento do FLUX de 25 minutos para 30 segundos.
Storage é tão crítico quanto a GPU.

Compilar llama.cpp com Vulkan

# Executar no Developer PowerShell do VS
cd E:\
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j20

Validação:

cd build\bin\Release
.\llama-cli.exe --list-devices
# Esperado: Vulkan0: AMD Radeon RX 580 2048SP ✅

Compilar stable-diffusion.cpp com Vulkan

git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp && mkdir build && cd build
cmake .. -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release -j20

Subir o servidor

E:
cd "E:\stable-diffusion.cpp\build\bin\Release"
sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 ^
  -m "E:\models\dreamshaper_8.safetensors"

No OpenWebUI → Admin → Imagens → Automatic1111 → http://SEU_IP_LOCAL:7860/

⚠️ Crítico: dois tipos de GGUF incompatíveis

Se você tentar rodar FLUX e receber new_sd_ctx_t failed — você baixou o GGUF errado.

Fonte	Compatível com
city96 (HuggingFace)	ComfyUI apenas
leejet (HuggingFace)	stable-diffusion.cpp ✅

Sempre use: https://huggingface.co/leejet/FLUX.1-schnell-gguf

O que não funcionou (documentado com causa raiz)

Tentativa	Erro	Motivo
DirectML	`OpaqueTensorImpl`	Tensores MS incompatíveis com ComfyUI
ROCm	Kernel panics	GCN4 removido no v5.x — permanente
OpenVINO	`No module 'ldm'`	Extensão para arquitetura antiga A1111
CPU + HDD	19 min/imagem	Zero GPU + gargalo de I/O mecânico

Documentação completa

📖 Guia master (PT/EN/ES/FR/AR) com diagramas, benchmarks, scripts de automação:
👉 setup-ia-local-rx580-vulkan.web.app

📦 GitHub (scripts + docs):
👉 github.com/aivisionslab-studios/rx580-local-ai-guide

O problema nunca foi a placa.

I ran Flux Schnell + LLMs on a $50 GPU. No CUDA. No cloud. No ROCm.

AIVisionsLab — Sun, 24 May 2026 13:04:51 +0000

All images in this article were generated locally on the RX 580 8GB described below.

The narrative was clear

In 2026, every guide says the same thing:

"Your AMD RX 580 can't run AI. Buy a new GPU."

AMD dropped ROCm support for Polaris/GCN4 in v5.x.
DirectML crashed with OpaqueTensorImpl errors.
OpenVINO failed silently.

So we had a 8GB GPU sitting at 0% utilization while the CPU burned through LLM responses at 3 tokens/second.

We refused to buy a new GPU.

The fix: Vulkan

The ggml project — the engine behind llama.cpp and stable-diffusion.cpp — supports Vulkan as a GPU backend. Vulkan is an open standard that still supports the RX 580 natively since its 2017 drivers.

No CUDA. No ROCm. No DirectML. Just Vulkan.

Results (real terminal logs, not benchmarks)

Workload	Model	Speed
LLM inference	Mistral 7B Q4	15–16 tok/s
Image generation	DreamShaper 8 GGUF	~72s/image
FLUX.1 Schnell	flux1-schnell-q4_k (hybrid)	~14 min @ 1024×1024

CPU baseline without GPU: 3–5 tok/s.
Vulkan uplift: 3–4× on a GPU that "doesn't support AI."

Hardware

GPU:     AMD RX 580 2048SP — 8GB GDDR5 (Polaris / GCN4)
CPU:     Intel Xeon E5-2690 v3 — 12c/24t (2014)
RAM:     32GB DDR4 REG ECC
Storage: NVMe 1TB — 1.7–3.5 GB/s
OS:      Windows 10 Pro + WSL2 Ubuntu 22.04

The NVMe alone reduced FLUX model load time from 25 minutes to 30 seconds.
Storage is as critical as the GPU.

Build llama.cpp with Vulkan

# Run in Developer PowerShell for VS
cd E:\
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j20

Validate:

cd build\bin\Release
.\llama-cli.exe --list-devices
# Expected: Vulkan0: AMD Radeon RX 580 2048SP ✅

Build stable-diffusion.cpp with Vulkan

git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp && mkdir build && cd build
cmake .. -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release -j20

Run the server

E:
cd "E:\stable-diffusion.cpp\build\bin\Release"
sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 ^
  -m "E:\models\dreamshaper_8.safetensors"

Connect OpenWebUI → Admin → Images → Automatic1111 → http://YOUR_LOCAL_IP:7860/

⚠️ Critical: two types of GGUF

If you try to run FLUX and get new_sd_ctx_t failed — you downloaded the wrong GGUF.

Source	Compatible with
city96 (HuggingFace)	ComfyUI only
leejet (HuggingFace)	stable-diffusion.cpp ✅

Always use: https://huggingface.co/leejet/FLUX.1-schnell-gguf

What failed (documented)

Attempt	Error	Why
DirectML	`OpaqueTensorImpl`	MS tensors can't talk to ComfyUI backends
ROCm	Kernel panics	GCN4 dropped in v5.x — permanent
OpenVINO	`No module 'ldm'`	Extension targets old A1111 arch
CPU + HDD	19 min/image	No GPU + mechanical I/O bottleneck

Full documentation

📖 Complete guide (PT/EN/ES/FR/AR) with architecture diagrams, benchmarks, automation scripts:
👉 setup-ia-local-rx580-vulkan.web.app

📦 GitHub (scripts + docs):
👉 github.com/aivisionslab-studios/rx580-local-ai-guide

The problem was never the GPU.

Запуск Flux Schnell (12B) + LLM на устаревшей AMD RX 580 (8 ГБ) через Vulkan — Полное архитектурное руководство [2026]

AIVisionsLab — Fri, 22 May 2026 18:24:02 +0000

Многие считали, что RX 580 «мертва» для ИИ в 2026 году. Экосистемы, завязанные только на CUDA, прекращение поддержки Polaris в ROCm начиная с версии 5.x, и DirectML, который так и не был доведен до ума. Это подробный технический отчет о том, как мы доказали обратное.

Аппаратное обеспечение

GPU: AMD RX 580 2048SP — 8 ГБ GDDR5 VRAM (нативная поддержка Vulkan 1.x)
CPU: Intel Xeon E5-2690 v3 — 12 ядер/24 потока @ 3.5 ГГц boost
RAM: 32 ГБ DDR4 REG ECC Quad Channel
Накопитель: NVMe 1 ТБ (критически важно для устранения «узких мест»)
ОС: Windows 10 Pro + WSL2 Ubuntu 22.04.5

Почему другие решения не работают?

Решение	Статус	Причина
CUDA	❌	Только для Nvidia
ROCm	❌	Поддержка Polaris прекращена в v5.x
DirectML	❌	Ошибка `OpaqueTensorImpl` в CLIPTextEncode
OpenVINO	❌	Отсутствие модулей `ldm/sgm` в Forge

Фатальная ошибка DirectML:

NotImplementedError: Cannot access storage of OpaqueTensorImpl

Драйвер упаковывает память в непрозрачные тензоры (opaque tensors), которые бэкенды внимания ComfyUI не могут считать. Это тупик.

Решение — Двухуровневая архитектура

ПУТЬ 1 — GPU Vulkan (ускорение RX 580)

Нативная сборка stable-diffusion.cpp, скомпилированная с -DGGML_VULKAN=ON. Движок ggml работает напрямую с GPU без необходимости в ROCm или CUDA. Модели SD 1.5 GGUF генерируют изображение примерно за 72 секунды.

ПУТЬ 2 — CPU Xeon (тяжелые SOTA модели)

FLUX.1 Schnell (16 ГБ) превышает объем физической VRAM. ComfyUI работает через CPU внутри WSL2, используя ECC RAM в качестве стабильной виртуальной VRAM. Генерация 768x768 занимает ~24 минуты.

Гибридная сегментация памяти (Flux 12B Q4_K)

Компонент	Файл	Выделение памяти
Diffusion Model	flux1-schnell-q4_k.gguf	GPU VRAM ~6.5 ГБ
VAE	ae.safetensors	CPU RAM ~160 МБ
CLIP L	clip_l.safetensors	GPU VRAM ~235 МБ
T5XXL	t5xxl_fp16.safetensors	CPU RAM ~9.3 ГБ

Команда для запуска

sd-server.exe --listen-ip 0.0.0.0 --listen-port 7860 \
  --diffusion-model "E:\models\flux1-schnell-q4_k.gguf" \
  --vae "E:\models\ae.safetensors" \
  --clip_l "E:\models\clip_l.safetensors" \
  --t5xxl "E:\models\t5xxl_fp16.safetensors" \
  --cfg-scale 1.0 --steps 4 --clip-on-cpu --vae-on-cpu --vae-tiling

--vae-on-cpu и --vae-tiling обязательны. Без них ошибка DeviceMemoryAllocation возникает мгновенно.

Бенчмарки

Задача	Бэкенд	Результат
LLM инференс	Только CPU	3–5 токенов/с ❌
LLM инференс	RX 580 Vulkan	15–16 токенов/с ✅
SD 1.5 20 шагов	DirectML	~450с + сбой ❌
SD 1.5 20 шагов	Vulkan натив	~72с ✅
Flux 1024x1024	Xeon CPU WSL2	~24 мин ✅

Примечание: Время загрузки моделей сократилось с 25 мин (HDD) до 4 мин (NVMe).

Карта сервисов

OpenWebUI Docker :3000
  ├── llama-server.exe :8081  (Vulkan — RX 580)
  ├── sd-server.exe    :7860  (Vulkan — RX 580)
  └── ComfyUI          :8188  (CPU — Xeon WSL2)

Ресурсы

Полная документация, .bat скрипты оркестрации и скомпилированные бинарные файлы:
👉 https://setup-ia-local-rx580-vulkan.firebaseapp.com/

Железо не умирает. Оно просто получает вторую жизнь благодаря правильному ПО. Используете старые карты AMD для ИИ? Давайте обсудим оптимизацию буферов и задержки в комментариях.

Совет: Для тегов на Dev.to используйте: russia, ai, hardware, amd, vulkan.