How to Run Whisper AI on AMD GPUs Without VRAM Crashes

Run OpenAI Whisper locally on AMD GPUs using ROCm without VRAM crashes or OOM errors. Complete setup guide with Python batch scripts and tuning tips.
Author: John Moore

Running OpenAI Whisper locally on an AMD GPU is attractive for privacy, cost control, and avoiding cloud dependencies. The less attractive part is that most Whisper deployment advice assumes an NVIDIA CUDA environment. On AMD hardware, the path is fundamentally different. You need ROCm, a ROCm-aware PyTorch build, and a realistic understanding of how Whisper consumes GPU memory.

This guide addresses the specific problem of obtaining reliable local transcription on AMD GPUs without constant VRAM exhaustion, out-of-memory errors, illegal instruction crashes, or unstable inference. It is not a general introduction to speech recognition; it assumes you already know what Whisper does and want a working AMD-specific configuration.

The approach described here uses the OpenAI Whisper GitHub Repository pipeline with PyTorch ROCm. That combination is not always the fastest option, but it has the most predictable behavior on AMD GPUs when configured conservatively. Alternative Whisper implementations, such as faster-whisper, can be faster, but their ROCm support is less consistently packaged and often requires building CTranslate2 with manual ROCm flags. For a stable AMD deployment, the standard Whisper repository is significantly easier to debug.

Before we start, here is the full data flow we are aiming for:

Audio/video file
      |
FFmpeg converts to 16 kHz mono PCM WAV.
      |
Whisper loads the WAV file on the CPU and extracts an 80-channel log-Mel spectrogram.
      |
Mel features move to AMD GPU.
      |
Encoder runs on GPU
      |
Decoder generates tokens on GPU.
      |
Text output written to disk

1. What You Need Before Installing Whisper

AMD GPU support is not uniform. The single most important prerequisite is knowing your GPU architecture and whether your ROCm build actually supports it. If you are building a new rig, review the best AMD GPUs for AI and machine learning in 2026: Budget to Pro to avoid architecture bottlenecks.

Compatibility Checklist

• Operating system: Ubuntu 22.04 or 24.04 LTS are the most tested with ROCm. RHEL/Rocky/AlmaLinux can work, but often require manual DKMS steps.
• AMD GPU: Must be on the official ROCm supported hardware list, or you must be willing to use an override such as HSA_OVERRIDE_GFX_VERSION.
• ROCm version: Installed and visible via /opt/rocm/bin/rocminfo. The ROCm version must exactly match the PyTorch wheel version.
• Python: 3.10 or 3.11 are the safest choices.
• System RAM: 16 GB minimum for medium and larger models. 8 GB can work for tiny/base but will struggle with long files.
• Disk space: 20 GB free. ROCm libraries alone can use several GB, and model files are downloaded to ~/.cache/whisper.
• FFmpeg: Must be installed and on PATH. Whisper uses FFmpeg to decode almost all audio formats.

ROCm does not support all Radeon cards. A Radeon RX 580 may have plenty of VRAM, but its gfx803 architecture is no longer a practical target for current PyTorch ROCm wheels. A Radeon RX 7900 XTX on gfx1100 is much more likely to work with recent ROCm releases, but not universally guaranteed.

To identify your GPU architecture, run:

bash
/opt/rocm/bin/rocminfo | grep -E 'Name:|gfx'
If rocminfo does not exist, ROCm is not installed correctly. Reference the Official AMD ROCm Documentation to resolve base driver issues.

2. Installing PyTorch with ROCm
Do not install ordinary CPU-only PyTorch. A CPU-only build will not see your AMD GPU at all and will fail with confusing errors later. For deeper kernel-level tuning, bookmark PyTorch on AMD GPUs: The complete ROCm setup & tuning guide.

Step 1: Create a clean Python virtual environment
Using a fresh environment prevents contamination from previous PyTorch installs. That contamination is one of the most common causes of half-working ROCm setups.

Bash


python3 -m venv whisper-rocm
source whisper-rocm/bin/activate
pip install --upgrade pip setuptools wheel
Step 2: Install ROCm-enabled PyTorch
Always check the official PyTorch ROCm installation page before running anything. If you encounter broken wheel links, refer to Fix PyTorch CUDA not available on AMD GPUs: Complete ROCm setup guide.

Bash


pip install torch torchvision torchaudio --index-url [https://download.pytorch.org/whl/rocm6.2](https://download.pytorch.org/whl/rocm6.2)
Step 3: Verify that PyTorch sees the AMD GPU
This is the critical diagnostic step. Create a quick test script to prevent quote-escaping errors in the terminal:

Bash


cat << 'EOF' > check_gpu.py
import torch
print(torch.__version__)
print(torch.version.hip)
print(torch.cuda.is_available())
if torch.cuda.is_available():
    print(torch.cuda.get_device_name(0))
EOF
python check_gpu.py
On a working ROCm install, torch.version.hip is None, and torch.cuda.is_available() returns True. This is normal even on AMD, because PyTorch uses the CUDA API abstraction for all GPU backends. If it returns False, your ROCm runtime is broken or missing. Read Fix GPU not detected in PyTorch: The Ultimate 2026 Guide to isolate the failure.

3. FFmpeg and Real Whisper VRAM Usage
Whisper does not directly parse MP3, M4A, MP4, or FLAC files. It calls FFmpeg to decode audio into a raw waveform.

Why audio preparation matters for VRAM stability: Whisper internally resamples audio to 16 kHz mono before generating mel spectrograms. If you pass a 48 kHz stereo file, FFmpeg performs the conversion on the CPU, not the GPU. However, problematic audio files can cause FFmpeg to fail, and Whisper may then fall back to an error path that leaves PyTorch GPU tensors permanently allocated. Pre-converting audio using a known command helps avoid that.

Bash


ffmpeg -y -i input.m4a -ar 16000 -ac 1 -c:a pcm_s16le output.wav