DEV Community

Cover image for Running Ollama on AMD Ryzen AI Max+ 395 with ROCm and Docker
Haruki Niwa
Haruki Niwa

Posted on Edited on AI-assisted

Running Ollama on AMD Ryzen AI Max+ 395 with ROCm and Docker

#ollama #rocm #docker #linux

Running large language models locally on an AMD integrated GPU takes more than installing Ollama. The operating system, ROCm runtime, container configuration, and GPU permissions all need to work together. A container can start successfully while inference still runs on the CPU—or fails when the model is loaded.

I set up Ollama on an EVO-X2 equipped with an AMD Ryzen AI Max+ 395 and Radeon 8060S, using Ubuntu, ROCm, Docker, and Open WebUI. This article documents that setup, how I checked GPU detection, and the compatibility problems I encountered later. It is based on my installation notes from late 2025, with later observations clearly distinguished from the initial procedure. It is not a new benchmark or a freshly validated installation on the latest software releases.

Hardware and software

My system has the following hardware:

Component Configuration
System EVO-X2
Processor AMD Ryzen AI Max+ 395
Integrated GPU AMD Radeon 8060S (RDNA 3.5, gfx1151)
System memory 128 GB unified system memory
Operating system Ubuntu 24.04 series
Inference server Ollama in Docker, using its ROCm image
Web interface Open WebUI in a separate container

The original notes used floating container tags and did not record a complete, reproducible set of package versions. In subsequent work, I used the ROCm 7.1.1 installer and, in a March 2026 snapshot, confirmed Ollama 0.15.6. These observations do not establish the exact versions used at every stage of the original setup.

AMD's ROCm 7.1.1 Ryzen compatibility matrix explicitly lists the Ryzen AI Max+ 395 and its gfx1151 GPU target. Supported Ubuntu releases and kernel versions vary by ROCm release. For a new deployment, check the current AMD ROCm documentation for the selected release rather than assuming that instructions for ROCm 7.1.1 also apply to newer versions.

1. Prepare Docker

Docker Engine and the Docker Compose plugin are required. On a machine where Docker is already installed, verify both components:

docker --version
docker compose version
Enter fullscreen mode Exit fullscreen mode

If they are missing, follow the official Docker Engine installation guide for Ubuntu. Installing a Compose package alone does not necessarily install and configure Docker Engine.

To run Docker without sudo, I added my account to the docker group:

sudo usermod -aG docker "$USER"
Enter fullscreen mode Exit fullscreen mode

Log out and sign in again before retrying Docker commands. Membership in the docker group grants root-level privileges; it should only be assigned to trusted users. See Docker's post-installation documentation.

2. Install ROCm and verify the GPU

Install a compatible ROCm release

For the earlier ROCm 7.1.1-based environment, the installer package for Ubuntu 24.04 was:

Kernel prerequisite: AMD's version-specific ROCm 7.1.1 Ryzen installation guide calls for the Ubuntu 24.04 6.14 OEM kernel. The commands below do not install or select that kernel. Check the host kernel with uname -r and follow the requirements for the ROCm release you are installing before proceeding. This historical requirement should not be assumed to apply unchanged to newer ROCm releases.

cd ~/Downloads
wget https://repo.radeon.com/amdgpu-install/7.1.1/ubuntu/noble/amdgpu-install_7.1.1.70101-1_all.deb
sudo apt install ./amdgpu-install_7.1.1.70101-1_all.deb
Enter fullscreen mode Exit fullscreen mode

The Ryzen installation procedure used the Ubuntu inbox kernel driver rather than installing a separate DKMS driver:

sudo amdgpu-install -y --usecase=rocm --no-dkms
sudo usermod -aG render,video "$USER"
sudo reboot
Enter fullscreen mode Exit fullscreen mode

Version-specific note: These commands document the earlier ROCm 7.1.1 installation path; they are not a recommendation to downgrade a currently working ROCm installation. AMD's ROCm 7.1.1 Ryzen installation guide specifies additional OS and kernel requirements. New installations should use the instructions for their chosen, supported release.

Check detection after rebooting

On the Ubuntu host, run:

uname -r
rocminfo | grep -E 'Name:|gfx1151'
rocm-smi
Enter fullscreen mode Exit fullscreen mode

If available, rocminfo should identify a GPU agent with the target gfx1151. In my recorded output, ROCm detected the Radeon 8060S / Ryzen AI Max+ 395 and reported the expected gfx1151 architecture. A GPU memory figure of approximately 96 GiB also appeared in one of my detection logs; on this APU, that figure describes GPU-accessible memory in a unified-memory system, not a separate 96 GiB graphics card.

The availability and output format of rocm-smi depend on the ROCm package version. Some newer tooling also uses amd-smi. A reported GPU-addressable memory figure is not a guarantee that the same amount can be allocated to a single model; actual limits depend on the kernel, GPU memory management, and other allocations.

3. Run Ollama and Open WebUI with Docker Compose

I used separate containers for Ollama and its web interface, with persistent volumes for downloaded models and application data.

mkdir -p ~/ollama-rocm
cd ~/ollama-rocm
nano docker-compose.yml
Enter fullscreen mode Exit fullscreen mode

Here is a locally bound adaptation of the original Compose configuration. The original notes published ports on all host interfaces; binding them to 127.0.0.1 is a safer starting point when the browser runs on the same computer.

services:
  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    restart: unless-stopped
    environment:
      OLLAMA_HOST: 0.0.0.0:11434
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    volumes:
      - ollama:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
      WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY:?Define WEBUI_SECRET_KEY in .env}
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama

volumes:
  ollama:
  open-webui:
Enter fullscreen mode Exit fullscreen mode

The key settings are:

  • /dev/kfd and /dev/dri expose the AMD GPU interfaces to the Ollama container.
  • OLLAMA_HOST=0.0.0.0:11434 lets Ollama listen inside its container so Open WebUI can reach it over the Compose network. This is different from publishing the port on every host interface.
  • OLLAMA_BASE_URL=http://ollama:11434 uses the Compose service name for container-to-container communication.
  • Named volumes preserve models and Open WebUI data when containers are recreated.
  • WEBUI_SECRET_KEY provides a persistent session-signing key. In the example above, Compose reads it from the local .env file created below, rather than storing it in the publicly shared Compose YAML.

About HSA_OVERRIDE_GFX_VERSION

My original configuration included:

environment:
  - HSA_OVERRIDE_GFX_VERSION=11.5.1
  - OLLAMA_HOST=0.0.0.0:11434
Enter fullscreen mode Exit fullscreen mode

This was a compatibility override used in the earlier Strix Halo setup. It should not be treated as a universally required setting. Current Ollama AMD GPU documentation explicitly lists the Ryzen AI Max+ 395 and gfx1151. Start without the override on a supported stack; consider an override only when diagnosing a specific compatibility problem with a known software version.

Start the services

Before the first start, create a private .env file in the project directory to hold Open WebUI's persistent secret key. Do this once and keep the file when updating containers; regenerating the key invalidates existing login sessions. Do not commit .env to Git or publish its contents.

cd ~/ollama-rocm
if [ ! -e .env ]; then
  umask 077
  printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > .env
fi
Enter fullscreen mode Exit fullscreen mode

Then validate and start the services:

cd ~/ollama-rocm
docker compose config
docker compose up -d
docker compose ps
docker logs --tail 50 ollama
Enter fullscreen mode Exit fullscreen mode

docker compose config checks and renders the Compose configuration. A running container and an Ollama listening message indicate that the server started, not that GPU inference has been verified. depends_on sets the startup order but does not, by itself, guarantee that Ollama is ready to accept requests before Open WebUI starts.

Open http://localhost:3000 to access Open WebUI and configure its initial account.

Security and reproducibility: ollama/ollama:rocm and ghcr.io/open-webui/open-webui:main are moving image tags. For a reproducible deployment, pin tested image versions or digests and record them alongside the host kernel and ROCm versions. If you want to access Open WebUI from another machine, configure authentication and appropriate network restrictions before exposing its port. Avoid exposing the Ollama API directly to untrusted networks.

4. Download and run a model

The model names in my original notes were:

docker exec ollama ollama pull llama3.3:70b
docker exec ollama ollama pull qwen3-coder:30b
docker exec ollama ollama pull gemma3:27b-it-qat
Enter fullscreen mode Exit fullscreen mode

These are alternatives; there is no need to download all three merely to verify GPU access. For a single model, start an interactive session:

docker exec -it ollama ollama run llama3.3:70b
Enter fullscreen mode Exit fullscreen mode

The exact tag matters. llama3.3 currently resolves to a 70B model, but explicitly using llama3.3:70b keeps the commands consistent with the model you downloaded.

Other useful commands are:

docker exec ollama ollama list
docker exec ollama ollama ps
# Remove a model only when it is no longer needed:
# docker exec ollama ollama rm llama3.3:70b
Enter fullscreen mode Exit fullscreen mode

Large models can require substantial system memory and GPU-addressable memory, especially with longer context windows. Model size alone does not guarantee that inference will fit or remain stable.

5. Verify that inference is actually using the GPU

Successful model execution is only the first check. I used two complementary methods.

Check Ollama's processor allocation

While the model is loaded, run:

docker exec ollama ollama ps
Enter fullscreen mode Exit fullscreen mode

Inspect the PROCESSOR column. It indicates whether Ollama has placed the model on GPU, CPU, or a mixture of both. This is more informative than assuming GPU acceleration because a ROCm-enabled image was used.

Monitor GPU activity on the host

In another terminal:

watch -n 1 rocm-smi
Enter fullscreen mode Exit fullscreen mode

Run a prompt while monitoring utilization and memory reporting. A change in GPU activity can support the conclusion that the GPU is being used, but the readings may vary across tools and UMA devices. Use the Ollama allocation report together with the host metrics and server logs.

My recorded logs confirmed that Ollama discovered a ROCm device identified as gfx1151. Subsequent checks also included successful inference requests. These observations establish GPU detection and operation in those tested configurations; I did not record a controlled tokens-per-second benchmark for this article.

6. Troubleshooting: GPU detected, but model execution fails

A particularly instructive problem appeared during later maintenance. After an Ubuntu kernel update, ROCm still detected gfx1151, but Ollama could fail during inference, including runner errors and an out-of-memory error. In the recorded investigation, kernel 6.17.0-19-generic was associated with failure, while a previously working 6.17.0-14-generic configuration helped isolate the regression.

This is a case-specific observation, not evidence that every newer kernel is incompatible. AMD's newer RDNA 3.5 guidance documents kernel fixes for GPU queue creation and memory checks on Ryzen AI Max devices; its supported combinations differ from those in the historical ROCm 7.1.1 guide. Always consult the kernel requirements for the exact ROCm release and Ubuntu package versions you plan to use.

For troubleshooting, collect these diagnostics before changing drivers or kernel settings:

uname -r
rocminfo | grep -E 'Name:|gfx1151'
docker exec ollama ollama --version
docker exec ollama ollama ps
docker logs --tail 150 ollama
ls -l /dev/kfd /dev/dri/
Enter fullscreen mode Exit fullscreen mode

Check the results in this order:

  1. Device detection: Does rocminfo list the expected gfx1151 GPU?
  2. Container access: Are /dev/kfd and /dev/dri passed through, and are permissions sufficient?
  3. Runtime compatibility: Are the host kernel, ROCm-related driver components, and Ollama's bundled ROCm libraries compatible?
  4. Memory allocation: Does the failure occur only when a model or large context is loaded?
  5. Regression isolation: Did the problem begin after a specific package or kernel update?

An out-of-memory message is not equivalent to “GPU not detected.” Likewise, a healthy rocminfo result does not prove that a model runner can allocate memory successfully. Avoid applying overrides or downgrading kernels without first identifying the cause.

For more on device permissions and logs, see Ollama troubleshooting and AMD's RDNA 3.5 kernel notes.

7. Limitations and lessons learned

The practical workflow is straightforward once the stack is compatible: check the AMD GPU on the host, pass its device nodes into an Ollama ROCm container, run a model, and verify GPU allocation with ollama ps and host monitoring.

The difficult part is version compatibility. The CPU, integrated GPU, and memory capacity do not by themselves establish that an arbitrary Ubuntu kernel, ROCm release, or Ollama container will work together. My later kernel regression reinforced the importance of recording the exact versions of the components that were actually tested.

For a reproducible setup, I recommend recording uname -r, the ROCm version, image digests, model tags, and whether inference was confirmed on GPU. The commands above are based on my documented setup and official references, but the revised example has not been rerun end-to-end as a fresh installation for this article.

References

AI disclosure: This article documents my own technical setup and results. An LLM was used to help draft, organize, and edit the English text. I remain responsible for its technical accuracy.

Top comments (0)