DEV Community

Vigoss Luke
Vigoss Luke

Posted on

DiffusionGemma in July 2026: What Works, What Doesn't, and When Ollama Gets It

#llamacpp #llm #ai #diffusiongemma

You've probably seen the benchmarks — 4x faster, 6x more mistakes, 273-vote threads on r/LocalLLaMA. DiffusionGemma is the most interesting image generation model to land this year. But actually running it outside of a HuggingFace notebook is a different story.

Here's the honest state of every platform as of July 2026.

The One PR Blocking Everything

Ollama, LM Studio, and native GGUF support all depend on a single pull request: llama.cpp #24423. It's been open since June 10, initially a proof-of-concept from danielhanchen, now mergeable (conflict resolved). The hold-up is design direction — one maintainer wants a general diffusion server rather than a model-specific implementation. No ETA.

What Works Today

vLLM (Recommended)

Day-zero support from the vLLM team. Single GPU works out of the box. Multi-GPU was broken until PR #46177 landed June 26 — tensor parallelism is now in main.

pip install vllm
vllm serve RedHatAI/diffusiongemma-26B-A4B --tensor-parallel-size 2
Enter fullscreen mode Exit fullscreen mode

llama.cpp (Build from PR Branch)

Not in mainline yet. You need to fetch the PR branch and build with CUDA.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
git fetch origin pull/24423/head:diffusion-gemma
git checkout diffusion-gemma
cmake -B build -DGGML_CUDA=ON
cmake --build build -j
Enter fullscreen mode Exit fullscreen mode

Q4_K_M GGUF available on HuggingFace. Works, but you're on your own for build issues — no guarantees until it lands in main.

Ollama (Bridge Workaround)

No native ollama pull diffusiongemma yet. The bridge approach: build llama.cpp from the PR branch, create a custom Modelfile pointing at the GGUF, and serve through Ollama's import mechanism. Clunky but functional. Full steps at diffrun.dev/ollama.

What Doesn't Work

  • LM Studio — blocked on #24423. Can't bundle the engine until it's in llama.cpp mainline
  • GGUF in vLLM — PR #46454 fixes GGUF tied embedding loading for DiffusionGemma, mergeable but not merged. Use BF16 weights for now
  • nvfp4 quantization — vLLM's nvfp4 MoE kernels reject DiffusionGemma's expert intermediate size (not a multiple of 128). Separate issue, tracked in the vLLM repo

The Signal That Changes Everything

When #24423 merges, the dominoes fall fast: Ollama ships an engine update within days, LM Studio follows, and native GGUF conversion lands in mainline llama.cpp. Searches for "diffusiongemma ollama" and "diffusiongemma lm studio" will spike overnight.

I track all three PRs daily at diffrun.dev/status, with step-by-step setup guides for each platform at diffrun.dev.

Top comments (0)