The stock Qwen-Image-2.1-Turbo in BF16 needs about 20GB just to load and peaks near 27GB during text-to-image generation — a 10GB card like the RTX 3080 simply runs out of VRAM. The fix in one sentence: quantize the image Transformer and quantize the text encoder too, all while keeping image quality high and making it even faster!
Enter the star:tonera/Qwen-Image-2.1-Turbo-Nunchaku** — an SVDQuant 4-bit quantization of the official original!**
You can download this quantized model from Hugging Face or the ModelScope community. The companion text encoder is tonera/Qwen3-VL-8B-Instruct-Nunchaku. Tested on a 32GB machine: the native BF16 model OOMs — there simply is not enough VRAM to run it. After quantizing the Transformer (1024×1024 resolution, 8 steps), VRAM usage drops to 29.26GB with denoising at 7.64 it/s — images feel instant, done in about a second. With the text encoder also quantized, VRAM usage falls to 18.73GB at a denoising speed of 7.73 it/s. Even then, we could only run it on the RTX XX90 series — 24GB cards. With CPU offload enabled, VRAM usage is just 8.73GB, meaning as long as you have more than 9GB of VRAM, you can fully run this model. Some may say running the GGUF format works just as well. Yes, true — but this model's runtime is the Nunchaku community edition (tonera/vitoom-nunchaku). Measured with CPU offload on, its denoising speed still hits 6.11 it/s; for 8-step inference it still feels like ~1-second image generation.
Data below: measured on the same model card, an RTX 5090, quantized version vs. the BF16 original:
| Metric | Key result |
|---|---|
| VRAM after load | 19.83 → 10.63 GiB, saving 9.20 GiB (46.4%) |
| Text-to-image peak VRAM | 26.44 → 17.23 GiB, saving 9.22 GiB (34.8%) |
| Stable text-to-image inference | 3.080s → 1.269s, ~2.43×; editing ~2.02× |
| Image quality (25-image comparison) | Text-to-image SSIM 0.843, LPIPS 0.185 — usable but not pixel-perfect |
Note: the 2.43× is the relative speedup of the quantized version vs. BF16 on the same 5090; absolute times differ on other cards. Loading is only ~1.15–1.18× faster — most of the speedup comes from step-by-step inference.
An LPIPS of 0.185 after 4-bit quantization of a large image model is already an excellent score — it means fidelity is quite high. Sacrificing a bit of quality for speed and usability is perfectly acceptable.
Below is a comparison of quantizing only the Transformer vs. quantizing both the Transformer and the text encoder.
 |
 |
|---|
Both images perfectly follow the prompt constraint — "a cute girl is holding a sign reads: CreatePPTX.com" — and the version with the quantized text encoder even rendered a richer background: a potted plant.
We also have to mention the tonera/flux-3-image-klein-pup-nunchaku model here. It was originally released by Black Forest Labs for flux3-klein, designed to expand a one-line prompt into a richer, more detailed image prompt. But it works for Qwen-Image-2.1 as well — if you are not great at writing prompts, this gem is for you.
Quick Reference for Other GPUs
With the same "quantized image model + INT4 encoder" combo, all of the following cards can run it locally; 30/40 series use INT4, 50 series use FP4.
| VRAM tier | Models (precision tier) | Recommendation |
|---|---|---|
| 32GB | RTX 5090 (FP4) | Full 1024², no offload |
| 24GB | 3090, 3090 Ti, 4090, 4090D (INT4); 5090D v2 (FP4) | Runs with default settings |
| 16GB | 4080 SUPER, 4080, 4070 Ti SUPER, 4060 Ti 16GB (INT4); 5080, 5070 Ti, 5060 Ti 16GB (FP4) | Text-to-image works; offload on demand for editing |
| 12GB | 3080 Ti, 3060, 4070 Ti, 4070 SUPER, 4070 (INT4); 5070 (FP4) | Enable offload, start at lower resolution |
| 10GB | RTX 3080 (INT4) | Offload + lower resolution, good for prompt sketching |
Note: use FP4-quantized Transformer weights on Blackwell chips; use INT4 everywhere else. The text encoder always uses INT weights.
Boundary Notes
- Being able to run on 10–12GB cards depends on offload and lower resolution — it is not full-power, zero-wait image generation.
- Quantization is great for fast local iteration; for high-precision final deliverables, re-render the few selected images in BF16 or in the cloud.
- The weights are derived from Qwen/Qwen-Image-2.1-Turbo under the Qwen Research License; check the upstream license before commercial use.
In one sentence: running Qwen-Image-2.1 on a 10GB card is not about more VRAM — it is three things: model quantization + encoder quantization + offload scheduling.
Top comments (0)