I needed German product descriptions for a B2B workwear and corporate fashion shop (Shopware 6, 10,000+ articles). Requirements: correct German, no invented properties, and the client's brand rules. Everything runs locally on one ASUS GX10 (NVIDIA GB10, 128 GB memory shared by CPU and GPU, no discrete graphics card).
I ran the test in three steps. Every text was checked against the supplier data and scored out of 10 against a fixed rubric, with Claude (Anthropic) as the reviewer and a written reason per text. That's a reasoned judgement, not a measurement.
Full report with all tables, per-article scores and runtime settings: GX10 Product Copy Test
Step 1: ten models, plain instruction
Ten local models, 30 articles, same data, same prompt. The Gemma 4 family won (Gemma 4 26B: 288/300, with 12B and 31B right behind). Typical failures elsewhere: dropped safety standards on hi-vis jackets, prompt examples copied in as industries, an invented "business customer offer" in meta descriptions.
The three largest models (71 to 86 GB) ranked below the 18 GB Gemma 4 26B.
Step 2: the real brand prompt
The client, a designer, wrote her brand voice as a prompt: plain language, banned clichés, strict fact rules. Same 30 articles, top five models:
| Model | Plain prompt | Brand prompt |
|---|---|---|
| Gemma 4 31B | 287.5 | 238 |
| Gemma 4 26B | 288 | 229 |
| Qwen 3.8 27B | 281 | 182 |
| Gemma 4 12B | 287.5 | 181 |
| Mistral Small 3.2 24B | 282 | 127 |
Scoring was stricter because tone counted, so look at the spread: 7 points with the plain prompt, 111 with the real one. Benchmark with your production prompt.
Step 3: Gemma 4 31B in eight setups
First I fixed the pipeline (50 new articles from 50 product types, shop-ready HTML plus SEO fields). What helped most:
- Clean the data in code before it reaches the prompt.
- Use fixed rules for anything you can describe as a pattern.
- Let the LLM fact check rewrite only the flagged sentence. Whole-text rewrites made the copy flat.
Then the same pipeline with different builds and runtimes:
| Gemma 4 31B | s / article | Score /500 |
|---|---|---|
| 4-bit QAT, Ollama | 46.6 | 423 |
| BF16, Ollama | 110.7 | 413 |
| 4-bit NVFP4 (post-hoc), vLLM ×8 | 17.5 | 399 |
| 4-bit QAT w4a16, vLLM ×8 | 13.6 | 421 |
| same, thinking on | 119.1 | 419 |
- Full precision bought nothing. BF16 was slower and not better.
- 4-bit is not 4-bit. Quantization-aware training kept the quality, the post-hoc quant lost 22 points.
- Batching is the speed lever. vLLM with 8 concurrent requests instead of Ollama with 2: 10,000 articles in about 1.6 days instead of 5.4.
- Thinking mode: same quality, 9x the time.
Two brand-new models from October 2026 didn't come close: NVIDIA Nemotron 3.5 Bolt scored 305, Aleph Alpha Kolibri-1 290. Repeated runs of the same Gemma setup landed between 416 and 421, so a few points are noise. Gaps of 100 are not.
The serving call, for reference. Thinking off, JSON schema where structured output is needed:
body = {"model": "gemma4-31b-qat", "messages": messages, "temperature": 0.3, "max_tokens": 1200,
"chat_template_kwargs": {"enable_thinking": False},
"response_format": {"type": "json_schema", "json_schema": {"name": "answer", "schema": schema}}}
Takeaways
- Benchmark with your real prompt and your worst data.
- Pick the build, not just the model.
- Use a batching server for bulk jobs.
- Check mandatory facts with fixed rules.
- Keep a human spot check.
Everything else (step-by-step tables, per-article scores for all 11 runs, the vLLM settings for Gemma, Nemotron and Kolibri on a GB10, one article compared across models) is in the full report.
I'm a Shopware freelancer. If you've measured QAT against post-hoc quants on other tasks, I'd like to compare notes in the comments.
Top comments (0)