TL;DR — Gemma 4 26B (A4B) is a 26-billion-parameter Mixture-of-Experts model that only activates about 4 billion parameters per token, which is why it's cheap and fast despite its size. Probes show it nailing interval-merging code and a two-pump math problem, with a minor slip on strict JSON formatting. At $0.07/M input and $0.34/M output tokens with a 262K context window, it's a strong case study for why MoE is the trick that makes 'run a big model on modest hardware' actually plausible.
Every model in this series so far has been dense: every parameter does work on every token, whether that token needs it or not. Gemma 4 26B breaks that assumption in its own name. The "A4B" isn't marketing — it means that out of 26 billion total parameters, only about 4 billion get activated for any given token. That single design choice is the whole reason this model is worth an episode, and it's the cleanest real-world example I've seen of what Mixture-of-Experts actually buys you.
The Trick: Mixture-of-Experts, Explained Through One Model
A dense 26B model runs all 26 billion parameters through every token, every layer. A Mixture-of-Experts model splits big chunks of the network into separate "experts," and a small routing network decides, per token, which handful of experts actually get used. Gemma 4 26B A4B routes each token to a subset of experts that together add up to roughly 4B active parameters, while the full 26B still exists on disk and in memory as the pool those experts are drawn from.
The practical effect: the model has the knowledge capacity of something in the 26B class, but the per-token compute bill of something closer to a 4B model. That's the entire trick. It's not a smaller model pretending to be big — it's a big model that's stingy about which parts of itself it wakes up for any given word.
Why This Matters for Modest Hardware — With a Catch
Here's the part that gets glossed over in most MoE explainers: activating fewer parameters saves compute, not memory. You still need to hold all 26B parameters somewhere accessible, because you don't know in advance which experts a given token will need. So the honest framing is: MoE trades FLOPs for RAM. That's a great trade if you're on a machine with generous memory but limited compute bandwidth — a Mac with a big unified-memory pool, or a CPU box with plenty of RAM but no serious GPU — because your throughput ends up tracking the ~4B active count, not the 26B total.
The other number worth sitting with is the context window: 262,144 tokens, per the model's metadata. Combined with light per-token compute, that's a combination that specifically favors workloads with long inputs and frequent, cheap calls — not workloads that need maximum depth from every single token.
What the Probes Actually Show
I ran three tasks against it and read the transcripts rather than trusting a scoreboard. The reasoning probe — a two-pump tank-filling problem — was handled cleanly: net fill rate of 30 L/min for 20 minutes gives 600 liters, leaving 1,800 liters for pump A alone at 90 L/min, which is 20 more minutes. The arithmetic is right at every step and the final answer is correct, delivered in 255 completion tokens at 43.2 tokens/sec.
The code probe asked for an interval-merging function. The logic is genuinely correct: sort by start time, then walk the list merging overlaps by comparing against the last merged interval's end — that's the standard, right approach, and it would work on real input. The catch is the response got cut off mid-sentence before it stated the promised time-complexity line, at 207 completion tokens and a noticeably slower 21.1 tokens/sec than the reasoning probe. Same model, same session — the throughput difference is a reminder that MoE routing can behave differently depending on which experts a given prompt lights up, and code-heavy prompts here ran slower than math-heavy ones.
The structured-output probe is the most instructive miss. Asked to "return ONLY the JSON object," it wrapped the output in a markdown code fence anyway, and it returned "total": "$445.50" as a string with a dollar sign rather than a clean number. The JSON itself parses fine, and the vendor/date fields are extracted correctly, but if you're piping this into a strict schema validator or a downstream numeric field, that's exactly the kind of small deviation that bre
The numbers (measured, not quoted)
I ran three quick probes against Gemma 4 26B (A4B MoE) via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:
| Probe | Wall-clock | Output tokens | Tokens/sec (effective) | Result |
|---|---|---|---|---|
| Code | 9.8s | 207 | 21.1 | completed |
| Reasoning | 5.9s | 255 | 43.2 | completed |
| Structured output | 4.6s | 48 | 10.5 | completed |
Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.
Model card: context window 262,144 tokens · hosted pricing $0.07/M input · $0.34/M output · weights: google/gemma-4-26B-A4B-it on Hugging Face
Credits — where it's due
- Google DeepMind — for training Gemma 4 26B (A4B MoE) and releasing the weights openly: google/gemma-4-26B-A4B-it. Open releases like this are why a series like this can exist at all.
- OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
- The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.
Top comments (0)