Tencent Hy3: How a 295B Sparse MoE Model Runs on 21B Active Parameters

Tencent released Hy3 on July 6, 2026 — a 295-billion-parameter Mixture-of-Experts model under the Apache 2.0 license. The headline numbers are striking, but the more interesting story is in the architecture: despite 295B total parameters, only 21 billion are active on any given forward pass. That gap between total and active parameters is the core design decision worth understanding.

What "Mixture of Experts" Actually Means Here

In a standard dense transformer, every parameter participates in every token's computation. A 295B dense model would require roughly 590GB of GPU memory just for weights, and every inference call would use all of it.

Hy3 takes a different approach. Each of its 80 transformer layers contains 192 routed experts — small feed-forward networks — plus one always-active shared expert. A learned router examines each token and selects the top 8 experts to process it. The other 184 experts sit idle for that token. The result: 21B parameters do the actual computation, while the remaining 274B provide specialized capacity that gets called on selectively.

This is why the model's inference cost is closer to a 21B dense model than a 295B one. The tradeoff is that all 295B weights must remain resident in GPU memory so the router can dispatch to any expert at any time — you can't swap experts in and out without introducing latency. In practice, serving Hy3 requires 8× H200-class GPUs in BF16 (about 590GB VRAM), or 4× H200s with FP8 quantization (around 295GB).

The Multi-Token Prediction Layer

Beyond the MoE routing, Hy3 includes a 3.8B-parameter Multi-Token Prediction (MTP) layer. Standard autoregressive generation produces one token per forward pass. MTP predicts several tokens ahead simultaneously, enabling speculative decoding: the model proposes a batch of candidate tokens, then verifies them in parallel rather than sequentially.

According to Tencent's deployment data, the MTP layer reduces time-to-first-token by 54% and end-to-end latency by 47% in production workloads. For agentic applications where the model is called repeatedly in a loop — tool calls, code execution, multi-step planning — that latency reduction compounds significantly.

The MTP layer is compatible with vLLM and SGLang's EAGLE-style speculative decoding, so teams already using those serving stacks can enable it without custom infrastructure.

Reasoning Modes and Tool Calling

Hy3 exposes three inference modes via API: no_think (direct response, lowest latency), think_low (light chain-of-thought), and think_high (extended reasoning). Developers can tune this per request depending on whether they need fast responses or careful deliberation.

For tool calling, Tencent ships dedicated parsers for both vLLM and SGLang:

vllm serve tencent/Hy3 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 2 \
  --tool-call-parser hy_v3 \
  --reasoning-parser hy_v3 \
  --enable-auto-tool-choice

The model is also OpenAI API-compatible when served this way, which means existing agent frameworks — Cline, Kilo Code, custom MCP loops — can swap in Hy3 without rewriting orchestration code.

What Changed Between the April Preview and the July Release

Tencent ran a preview of Hy3 in late April 2026 and collected feedback from over 50 internal product teams before the general release. The post-training improvements are measurable:

• Hallucination rate dropped from 12.5% to 5.4%
• Commonsense errors fell from 25.4% to 12.7%
• Multi-turn intent drift decreased from 17.4% to 7.9%

These aren't benchmark numbers — they come from 270 expert human evaluators rating outputs across coding, document analysis, and frontend development tasks. The model scored 2.67/4 in blind evaluation against GLM-5.1's 2.51/4.

Benchmark Context

On public leaderboards, Hy3 scores 78% on SWE-bench Verified and 90.4% on GPQA Diamond. The SWE-bench number trails GLM-5.2's 84.2%, which has a higher active parameter budget (~40B active vs. Hy3's 21B). On GPQA Diamond — a graduate-level science reasoning benchmark — Hy3 is competitive with models several times its active size.

The more relevant metric for most teams is tool-calling reliability. Tencent reports less than 4% accuracy variance across different agent scaffoldings, which matters when you're running hundreds of tool calls in a single session and need consistent output formatting.

The Licensing Angle

Hy3 is released under Apache 2.0, which means unrestricted commercial use, no registration requirements, and no revenue caps. This contrasts with Meta's Llama license, which requires registration and caps commercial use based on monthly active users.

For teams that have been waiting for a permissive license on a capable open-weight model, Hy3 fills a gap. The 256K context window also makes it viable for repository-scale code analysis and long-document workflows that would otherwise require proprietary APIs.

The practical constraint is hardware. Consumer GPUs can't run this model — even an FP8-quantized version needs 4× H200s. For teams without that infrastructure, the model is available free on OpenRouter through July 21, 2026, and via the Nous Research portal during the same window.

Who This Is For

Hy3 is most useful for teams building agentic systems that need a capable, commercially permissive model with a large context window. If you're running agent frameworks that make repeated tool calls, the MTP-accelerated latency and reliable tool-call parsing are concrete advantages.

It's less suited for teams that need the absolute highest coding benchmark scores (GLM-5.2 leads there), or for anyone running on consumer hardware. The self-hosting economics work out to roughly $0.90–$1.62 per million output tokens on 4× H200 FP8 — significantly cheaper than proprietary APIs, but only if you have the infrastructure budget.

The model weights and an FP8-quantized version are both available on Hugging Face. The GitHub repository includes deployment recipes for vLLM and SGLang with MTP enabled.

The Broader Pattern

Hy3 is part of a broader trend in open-weight model releases: the interesting design decisions are increasingly in the efficiency layer rather than raw parameter counts. Sparse MoE routing, speculative decoding via MTP, and adjustable reasoning depth are all mechanisms for getting more useful work out of a given compute budget.

The 295B vs. 21B gap is a useful reminder that model size headlines don't tell you much about inference cost or practical capability. What matters is how many parameters are active per token, how efficiently the serving stack uses them, and whether the model's reliability holds up across the kinds of tasks you actually need to run.