Yandex just open-sourced a language model it trained entirely from scratch — no borrowed weights, no initialization from Qwen or Llama. It's called AliceAI-Foundation-80B-A3B-Base, it's on Hugging Face under Apache 2.0, and it's a surprisingly interesting release if you care about MoE architecture or non-English models.
Here's what's inside, where it actually wins, and where the benchmark table deserves a second look.
The short version
| Parameters | 80B total, 3B active per token |
| Architecture | MoE: 512 experts, top-10 routed + 1 shared |
| Layers | 48 — hybrid Kimi Delta Attention + gated attention (3:1) |
| Context | 262,144 tokens |
| Training | ~18T tokens, from scratch |
| Languages | Russian, English |
| License | Apache 2.0 (commercial use OK) |
It's a base model — not instruction-tuned, not a chatbot. Yandex calls it experimental: a testbed for the architecture of its upcoming unified reasoning model, which will power agentic features in its Alice AI assistant.
Compared to Yandex's previous flagship (Alice AI LLM, 235B, October 2025), it's almost 3x smaller overall and 7x smaller in active parameters — and, per the technical report, beats it on facts, math, code and long context.
80 billion parameters, three at work
Every token is routed through 10 of 512 small experts plus one shared expert that's always on. You get the knowledge capacity of an 80B model with roughly the per-token compute of a 3B one.
The attention stack is the other notable bit. Three out of every four layers use Kimi Delta Attention — a linear attention variant that folds history into a fixed-size state instead of keeping a KV entry for every past token. Only every fourth layer is classic (gated) attention. In practice that means 262K context without the usual KV-cache blowup — exactly the bottleneck you hit with "read this whole contract" or "understand this repo" workloads.
Benchmarks: where it wins and where it doesn't
Yandex compared base versions against open models in the same weight class and above. A selection from the official model card, including rows where it loses:
| Benchmark | AliceAI 80B-A3B | Qwen3.5 35B-A3B | Nemotron-3 Super 120B-A12B | DeepSeek-V4 Flash 284B-A13B |
|---|---|---|---|---|
| WikiWebFacts (Yandex's own) | 86.5 | 62.4 | 72.8 | 83.2 |
| HardMultiQA (Yandex's own) | 67.9 | 47.2 | 54.5 | 65.4 |
| EduBench Russian | 74.2 | 42.9 | 44.0 | 67.7 |
| ExpertFactsQA Law | 49.6 | 27.9 | 24.3 | 40.5 |
| MATH-500 | 91.1 | 81.9 | 84.8 | 80.7 |
| LiveCodeBench v5-6 | 50.5 | 50.4 | 50.4 | 38.1 |
| BigCodeBench | 48.3 | 43.5 | 48.8 | 49.1 |
| TriviaQA | 79.0 | 71.4 | 89.8 | 89.4 |
| MMLU-Pro | 66.8 | 63.2 | 69.9 | 66.5 |
| LongMemEval 128k | 64.6 | 55.6 | 64.8 | 68.0 |
The pattern is clear. Anything that needs knowledge of the Russian-speaking world — language, law, school curriculum, local facts — it beats models 4x its active size. That's the training corpus talking. Math is strong too: HMMT Feb 2026 at 96.9 (pass@32) vs 87.9 for Qwen3.5.
On English trivia and general knowledge (TriviaQA, MMLU-Pro), the bigger Nemotron and DeepSeek lead. No magic: 80B parameters can't hold as much about the world as 284B.
Can you run it locally?
3B active sounds laptop-friendly, but all 80B have to sit in memory: ~160 GB in bf16, roughly 45 GB at 4-bit by my estimate. So in theory it's 64 GB Mac territory — and Apple Silicon is where MoE models shine, since you only read a small slice of weights per token.
In practice, not yet. The official recipes are transformers and vLLM on NVIDIA GPUs (the example uses four), and the KDA layers need the flash-linear-attention kernels. I couldn't find GGUF or MLX quants on release day, and non-standard attention is exactly what tends to delay llama.cpp support by weeks.
Caveats
- It's a base model. It continues text; it doesn't chat. Instruction tuning is on you (or on Yandex's own future releases).
- The two most impressive benchmarks are Yandex's own. To their credit, WikiWebFacts and HardMultiQA were published alongside the weights with evaluation protocols — anyone can re-run them. Until someone does, I'd weight third-party benchmarks more.
- It's not a frontier model. This is a compact model winning on data and architecture, not a GPT or Claude competitor.
Still, a fully in-house model with a permissive license, open benchmarks and a detailed tech report is a genuinely useful foundation for anyone building products in Russian — especially in legal, education and reference use cases where its lead is largest.
Originally published at klukyanov.ru.
Shorter weekly write-ups (in Russian) — on Telegram.
Top comments (0)