VIDRAFT Releases Fully Open Foundation Model Aether-7B-5Attn: Weights, Code, Data, and Training Logs — All Public

#opensourcellm #mixtureofexperts #foundationmodel #koreanai

VIDRAFT Releases Fully Open Foundation Model Aether-7B-5Attn: Weights, Code, Data, and Training Logs — All Public

TL;DR: Korean AI startup VIDRAFT has released Aether-7B-5Attn, a ~6.59B-parameter Mixture-of-Experts foundation model on Hugging Face under the Apache-2.0 license. Unlike most "open-source" LLMs that only ship weights, this release includes training data recipes, full training code, configuration files, training logs, intermediate checkpoints, and evaluation code — making the entire pipeline reproducible. If you care about verifiable, sovereign AI or want a genuinely open Korean/English/math-capable base model, this is worth a look.

What it is

Aether-7B-5Attn is a foundation language model independently developed by VIDRAFT (비드래프트), a Korean Pre-AGI AI startup. Key facts from the announcement:

Parameter count: ~6.59 billion total
Architecture: Mixture-of-Experts (MoE) with a combination of five distinct attention mechanisms
Training tokens: approximately 144.2 billion tokens
Training data composition:
- Mathematics: 37.8%
- Korean: 21.6%
- English: 21.6%
- (remaining percentage across other domains)
License: Apache-2.0 — permissive for commercial and research use
Release artifacts: model weights, training data recipe, full training code, hyperparameters, training logs, intermediate checkpoints, and evaluation code

VIDRAFT explicitly positions this release in the same philosophical category as Allen AI's OLMo, Apertus, and LLM-jp — the small but growing family of fully open (not just open-weight) foundation models. The company describes this as a practical implementation of "Sovereign AI" (소버린 AI): the idea that true AI independence requires being able to reproduce, verify, and audit the entire model pipeline, not merely download someone else's weights.

How it works

At a conceptual level, Aether-7B-5Attn makes two architectural choices worth noting:

Mixture-of-Experts (MoE): Rather than activating all parameters for every token, an MoE architecture routes each input through a learned subset of "expert" sub-networks. This allows the model to have a larger total parameter count while keeping per-token compute costs lower than a dense model of equivalent size — a common efficiency trade-off in modern LLMs.

Five-attention hybrid: The model combines five different attention mechanisms within its architecture. The specifics of each mechanism are not detailed in the public announcement, but combining multiple attention variants (e.g., local, global, sliding-window, or other forms) is a recognized technique for balancing efficiency and long-range context handling.

Reproducibility-first release philosophy: The distinction VIDRAFT draws between an open-weight model and a fully open model is operationally significant for engineers. With open weights alone, you can run inference and fine-tune, but you cannot verify training, reproduce the model from scratch, or audit data provenance. By shipping the full training stack — code, configs, logs, intermediate checkpoints, and evaluation harness — VIDRAFT is enabling independent reproduction and scientific verification of results, which is a substantially higher bar than the industry norm.

Benchmarks & results

The press article does not report specific benchmark scores (e.g., MMLU, HumanEval, GSM8K, or Korean-language benchmarks). No numerical evaluation results are available from this source.

What is stated qualitatively:

The model is designed with a heavy mathematics data weighting (37.8%), suggesting intentional optimization for reasoning-heavy tasks.
The balanced Korean/English split (21.6% each) indicates the model is built as a genuinely bilingual base, not an English model with Korean post-training.
Evaluation code is included in the release, meaning third-party benchmark results from the community should emerge quickly.

Developers are encouraged to run their own evaluations using the bundled evaluation code once the Hugging Face release is accessible.

How to try it

The model is publicly released on Hugging Face under the Apache-2.0 license. To find it, search for Aether-7B-5Attn by VIDRAFT on the Hugging Face Hub. Once you locate the repository, you can download it with:

huggingface-cli download VIDRAFT/Aether-7B-5Attn

⚠️ Verify the exact repository namespace directly on Hugging Face — always confirm the publisher before downloading any model weights.

Training code, configuration files, and evaluation scripts are described as part of the release package. Check the model card and linked GitHub repository (if provided on the Hub page) for setup instructions and reproduction steps.

No OpenAI-compatible API endpoint or hosted inference service has been announced in this release.

FAQ

Q: How is this different from models like Llama or Mistral that are already "open source"?
A: Llama and most Mistral releases are open-weight — they publish the trained weights but not the training data pipeline, hyperparameters, or training code in full reproducible form. VIDRAFT's release also includes the training data recipe, full training code, logged hyperparameters, intermediate checkpoints, and evaluation code, making it possible to reproduce the model from scratch and independently verify its behavior. This is the same standard set by Allen AI's OLMo project.

Q: Is the Apache-2.0 license truly permissive for commercial fine-tuning and deployment?
A: Yes. Apache-2.0 is a permissive open-source license that allows commercial use, modification, and redistribution, provided attribution is maintained and the license notice is included. It imposes no restrictions on fine-tuning or deployment comparable to, for example, the Llama community license. Always review the model card for any additional terms VIDRAFT may have appended.

Originally reported by 중앙일보 (2026-07-20) — source article.