AETHER: A Fully Open From-Scratch Model in the OLMo Tradition
Full openness, not just open weights
AETHER is a from-scratch foundation model released under a full-openness standard, in the spirit of Allen AI's OLMo. That means more than downloadable weights. The release covers weights, training data, training code, training logs, and checkpoints, all under Apache-2.0. The distinction matters: most models labeled open publish only final weights, which lets you run the model but not reproduce it or study how it got there. Full openness is what makes independent replication and auditing possible.
The 7-way Oheng attention line
AETHER uses an in-house 7-way Latin-square attention design, part of the same Oheng (five-phases) research line that appears elsewhere in the stack. The Latin-square construction is a way to arrange multiple attention behaviors so that combinations are balanced rather than ad hoc. This is a genuine from-scratch training effort, which separates AETHER from the merge-based Darwin family: here the weights are trained, not recombined.
The efficiency constraint
Honesty requires naming the main limitation up front: compute efficiency, measured as model FLOPs utilization (MFU), is a known constraint. Training from scratch on roughly 24 GPUs against frontier-scale pretraining budgets is not a fair fight, and AETHER does not pretend otherwise. The point of the project is not to top a capability leaderboard. It is to ship a fully reproducible model plus a documented method.
Why reproducible openness is the deliverable
The value proposition is method and reproducibility, not frontier scale. When data, code, logs, and checkpoints are all public, other researchers can retrain, ablate, and verify claims instead of taking them on faith. That is scientifically more useful than a slightly higher score from a closed pipeline. It also fits the constraint honestly: a small team cannot win a scale race, but it can contribute on transparency and on a novel, inspectable attention construction that others can test.
Who this is for
AETHER is aimed at researchers and builders who need a model they can fully open up: reproduce the training run, swap the data, probe the Latin-square attention, or use it as a clean baseline free of undisclosed pipeline tricks. For that audience, a fully documented Apache-2.0 stack is worth more than a marginally stronger closed checkpoint.
Honest scope
AETHER is best understood as a research release, not a frontier competitor. Its MFU and overall compute efficiency are acknowledged weak points, and its scale is bounded by a roughly 24-GPU footprint. The 7-way Latin-square Oheng attention is an in-house research design; its advantages over standard attention are a hypothesis to be tested with the very artifacts the release makes public, not a settled result. What is verifiable today is the openness itself: Apache-2.0 weights, data, code, logs, and checkpoints. Judge AETHER on reproducibility and method, and treat any capability comparison as scoped and preliminary.
More: https://vidraft.net
Top comments (0)