Disclosure: this account is run by members of the VIDRAFT team, and the work discussed below is VIDRAFT's own result. It is not an independent review.
TL;DR
VIDRAFT reports pretraining a 325.6M parameter language model from random initialization on the Rebellions ATOM-Max, a commercial inference NPU released in 2025, and matching the same model trained on a B200 GPU: 0.7993 vs 0.7994 loss on 1,024 windows of unseen text. Weights for both runs and the evaluation script are public.
Why is this news?
Inference accelerators are sold to run models, not to train them. Their software stacks execute forward passes. A full pretraining run needs backpropagation, optimizer state and tens of thousands of stable updates. Showing that such a chip can carry a 5-billion-token run to GPU quality challenges a widely held assumption about what inference hardware is for.
What exactly was shown?
| Item | Value |
|---|---|
| Model | VKORE-325M-NPU-Base, 325.6M parameters |
| Data | 5.0B byte tokens of FineWeb-Edu |
| Run | 76,292 steps, about four days |
| Hardware | Rebellions ATOM-Max NPU (2025) |
| Reference | identical model on a B200 GPU |
What does it not show?
- Speed or cost. The report covers quality, not throughput or price per run.
- Scale. 325M parameters on 5B tokens is research scale.
- That the chip alone is enough. The NPU and its standard software are built for inference; training depends on VIDRAFT's own training stack and a stability technique under patent application.
- Seed variance. One run per hardware.
FAQ
Which NPU? Rebellions ATOM-Max, released in 2025.
How close are the two models? Paired difference -0.0001, standard error 0.0007.
Can I check it? Yes, both weights and evaluate_unseen.py are in the model repo.
Top comments (0)