TL;DR: VIDRAFT pretrained a small language model from scratch on an inference-only NPU, running real backpropagation on a chip the industry considers inference-only. The result, VKORE-46M-NPU, is published on Hugging Face with weights, a model card, and comparison data. Trained under identical conditions on a GPU, the loss trajectories correlate at 0.99.
Why training on an inference-only NPU is hard
Inference-only NPUs are built for the forward path: static graphs, low precision, weights as constants. Training needs the opposite, a backward graph with saved activations, optimizer state, numerical stability for small gradients, and multi-chip gradient synchronization. Those paths usually do not exist in an inference stack. That is why "NPUs cannot do backprop, so they cannot train" became the default view. This work runs backprop directly on such a chip and trains a model from scratch, with no pretrained weights.
The evidence: same model, two kinds of hardware
The most convincing part is the methodology. The same model was trained from scratch under identical initialization, data, and schedule on a 32-chip NPU and on a GPU.
- The two loss trajectories correlate at 0.99.
- Throughput (tokens per second) converged to a comparable level.
A single falling-loss curve proves nothing on its own. If low-precision kernels mangle gradients, the curve drifts away from a GPU run. A 0.99 correlation with the GPU reference under identical conditions is strong evidence that backprop and optimizer updates are doing numerically the same work as the reference.
It worked on an older chip
This was done on a first-generation inference-only NPU released in 2025 (32 chips), not the newest silicon. If older hardware can do it, next-generation NPUs leave clear headroom.
One NPU now covers the whole loop
- Pretraining from scratch
- LoRA fine-tuning
- Inference acceleration (confirmed on general open models too)
- Quantized-model execution and speedup
VIDRAFT is consolidating this into an NPU software layer called VKORE. Just as Nvidia built dominance through the CUDA software ecosystem rather than the GPU alone, VKORE aims to be that foundation for NPUs, a common acceleration layer that lowers dependence on a specific GPU and its software ecosystem. The company describes it as "CUDA for NPUs."
Why it matters
Top GPU servers are expensive, back-ordered, and power-hungry. A cheaper, low-power NPU that can also train lets a company grow its own model on-site, installed like a desktop in the office, without waiting on GPUs and without sending sensitive data outside.
The released model is trained on a public dataset and generates short English stories. The core training technique is proprietary and patent-pending, with details to follow in a paper; the core training code is kept private for IP protection.
FAQ
Did it really run backprop on an inference-only NPU?
Yes. Backpropagation ran directly on a chip designed for inference only, pretraining from scratch.
What does 0.99 mean?
The NPU loss curve nearly overlaps a GPU reference trained under identical conditions, evidence that training worked numerically.
How large is the model?
46M parameters. The point is a working proof of the NPU training path, not model quality. Scaling to larger models and real data is the next step.
Where is the model?
https://huggingface.co/FINAL-Bench/VKORE-46M-NPU
Further reading
- VIDRAFT Engineering Notes: Running large models on-device without a GPU
- VIDRAFT Engineering Notes: Decision models that judge in one forward pass
Top comments (0)