Training an LLM from Scratch on Inference-Only NPUs — No GPU Required

TL;DR: VIDRAFT has successfully pre-trained a small language model entirely on inference-only NPUs, bypassing GPUs at every stage of training. The model — VKORE-46M-NPU — is publicly available on Hugging Face, along with comparative analysis materials. For ML engineers exploring alternative accelerator stacks, this is a concrete proof-of-concept that backpropagation on inference-dedicated silicon is achievable.

What it is

VKORE-46M-NPU is a 46-million-parameter small language model (SLM) pre-trained from scratch using only a first-generation, inference-only NPU — a class of chip that conventional ML wisdom treats as unsuitable for training workloads. The model was trained on a public dataset and is capable of generating short English-language stories.

The broader context: VIDRAFT is building VKORE, an NPU-native acceleration software layer intended to do for NPUs what CUDA does for NVIDIA GPUs. The target capability set includes:

• Inference acceleration
• Full pre-training (demonstrated here)
• LoRA fine-tuning

This release is the public validation artifact for the pre-training milestone.

How it works

The core engineering challenge is that inference-only NPUs are architected to run forward passes efficiently — they are not designed to support the gradient computation and weight-update operations that backpropagation requires. Hardware-level features that GPU-based training takes for granted (flexible memory access patterns, high-precision intermediate buffers, in-place tensor mutation) are typically absent or restricted on inference NPUs.

VIDRAFT's approach, at a conceptual level:

• Custom software abstraction over the NPU: Rather than relying on the NPU vendor's training stack (which typically doesn't exist for inference-only silicon), VIDRAFT implemented backpropagation through their own VKORE software layer, making the NPU behave as a trainable compute substrate.
• From-scratch pre-training, no warm-start: The model was initialized with random weights — no pre-trained checkpoint was used as a starting point. This rules out the possibility that the result is simply transfer learning or fine-tuning dressed up as pre-training.
• Parallel GPU reference run: To validate correctness, VIDRAFT ran an identical training job on a high-performance GPU under matched conditions and compared the resulting loss trajectories.

The precise implementation details — operator fusion strategy, memory management, gradient accumulation mechanisms — are not publicly disclosed and will be described in a forthcoming paper.

Benchmarks & results

VIDRAFT reports one key quantitative result from the comparison between the NPU-trained and GPU-trained runs:

• Loss trajectory correlation coefficient: 0.99 between the NPU training run and an equivalent GPU training run under identical conditions.

This is a measure of training dynamics similarity, not throughput parity. The company explicitly notes this does not imply the NPU matches GPU training speed. What it does indicate is that the NPU-based training process converges along a statistically near-identical path to the GPU baseline — a meaningful signal that the gradient computation is numerically consistent, not approximate or divergent.

Additionally, VIDRAFT has verified that general-purpose public models can run inference and quantized-model execution on the same NPU hardware, suggesting the VKORE layer handles both the training and deployment sides of the lifecycle.

How to try it

The following artifacts are publicly available on Hugging Face:

• Model weights for VKORE-46M-NPU
• Model card with usage details
• Comparative analysis materials (the GPU vs. NPU training comparison)

You can browse or download the model using the Hugging Face CLI:

pip install huggingface_hub
huggingface-cli download vidraft/VKORE-46M-NPU

Note: The model name above follows the naming convention used in the press release. Verify the exact repository slug on the VIDRAFT Hugging Face page before running commands.

The core training code is not public — it is withheld for IP protection. A technical paper describing the implementation is planned for future release. There is no mention of a public API endpoint for this specific model at this time.

FAQ

Q: Is this actually backpropagation, or some gradient-free approximation like forward-mode differentiation or evolution strategies?
A: The press release explicitly states that VIDRAFT implemented backpropagation (역전파) on the NPU. It is described as standard reverse-mode gradient computation, not a gradient-free alternative. The technical mechanism is proprietary and will be detailed in a future paper.

Q: Does the 0.99 loss correlation mean NPU training is as fast as GPU training?
A: No — VIDRAFT explicitly clarifies this point. The correlation coefficient measures similarity of the training trajectory (i.e., how the loss curve evolves over steps), not wall-clock throughput or sample efficiency. Inference-only NPUs are generally slower for training workloads; the point of this result is correctness validation, not speed benchmarking.

Q: When will VKORE support LoRA fine-tuning and broader model pre-training?
A: VIDRAFT has announced LoRA fine-tuning as a roadmap item for the VKORE software layer, alongside inference acceleration and pre-training. No public timeline or availability date has been announced.

Q: Can I use this to train on my own NPU hardware?
A: Not yet. The training stack is internal. The public release covers the trained model weights and analysis materials. Watch for the forthcoming technical paper and any future VKORE SDK announcements.

---

Originally reported by ZDNet코리아 (2026-10-14) — source article.