VIDRAFT Trains a Language Model from Scratch on NPUs — No GPUs Required

TL;DR: Korean Pre-AGI startup VIDRAFT has pre-trained a large language model using NPUs (Neural Processing Units) instead of GPUs, challenging the industry assumption that GPU clusters are the only viable path for LLM pre-training. This represents a notable infrastructure diversification signal for ML engineers exploring alternative accelerator stacks. Access details are not yet publicly confirmed in the source.

What it is

VIDRAFT (비드래프트), a Korean AI startup self-described as working toward Pre-AGI systems, has announced that it has completed pre-training of a language model using NPU hardware — without relying on GPUs at any stage of the pre-training pipeline.

Key facts from the source:

• The training workload covered is pre-training (i.e., training a language model from scratch on large corpora), not fine-tuning or inference-only deployment.
• The hardware used is classified as NPU (Neural Processing Unit), a category of AI accelerator distinct from NVIDIA or AMD GPU products.
• The headline claim is explicitly that GPU infrastructure was not required for this pre-training run.
• The announcement was reported by BigGo 파이낸스, a Korean financial/tech outlet, on 2026-10-14.

No additional details about model architecture, parameter count, training data, or specific NPU vendor are confirmed in the available source text.

How it works

At a conceptual level, the significance here is hardware-stack portability for LLM pre-training workloads.

Historically, large language model pre-training has been tightly coupled to GPU clusters — particularly because:

1. Deep learning frameworks (PyTorch, JAX) and their CUDA kernels have been co-optimized with NVIDIA hardware for over a decade.
2. Distributed training primitives (NCCL, tensor parallelism, pipeline parallelism) have mature GPU-specific implementations.
3. Most public LLM training recipes assume GPU memory hierarchies and interconnects (NVLink, InfiniBand).

Running pre-training on NPUs instead implies that VIDRAFT has addressed — at some level — the following engineering challenges:

• Operator compatibility: Ensuring that the core tensor operations used in transformer pre-training (matrix multiplications, attention, layer norms, embedding lookups) are supported and performant on NPU hardware.
• Distributed training coordination: Managing gradient synchronization and parameter updates across NPU devices, which may use different interconnect topologies than GPU clusters.
• Framework integration: Either adapting an existing framework (or using a vendor-supplied SDK) to compile and schedule training graphs onto NPU compute units.
• Numerical precision handling: NPUs may support different floating-point formats than GPUs; pre-training stability requires careful management of precision across forward and backward passes.

The fact that this was pre-training rather than inference is the technically meaningful claim — inference on NPUs is already commercially common (mobile SoCs, edge chips), but pre-training at scale on NPUs is a far less documented territory.

Benchmarks & results

The source article body was not available due to a rate-limit error at retrieval time ({"message":"Rate limit exceeded"}), so no specific benchmark numbers, loss curves, training throughput figures, or downstream task evaluations can be reported here.

Qualitatively, the announcement positions this as a proof-of-concept or milestone demonstrating that:

• NPU hardware can sustain a full pre-training run to completion.
• GPU independence is achievable at the pre-training stage, not just inference.

Engineers should treat specific performance comparisons (e.g., NPU vs. GPU throughput, tokens/second, cost-per-token) as unconfirmed until VIDRAFT publishes a technical report or blog post with reproducible numbers.

How to try it

Based on the available source information, no public access path has been confirmed. There is no mention of:

• A Hugging Face model repository or organization page
• A GitHub repository with training code
• An OpenAI-compatible API endpoint
• A public demo or waitlist

If VIDRAFT releases the model weights, training code, or API access, those channels will likely be announced through their official channels. Engineers interested in following developments can watch for VIDRAFT's official announcements directly.

FAQ

Q: Why does it matter if pre-training runs on NPUs instead of GPUs?
A: GPU supply constraints and cost are a real bottleneck for AI development globally, including in Korea and other regions with limited access to NVIDIA allocations. Demonstrating that NPU hardware can run pre-training opens up alternative procurement and infrastructure paths for organizations that cannot access large GPU clusters. It also signals growing maturity in NPU software stacks.

Q: Does "NPU pre-training" mean the results are comparable to GPU-trained models?
A: The available source does not provide benchmark comparisons between NPU-trained and GPU-trained models, so no equivalence claim can be made here. Hardware-agnostic training is a necessary but not sufficient condition for quality parity — actual model quality depends on data, scale, and optimization choices that remain unpublished at this time.

Q: Is VIDRAFT's approach applicable to open-source frameworks like PyTorch or JAX?
A: The source does not specify which training framework or software stack was used. This is a key technical detail that would be needed to assess reproducibility or community applicability, and it remains unconfirmed.

---

Originally reported by BigGo 파이낸스(한국) (2026-10-14) — source article.