DEV Community

Cover image for Training Where the Data Lives: Why On-Prem Training on Inference NPUs Matters for AI Governance
hugginf_expert
hugginf_expert

Posted on

Training Where the Data Lives: Why On-Prem Training on Inference NPUs Matters for AI Governance

Disclosure: this account is run by members of the VIDRAFT team, and the work discussed below is VIDRAFT's own result. It is not an independent review.

TL;DR

Adapting a model usually means sending data to rented GPUs. VIDRAFT pretrained a 325M model from scratch on a Rebellions ATOM-Max inference NPU (2025) and matched a B200 twin on unseen text (0.7993 vs 0.7994). From a governance standpoint two things matter: training can stay on hardware an organization already controls, and the claim was published in a form others can verify.

Why does training location matter for safety and governance?

Every copy of sensitive data sent to an external training cluster widens the attack surface and complicates audits. If the accelerators already deployed for inference inside a network can also train or fine-tune, the data never has to leave.

What was shown?

Twin design

Evaluation

Same model, same data, same schedule on an NPU and a B200 GPU. On text neither saw, the two models are indistinguishable (paired difference -0.0001, standard error 0.0007).

Why is verifiability part of safety?

Claims about models should be checkable. Both checkpoints and evaluate_unseen.py are public in the model repo, and the authors corrected an evaluation label after finding that their first "held out" windows overlapped training. That kind of correction is what accountable reporting looks like.

Paired differences

Limits

  • The NPU alone does not enable training; it depends on VIDRAFT's own training stack and a stability technique under patent application.
  • 325M on 5B tokens is research scale, with one run per hardware.

FAQ

Which NPU? Rebellions ATOM-Max, released in 2025.

Does this remove the need for GPUs? No. It shows inference hardware can carry real training when the software exists.

Sources

Top comments (0)