DEV Community: AI OpenFree

Why Long bf16 Training Runs Stall: The Residual Stream Has No Room for Small Updates

AI OpenFree — Wed, 21 Oct 2026 16:05:00 +0000

TL;DR

bf16 keeps 8 significant bits. Once a residual-stream value grows to around 600, the gap between neighbouring bf16 numbers is 4, so any single update smaller than 2 is rounded away completely. Matrix multiplies tolerate this. The running sum that carries the residual stream does not, because it adds many small updates to one large number. In a simulation (illustrative, code below) a bf16 residual received between 0.3% and 43% of the drift an fp32 residual received over 4,000 steps, depending on the noise level, and in one setting it stopped moving entirely. The fix used by mainstream training code is cheap: keep the residual path in fp32, keep the matmuls in bf16. If you cannot, stochastic rounding recovers the average in expectation. Whatever you choose, watch residual and gate maxima over time, not just loss.

Why does a bf16 residual stream lose small updates?

A bf16 number has 1 sign bit, 8 exponent bits and 7 stored mantissa bits, which is 8 significant bits including the implicit leading one. The relative resolution is therefore about 2^-8 to 2^-7, roughly 0.4% to 0.8%. That is fine for a weight or an activation that gets consumed once.

A transformer's residual stream is different. Each block computes x = x + f(x). The same tensor is read-modified-written once per block, for dozens of blocks, and the quantity you care about is the sum. Floating-point addition rounds the result to the nearest representable number. If the update is smaller than half the gap at the current magnitude, the sum rounds back to the old value and the update has no effect.

I measured the gaps with torch.nextafter on bf16 tensors:

Magnitude of the value	Gap to the next bf16 number	Smallest update that registers
1	0.0078	above 0.0039
64	0.5	above 0.25
128	1	above 0.5
256	2	above 1
600	4	above 2
1,024	8	above 4
4,096	32	above 16

Round-to-nearest-even resolves an exact tie toward the even mantissa, so an update of exactly half the gap is also lost. At 600 an update of 2.0 does nothing, while 2.1 jumps the value by 4.

Are residual values really that large?

Yes, in specific dimensions. Sun et al. (Massive Activations in Large Language Models) report a handful of activations that are up to 100,000 times larger than typical ones, stable across inputs, and acting as implicit biases that shape attention. These live in the hidden states that flow between layers, which is the residual stream. Any model whose stream norm grows with depth can push some coordinates into the hundreds. A median coordinate near 1 and a few coordinates near 600 can coexist in the same tensor, and the large ones are exactly the ones that gate behaviour.

The consequence is a precision profile that is invisible in loss curves. A coordinate near 600 only moves in steps of 4. If the true gradient-driven drift on that coordinate is 0.3 per step, it takes a long run of same-direction updates before anything registers, and what registers arrives in lumps.

What does the failure look like in a simulation?

The numbers below are an illustrative simulation, not a model run. Each of 4,096 coordinates starts at 600. Every step adds mu + sigma * noise, with mu the small coherent drift and sigma the per-step noise. I compare three residual paths: a full fp32 residual, an fp32 residual that receives bf16-rounded updates (what you get when only the matmul output is bf16), and a fully bf16 residual.

import torch

def run(mu, sigma, steps=4000, d=4096, start=600.0):
    g = torch.Generator().manual_seed(1)
    x32 = torch.full((d,), start)                      # fp32 residual
    xmix = torch.full((d,), start)                     # fp32 residual, bf16 updates
    xb = torch.full((d,), start).to(torch.bfloat16)    # bf16 residual
    for _ in range(steps):
        u = mu + sigma * torch.randn(d, generator=g)
        x32 = x32 + u
        xmix = xmix + u.to(torch.bfloat16).float()
        xb = xb + u.to(torch.bfloat16)
    return [float((x - start).mean()) for x in (x32, xmix, xb.float())]

print(run(0.5, 1.0))   # [2000.5, 2000.4, 427.5]

Per-step update	fp32 residual	fp32 residual, bf16 updates	bf16 residual
mean 0.2, sd 0.5	800.2	800.2	2.4
mean 0.2, sd 1.0	800.5	800.5	348.5
mean 0.5, sd 1.0	2000.5	2000.4	427.5

Three things stand out.

First, the bf16 residual is not just noisy, it is biased. With a mean of 0.2 and a standard deviation of 0.5, almost no single update exceeds the rounding threshold, so the coherent drift of 800 collapses to 2.4. The signal is the small part, and it is the part that gets deleted.

Second, noise can hide the problem. With a standard deviation of 1.0 the bf16 residual still drifts, because occasional large noise draws cross the threshold and get rounded up to a full step of 4. It recovers 43% of the fp32 drift for mean 0.2 and 21% for mean 0.5. The result looks like progress, but it is progress that depends on how noisy the updates happen to be, which is not a property you want in an optimizer.

Third, in the third row the bf16 residual flattens at about 427. That is where the running value crosses 1,024 and the gap doubles to 8. At that point the updates of roughly 1 are far below half the gap and nothing registers. The residual freezes.

The middle column is the important control. Rounding the update itself to bf16, which is what a bf16 matmul output does, costs almost nothing as long as the accumulator is fp32. The loss comes from rounding the sum.

How do production training stacks handle this?

They keep the residual path wide. Megatron-LM exposes an fp32 residual connection option that is only valid together with fp16 or bf16 (its argument validation asserts exactly that), see NVIDIA/Megatron-LM. The Mamba reference implementation has a residual_in_fp32 flag that casts the residual to float32, see state-spaces/mamba. Both choices say the same thing: matmuls can be narrow, the running sum cannot.

The pattern in PyTorch is short. This block was run on CPU (PyTorch 2.4 or newer for nn.RMSNorm) to confirm it returns an fp32 residual:

import torch, torch.nn as nn

class Block(nn.Module):
    def __init__(self, d):
        super().__init__()
        self.norm = nn.RMSNorm(d)
        self.ff = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))

    def forward(self, x):                       # x is fp32 and stays fp32
        h = self.norm(x).to(torch.bfloat16)
        with torch.autocast(x.device.type, dtype=torch.bfloat16):
            u = self.ff(h)
        return x + u.float()                    # the sum happens in fp32

The cost is memory and bandwidth on one tensor per block boundary, not on the weights or the matmul inputs. For most training setups that is a small price compared with the weights, optimizer state and attention activations, though you should measure it on your own batch shape. The same idea applies on any low-precision backend: FP8 matmuls, fp16 on older GPUs, and accelerators with narrow accumulators.

What if you cannot keep the residual in fp32?

Use stochastic rounding for the residual add. Instead of rounding to the nearest bf16 value, round up with probability proportional to the fractional distance. The expected value of the result equals the true sum, so small updates survive on average, at the price of extra variance. A reference implementation in PyTorch adds random bits below the bf16 cutoff and truncates:

def sr_bf16(x32, g):
    b = x32.contiguous().view(torch.int32)
    r = torch.randint(0, 1 << 16, b.shape, generator=g, dtype=torch.int32)
    return ((b + r) & ~0xFFFF).view(torch.float32).to(torch.bfloat16)

In the same simulation, stochastic rounding of the bf16 residual recovered a drift of 797.5 against 800.2 for fp32 (mean 0.2, sd 0.5) and 1,998.8 against 2,000.5 (mean 0.5, sd 1.0). That is the average over 4,096 coordinates. Individual coordinates are noisier, so check that the variance is acceptable for your task. Hardware support for stochastic rounding varies, and a software version like the one above is slower than a plain add.

How do you catch this before the run is lost?

The trap is that nothing breaks early. The loss falls normally for thousands of steps, then a gate or norm starts creeping, and by the time the loss reacts the run is hard to recover. Three signals are worth logging:

Per-layer residual maxima. Log the max absolute value of the stream at each layer every N steps. A steady climb in a few coordinates is the warning that your resolution is shrinking.
Gate and router maxima. Anything that multiplies the residual (learned gates, per-channel scales, MoE router logits) amplifies a quantization bias. Track the max, not only the mean.
Gradient norm trend. A slow, monotone rise in the global gradient norm without a matching loss change is the typical signature of a stalled or biased state.

One more caution from experience: a single-step check will not find this. Compare the fp32 and bf16 gradient for one step and the cosine similarity looks excellent, because one rounding error per step is tiny. The failure is an accumulation effect over thousands of steps, so it needs a long-horizon test: run a short reference in fp32 and compare residual and gate maxima along the trajectory.

At VIDRAFT we saw a long low-precision run drift with exactly this signature, and it disappeared when the residual path was held in fp32 while the matrix multiplies stayed in bf16. This post stays on the general mechanism and the public evidence rather than our run details.

FAQ

Does this affect inference?
Much less. At inference each residual add happens once per layer on a fixed-length path, and errors do not accumulate across thousands of optimizer steps. Very deep models or long generations can still show small drift, so some inference stacks keep the residual in higher precision too.

Is fp16 better than bf16 here?
fp16 has 11 significant bits, so its gap at 600 is 0.5 instead of 4, which is better on resolution but has a much smaller range (maximum 65,504) and needs loss scaling. bf16 trades resolution for range. Neither is safe for an unbounded running sum, which is why the sum belongs in fp32.

Do I need fp32 for the weights too?
Separate question. Master weights in fp32 for the optimizer step is the standard defense against the same effect in parameter updates (a learning-rate-sized update added to a large weight). The residual stream is the activation-side version of the same arithmetic.

Will this show up as NaN?
Usually not. The typical outcome is silent: stuck coordinates, creeping gates, a loss curve that flattens or diverges slowly. NaN comes later or not at all.

How large is the memory cost of an fp32 residual?
One tensor of shape batch x sequence x hidden at each block boundary doubles from 2 to 4 bytes per element. If you use activation checkpointing the stored boundaries are exactly these tensors, so measure with your configuration rather than trusting a general estimate.

Can I reproduce your numbers?
Yes. Both scripts are about 20 lines, need only PyTorch on CPU, and use fixed seeds. The values above are from torch 2.12 on CPU. The simulation is illustrative: it shows the arithmetic, not the training dynamics of any particular model.

A Decision Model That Writes Nothing Just Took #1 on a 137-Benchmark Board

AI OpenFree — Tue, 20 Oct 2026 01:26:10 +0000

The result first

Darwin-27B-ZTC-v9 is #1 on the S1MB english-v1 leaderboard, read from the live board on 20 October 2026.

Rank	Model	Borda
1	Darwin-27B-ZTC-v9	90.38
2	kettei-fat v1.0	90.14
3	Superion S1	89.53
4	D3 (27B)	89.28
5	Darwin-27B-ZTC-v5	88.44

197 models, 137 benchmarks. Weights are Apache-2.0.

Before anyone quotes that number, the rest of this post explains what it does and does not mean, because "first on a leaderboard" is close to meaningless without the aggregation method attached.

What Borda aggregation actually measures

Most boards average a score over tasks. S1MB does not. It ranks each of the 137 benchmarks separately, then aggregates the ranks. That is a Borda count.

The difference bites. Under averaging, a benchmark whose metric happens to spread from 0.1 to 0.9 contributes far more to your average than one packed between 0.80 and 0.84. Nobody chose that weighting. It fell out of metric scaling. A model that is ordinary everywhere and extraordinary on three wide-spread tasks can land near the top.

Under Borda every benchmark contributes the same thing: your position among the entrants. Winning by 0.3 and winning by 0.001 earn the same rank. To finish first you have to place well on most of the 137 boards simultaneously.

What Borda gives up is margin. A board where the top five differ by a hair on every task still produces a confident-looking order. So read both columns. S1MB publishes a task average next to Borda, and on the current English board they disagree: this model is second on task average at 72.20, while the highest task average belongs to the model sitting third on Borda. Neither column is wrong. They answer different questions.

The model generates no text

ZTC stands for zero token choice. You give it a situation and a typed question. It returns a probability for every option in one forward pass and writes nothing.

Type	Returns	Example
noul	Probability the statement holds	Does this ticket need a reply today?
choice	A probability per option	Which team should own this?
score	A probability per level	How satisfied is this review, 1 to 5?

No output to parse. No format that can drift on the thousandth call. No reasoning tokens to wait through.

Calibration is the property that matters

Here is the part teams underrate. Accuracy alone does not let you build a gate.

A gate needs a threshold, and a threshold needs probabilities that mean what they claim. If 0.9 is right about nine times in ten, you can auto-approve above 0.9, route the rest to a human, and predict your error rate before shipping. If the model is accurate on average but its 0.9 means anywhere from 0.6 to 0.99, no threshold is safe.

This is why decision benchmarks score calibration directly. Brier score for yes or no, probability mass on the correct option for choice, expected score error for scales. A model can top an accuracy table and be useless as a gate.

Selected per-task results against the published baseline:

Task	Baseline	v9
boardgameqa-choice	0.500	0.953
snli-choice	0.790	0.968
gretel-pii-choice (entity type)	0.653	0.981
argument-quality-choice	0.857	0.959
ud-ewt-choice (part of speech)	0.433	0.833

What cheap checking changes

Count where an agent's tokens go. A large share is the agent asking itself small questions. Is this tool call safe. Does this passage answer the question. Did the last step succeed. Should this escalate.

Every one of those is a yes, a pick, or a score, and every one currently costs a full generation. At a thousand a minute the checking costs more than the work, so teams quietly drop the checks. Not because they decided the checks were unnecessary, but because they were unaffordable.

Make the check one forward pass and the calculus flips. Gate every tool call. Score every retrieved passage. Confirm every step. Keep a confidence number on each decision so escalation is a threshold rather than a guess.

Serving it

hf download FINAL-Bench/Darwin-27B-ZTC-v9 systemone_config.json --local-dir .
transformers serve FINAL-Bench/Darwin-27B-ZTC-v9 --decision-config systemone_config.json

It runs on the standard /v1/systemone endpoint of Hugging Face transformers serve, so clients written against that public shape work unchanged.

How to check this claim

Open the board and read the column and date yourself. It is live and 197 entries move.
Note that most public boards mix self-reported and verified entries, and S1MB says so.
Best test: run your own decision log through it and see whether the probabilities hold at the threshold you intend to use. Open weights make that a one-afternoon experiment rather than a procurement decision.

Darwin-180B-RSI Tops MMLU-Pro and GPQA Diamond on AIhoster's Independent Open-Model Benchmark Board

AI OpenFree — Mon, 19 Oct 2026 11:01:30 +0000

Darwin-180B-RSI Tops MMLU-Pro and GPQA Diamond on AIhoster's Independent Open-Model Benchmark Board

TL;DR: VIDRAFT's Darwin-180B-RSI, a 180-billion-parameter open model, has claimed the #1 position on both the MMLU-Pro and GPQA Diamond leaderboards as tracked by Russian benchmark aggregator AIhoster — scoring 88.1% and 94.4% respectively, outperforming models with parameter counts an order of magnitude larger. Developers interested in frontier reasoning capabilities in an open-weight format should take note.

What it is

Darwin-180B-RSI is a 180B-parameter open-weight language model developed by VIDRAFT (비드래프트), a Korean Pre-AGI AI startup. It is published under the Hugging Face repository identifier FINAL-Bench/Darwin-180B-RSI.

AIhoster is an independent Russian benchmark aggregation site that collects publicly reported evaluation results from model cards, papers, and leaderboards — without recomputing or normalizing scores into a composite index. Each benchmark entry is tied to an exact repository ID and a specific evaluation configuration, making comparisons methodologically transparent.

As of the AIhoster update on 2026-10-19, Darwin-180B-RSI holds:

#1 on MMLU-Pro — evaluated on all 12,032 test items
#1 on GPQA Diamond — evaluated on all 198 items

Notably, VIDRAFT has multiple Darwin-family models appearing in the GPQA Diamond rankings simultaneously: Darwin-397B-ZTC (#3 at 93.4%) and Darwin-398B-JGOS (#9 at 90.9%), suggesting an active, iterative development pipeline across different model scales and inference configurations.

How it works

At a conceptual level, Darwin-180B-RSI appears to be a large-scale transformer-based language model optimized for extended reasoning. A few things can be inferred from the publicly disclosed evaluation configurations:

Extended thinking budget: The MMLU-Pro and GPQA evaluations both used a thinking budget of up to 131,072 tokens — consistent with a chain-of-thought or "reasoning model" architecture that allocates substantial token budget for intermediate reasoning steps before producing a final answer.
Sampling strategy varies by benchmark: For MMLU-Pro, a single sample was drawn (temperature=1.0, top_p=0.95, top_k=20). For GPQA Diamond, majority voting over up to 16 samples was used under identical sampling parameters — a test-time compute scaling strategy that trades inference cost for accuracy on hard problems.
Precision: The model runs in bf16, which is standard for large-scale inference on modern accelerators.
Efficiency relative to parameter count: Achieving #1 on both benchmarks with 180B parameters — while outperforming models like DeepSeek-V4-Pro (1.6T parameters) and Kimi-K2.5 (1T parameters) on MMLU-Pro — suggests strong parameter efficiency, likely from architectural choices or training data curation rather than raw scale alone.

No architectural specifics, training hyperparameters, or dataset details are publicly disclosed in the source material.

Benchmarks & results

All figures below are sourced directly from AIhoster's leaderboard (updated 2026-10-19), which in turn cites the Darwin-180B-RSI model card.

MMLU-Pro (multi-domain expert knowledge; higher accuracy = better):

Rank	Model	Score	Params
1	Darwin-180B-RSI	88.1%	180B
2	MiniMax-M2.1	88.0%	229B
3	Intern-S2-Preview	88.0%	36.1B
4	Qwen3.5-397B-A17B	87.8%	403B
5	DeepSeek-V4-Pro	87.5%	1.6T

GPQA Diamond (expert-level physics, chemistry, biology; higher accuracy = better):

Rank	Model	Score	Params
1	Darwin-180B-RSI	94.4%	180B
2	Kimi-K3	93.5%	2.8T
3	Darwin-397B-ZTC	93.4%	404B
4	Ornith-1.5-397B	92.8%	403B
5	Qwen3.8-2.4T-A95B	92.6%	2.4T

The GPQA result is particularly notable: Darwin-180B-RSI achieves 94.4% using majority voting over up to 16 samples, while the #3 model, Darwin-397B-ZTC (also a VIDRAFT model), achieves 93.4% with greedy decoding and no voting — a different inference regime entirely.

How to try it

The model is listed under the Hugging Face organization FINAL-Bench with the repository FINAL-Bench/Darwin-180B-RSI. If the weights are publicly available there, you can browse or download via the standard Hugging Face CLI:

huggingface-cli download FINAL-Bench/Darwin-180B-RSI

Access details beyond what is on the model card — including any API endpoints, licensing terms, or inference service availability — are not confirmed in the source material. Check the model card directly for the most current access instructions.

FAQ

Q: How does Darwin-180B-RSI beat models with 10× more parameters on MMLU-Pro?
A: The source doesn't explain internal architecture, but the evaluation configuration gives a clue: the model uses a 131,072-token thinking budget, suggesting extended chain-of-thought reasoning. Parameter efficiency at inference is a known benefit of well-optimized mixture-of-experts or distillation-based architectures, though VIDRAFT has not publicly confirmed which approach is used here.

Q: Are the GPQA and MMLU-Pro scores independently verified?
A: AIhoster explicitly states it does not recompute scores — it aggregates from public model cards and leaderboard sources, matched by exact repository ID. The source for Darwin-180B-RSI's scores is listed as its own model card. Independent third-party reproduction has not been confirmed in this reporting.

Q: Can I use Darwin-180B-RSI commercially?
A: Licensing terms are not specified in the source article. Check the model card at FINAL-Bench/Darwin-180B-RSI on Hugging Face for the applicable license before any production or commercial use.

Originally reported by AIhoster (러시아) (2026-10-18) — source article.

Five Open Science Challenges Where a Verifier, Not the Submitter, Decides the Score

AI OpenFree — Mon, 19 Oct 2026 00:50:44 +0000

TL;DR

Verifiers are one of the busiest topics in AI research this week: several papers on stronger verifiers appeared on the same day. The idea is simple. A model proposes, and a separate checker decides whether the proposal is right.

FINAL-Bench runs five open science challenges built on exactly that split. You submit a molecule, a crystal, a prediction or a decoder. A verifier you cannot see scores it, and the answer key never leaves the organiser's hardware.

Challenge	What you submit	Verified against	Status
Open Discovery Challenge	A drug candidate (SMILES)	Target and counter-target structure prediction, ADMET, novelty	Seasons 2 to 4 open
Open Materials Challenge	A lithium electrolyte crystal	Oxidation and lithium-metal stability, novelty of use	Season 1 open to 2026-11-30
Open Superconductor Challenge	A d-wave pairing estimate per 2D material	A many-body reference solver on top entries	Season 1 open to 2026-12-31
Open Quantum Challenge	Hubbard energies or a surface-code decoder	A frozen private ground truth	Two seasons open into 2027
LEADBOARD	Property predictions for held-out molecules	Labels that are never released	21 boards open

Why a hidden verifier matters

If the grader is public, an entry can be shaped against the grader instead of against the biology or the physics. Each challenge therefore publishes its rules, weights and entry limits, and keeps the scoring implementation and answer key private.

Built for agents as well as people

None of these needs special hardware for a first entry. LEADBOARD ships a filled-in prompt and a runnable skeleton for every board, and the superconductor challenge says plainly that AI coding agents can take part. An agent with a code tool such as Claude Code and a set of agent skills can read the rules, write the pipeline and submit.

Repositories

open-discovery-challenge · open-materials-challenge · leadboard · quantum-bench-harness

All scores in these challenges are computational predictions, not laboratory measurements. A high score is a hypothesis worth testing.

Six Open-Source Projects from VIDRAFT, Now on GitHub

AI OpenFree — Sun, 18 Oct 2026 03:15:56 +0000

We keep most of our work on Hugging Face, but code belongs where developers already look for it. So the open parts of what we build now live on GitHub at github.com/final-bench. Here are six projects, what each one does, and where to start.

1. ONGRID: GraphRAG that also decides

Most GraphRAG tools retrieve and generate. ONGRID adds the step that usually goes missing: deciding whether the answer can be trusted.

Drop in documents and ONGRID builds a 3D ontology graph of typed entities and relations. Ask a question and an LLM answers from facts in that graph. Then a confidence judge checks the answer without generating any tokens.

Repository: final-bench/ongrid

2. FINAL Bench: does the model know when it is wrong?

Accuracy benchmarks ask whether the final answer is right. FINAL Bench asks a different question: can a model detect its own error, admit it, and fix it?

It covers functional metacognition across 100 tasks in 15 domains, from mathematics and medicine to ethics and economics. The paper is on arXiv and the dataset is on Hugging Face.

Repository: final-bench/leaderboard

3. Open Quantum Challenge harness: the yardstick is public

A leaderboard is only as trustworthy as its scoring code, so we published the scoring code.

The harness holds reproducible scoring and frozen test problems for the Open Quantum Challenge: quantum simulation tasks and surface-code decoder scoring, reported as logical error rate with 95% Wilson intervals. Anyone can run it and get the same numbers the leaderboard shows. Contestants keep their own methods to themselves.

Repository: final-bench/quantum-bench-harness

4. POCKET: strong models on your own laptop

POCKET is a family of models quantized to run locally on a laptop CPU, with no GPU and no network connection.

The repository is the model index plus run recipes. The GGUF weights are on Hugging Face, where the POCKET builds have been downloaded more than two million times in total.

Repository: final-bench/pocket

5. ZTC: can you trust this answer?

ZTC (Zero-Token Confidence) reads a question and an answer written by any model and returns whether the answer can be trusted. It does this in one forward pass, without generating text.

It comes in 4B, 9B and 27B sizes, from CPU-friendly to highest quality. The repository holds the inference code, the model index and a description of the method.

Repository: final-bench/ztc

6. ALL Bench Leaderboard: one place to compare

Instead of hunting through scattered tables, ALL Bench Leaderboard gathers public benchmark results in one place so models can be compared side by side. It is released under Apache-2.0.

Repository: final-bench/ALL-Bench-Leaderboard

Where to start

Building retrieval with verification: start with ONGRID and ZTC.
Evaluating models: FINAL Bench and ALL Bench Leaderboard.
Running models offline: POCKET.
Quantum error correction research: the Open Quantum Challenge harness.

Issues and pull requests are welcome on every repository. If something does not run, tell us there and we will fix it.

VIDRAFT's VKORE Enables Full Model Pre-Training on Inference-Only NPUs — 0.99 Loss Correlation with GPU Baselines

AI OpenFree — Fri, 16 Oct 2026 07:00:36 +0000

VIDRAFT's VKORE Enables Full Model Pre-Training on Inference-Only NPUs — 0.99 Loss Correlation with GPU Baselines

TL;DR: VIDRAFT Lab has demonstrated that inference-optimized NPUs — hardware traditionally limited to forward-pass workloads — can execute full-scale language model pre-training from scratch using a software abstraction layer called VKORE. A 46M-parameter model trained across 32 inference-dedicated chips achieved a 0.99 loss correlation against standard GPU training runs. For ML engineers, this signals a potential paradigm shift: training infrastructure may no longer require GPU-exclusive hardware stacks.

What it is

VKORE is VIDRAFT Lab's software-defined training layer designed to unlock backpropagation and optimizer state management on NPU hardware that was originally architected for inference-only workloads. The core claim is straightforward but structurally significant: hardware that the industry has historically excluded from the training pipeline can, with the right software orchestration, participate in full model pre-training.

The milestone demonstrated publicly involves:

A 46M-parameter language model trained entirely from scratch
Execution across 32 inference-dedicated NPU chips
A 0.99 loss correlation with equivalent GPU-based training benchmarks

This is not fine-tuning or distillation — it is pre-training, meaning gradient computation, weight updates, and optimizer state accumulation are all happening on hardware that was never designed with that workload in mind.

How it works

At a conceptual level, VKORE functions as a software abstraction layer that remaps the computational graph of a standard training loop — forward pass, loss computation, backward pass, and optimizer step — onto the execution primitives available on inference-optimized silicon.

The high-level approach works roughly like this:

Compute graph decomposition: The training workload is broken into operations that inference NPUs can natively execute (matrix multiplications, activation functions, elementwise ops), rather than relying on CUDA-specific primitives.
Gradient orchestration in software: Because inference NPUs lack dedicated hardware pathways for backpropagation, VKORE manages gradient flow and accumulation through software-level scheduling across the chip array.
Distributed state management: Optimizer states (momentum buffers, variance estimates, etc.) are managed across the 32-chip configuration via software-coordinated memory handling rather than hardware-native all-reduce operations.

The framing from VIDRAFT is explicitly "NPU-as-CUDA" — the goal is to treat the NPU cluster as a general-purpose training substrate, with software compensating for what the hardware was not built to do natively. This is analogous in spirit to how early ML frameworks abstracted over CPU SIMD units before GPU acceleration became standard, but applied in the reverse direction: taking specialized hardware and generalizing it upward through software.

Benchmarks & results

The publicly reported figures are specific:

Metric	Result
Model size	46M parameters
Hardware	32 inference-dedicated NPUs
Training task	Language model pre-training from scratch
Loss correlation vs. GPU baseline	0.99

A 0.99 loss correlation is a strong numerical claim — it means the training trajectory on NPU hardware is nearly statistically indistinguishable from an equivalent GPU run. This is the benchmark VIDRAFT is anchoring the disclosure on, and it is the figure that makes this more than an academic curiosity.

Important caveats noted in coverage: the result is demonstrated at 46M parameters. How this approach scales toward billion-parameter models — where memory bandwidth, sustained I/O throughput, and thermal load become substantially more demanding — has not yet been publicly addressed. Inference-optimized chips are rarely designed for the continuous, high-bandwidth memory access that large-scale training sustains over hours or days.

How to try it

VKORE and the associated pre-training framework have not been announced as publicly available in the source coverage. No Hugging Face repository, GitHub link, or API endpoint has been disclosed at the time of this article.

If and when VIDRAFT makes tooling available, the most likely channels based on their existing public presence would be:

Hugging Face Hub — for model checkpoints or training configurations
GitHub — for the VKORE software layer or example training scripts
OpenAI-compatible API — for inference access to resulting models

Watch VIDRAFT's official channels for announcements. This article will be updated when public access is confirmed.

FAQ

Q: Does this mean NPUs can fully replace GPUs for training today?
A: Not at scale — at least not based on current public evidence. The 0.99 loss correlation result holds for a 46M-parameter model on 32 chips. The open question is whether VKORE's software orchestration approach can sustain comparable fidelity and throughput as model and dataset sizes increase by orders of magnitude. The memory wall and thermal constraints of inference-optimized hardware are real engineering challenges that remain publicly unaddressed at larger scales.

Q: What does "inference-only NPU" actually mean in this context?
A: Inference-optimized NPUs are chips designed to execute a trained model's forward pass efficiently — think low-latency, low-power matrix operations for deployment scenarios. They typically lack hardware support for gradient computation, high-bandwidth bidirectional memory access patterns, and the communication fabric needed for distributed training. VKORE's contribution is making those capabilities available in software, layered on top of what the chip can already do.

Originally reported by Mukta AI(글로벌) (2026-10-14) — source article.

Darwin-27B-ZTC-v5 Tops the Typed Decisions Leaderboard with a 0.81 Accuracy Score

AI OpenFree — Fri, 16 Oct 2026 03:01:17 +0000

Darwin-27B-ZTC-v5 Tops the Typed Decisions Leaderboard with a 0.81 Accuracy Score

TL;DR: VIDRAFT's Darwin-27B-ZTC-v5 has reclaimed the #1 spot on the Hugging Face-official LocalLLaMA/typed-decisions benchmark, scoring 0.81 accuracy — the highest recorded on this leaderboard to date. Typed Decisions is a structured, probabilistic classification benchmark covering real-world agentic workflows, making this result particularly relevant to engineers building decision-automation and AI agent systems.

What it is

Darwin-27B-ZTC-v5 is the latest model release from VIDRAFT (비드래프트), a Korean Pre-AGI AI startup. It is evaluated under the Hugging Face-official LocalLLaMA/typed-decisions benchmark, which measures a model's ability to make calibrated, structured decisions across multi-dimensional classification tasks derived from enterprise workflow scenarios.

The typed-decisions dataset itself is worth understanding:

License: Apache-2.0
Size: 1K–10K samples
Modalities: Tabular + Text
Task type: Text Classification (structured decisions)
Tags: structured-decisions, calibration, probabilistic-classification, system-one, workflow-evaluation
Language: English
Subsets: agent_trace_observability, customer_service, invoice_processing, security_incidents — each with 400 rows, split into train (300) and test (100)

Each sample in the dataset includes rich, multi-label output fields: outcome, action, needs_review, risk, and urgency — each with a predicted label, a confidence score, and a full probability distribution. This makes it a calibration-aware benchmark, not just an accuracy benchmark.

How it works

At a conceptual level, Typed Decisions evaluates whether a model can simultaneously classify multiple decision dimensions from a structured workflow state — essentially simulating what an intelligent agent would need to do when observing a running process and deciding how to respond.

Each benchmark sample presents:

A workflow state description (a structured text blob describing what an agent or system is doing)
A set of structured questions the model must answer (e.g., "What is the risk level?" or "Does this trace need human review?")
Gold labels with supporting rationale and factor explanations

The model must output not just a label but a probability distribution across label classes, enabling assessment of calibration quality alongside raw accuracy. For example, the needs_review__probability_true field captures the model's confidence that a human should intervene — a nuanced signal beyond a binary yes/no.

This design mimics System-1 workflow evaluation: fast, structured judgment under realistic enterprise conditions (data erasure queues, security incident triage, invoice routing, customer service escalation, and agent trace observability).

Darwin-27B-ZTC-v5's previous iteration, Darwin-27B-ZTC, already held the top position with a 0.74 score. The v5 release achieves a +0.07 improvement in accuracy on the same leaderboard, representing a meaningful gain in this tightly contested benchmark space.

Benchmarks & results

From the public LocalLLaMA/typed-decisions leaderboard (as of 2026-10-16):

Rank	Model	Accuracy Score
1	FINAL-Bench/Darwin-27B-ZTC-v5	0.81
2	FINAL-Bench/Darwin-27B-ZTC	0.74
2	empiriolabsai/aplomb-1	0.74
4	ProKope-AI/ProKope-421M-8K	0.72
5	ProKope-AI/ProKope-421M	0.71
6	adaptive-classifier/typed-decisions-minilm-l6-specialist	0.61

Notable observations:

Darwin-27B-ZTC-v5 leads by +0.07 over the next closest competitors
The benchmark is filtered to base models only (no fine-tuning with task-specific supervision declared separately)
Competing models range from 421M to multi-billion parameters, making Darwin-27B-ZTC-v5's score especially notable given the parameter efficiency context

How to try it

The LocalLLaMA/typed-decisions dataset is publicly available on Hugging Face and can be accessed directly:

# Load the dataset via the Hugging Face Datasets library
pip install datasets

python -c "
from datasets import load_dataset
ds = load_dataset('LocalLLaMA/typed-decisions', 'customer_service')
print(ds['test'][0])
"

You can also explore it interactively via Data Studio on the dataset page, or query it with the built-in SQL Console available on the Hugging Face dataset viewer.

For the Darwin-27B-ZTC-v5 model itself: access details and availability should be confirmed directly through VIDRAFT's official channels. The leaderboard entry links to evaluation results via the FINAL-Bench organization on Hugging Face — check there for model card and availability updates.

FAQ

Q: Is Typed Decisions a calibration benchmark or just an accuracy benchmark?
A: Both. Each prediction includes a full probability distribution per label class (e.g., outcome__probabilities, risk__score, urgency__score), meaning models are evaluated on how well-calibrated their confidence is — not just whether the top label is correct. This makes it more demanding than flat classification benchmarks.

Q: What enterprise use cases does this benchmark cover?
A: The four subsets map directly to common enterprise automation scenarios: customer service triage, invoice processing routing, security incident classification, and AI agent trace observability (deciding whether an autonomous agent action needs human review). These are practical, high-stakes decision domains.

Q: How does Darwin-27B-ZTC-v5 differ from the earlier Darwin-27B-ZTC?
A: Based on the public leaderboard data, v5 achieves a 0.81 accuracy score versus 0.74 for the prior version — a +0.07 gain. Internal architectural or training differences are not publicly disclosed.

Originally reported by FINAL-Bench (2026-10-16) — source article.

VIDRAFT Trains a 46M-Parameter Model Entirely on Inference-Only NPUs — No CUDA Required

AI OpenFree — Thu, 15 Oct 2026 23:01:10 +0000

VIDRAFT Trains a 46M-Parameter Model Entirely on Inference-Only NPUs — No CUDA Required

TL;DR: VIDRAFT has pre-trained VKORE-46M-NPU, a 46-million-parameter language model, using only inference-oriented NPU hardware — completely bypassing NVIDIA GPUs and the CUDA stack. This proof-of-concept directly challenges the assumption that backpropagation requires high-end GPU memory hierarchies, and signals a potential path toward hardware-agnostic ML development that could meaningfully lower the cost of AI experimentation.

What it is

VKORE-46M-NPU is a 46-million-parameter neural network model developed by VIDRAFT, the Korean Pre-AGI AI startup. Its defining characteristic is not the parameter count — it is the hardware it was trained on.

Rather than relying on NVIDIA GPUs and the CUDA ecosystem that underpins virtually all large-scale model pre-training today, VIDRAFT completed the full pre-training run on inference-only NPU hardware — silicon that is typically designed for low-power, edge, or mobile inference workloads, not backpropagation.

Key facts from the source:

Model name: VKORE-46M-NPU
Parameter count: 46 million
Training hardware: Inference-oriented NPUs (no high-end GPU involvement reported)
Training type: Full pre-training (not fine-tuning or distillation from a GPU-trained teacher)
Framing: Explicitly described as a proof-of-concept targeting hardware-agnostic AI development

How it works

At a conceptual level, the challenge VIDRAFT is solving is significant. Standard neural network training relies on backpropagation: computing gradients through the entire network and using those gradients to update weights. This process is computationally intensive, requires high numerical precision (typically FP32 or BF16), and assumes a memory architecture that supports both the forward pass and the backward pass simultaneously.

Inference-only NPUs are generally optimized for the forward pass only. They often rely on quantized arithmetic, fixed-function datapaths, and tightly constrained memory hierarchies — all of which create friction when you attempt to run a standard training loop.

What VIDRAFT appears to have done, at a high level, is adapt the training pipeline to operate within those constraints:

Gradient computation had to be handled within the precision and datapath limits of the NPU's native operations
Memory management during the backward pass would need to be restructured relative to the assumptions baked into typical GPU training frameworks
Software abstraction is doing heavy lifting here — the hardware-level limitations are being worked around at the framework or compiler layer, not resolved at the silicon level

The source stops short of detailing the specific techniques used, and no internal hyperparameters, loss curves, or compiler internals are disclosed. The framing is proof-of-concept: demonstrating that the hardware barrier is a software constraint as much as a physical one.

Benchmarks & results

The source does not publish quantitative benchmark comparisons — no perplexity scores, downstream task accuracy numbers, or training throughput figures (tokens/second) are provided for VKORE-46M-NPU.

Qualitatively, the claim is that pre-training on inference-only NPUs reached convergence — meaning the model trained successfully and was not abandoned due to hardware failure or numerical instability. That convergence on this class of hardware is itself the result being reported.

What remains publicly unaddressed (and is worth watching for in follow-up reporting):

Whether the loss landscape is comparable to a GPU-trained baseline of equivalent size
Training wall-clock time relative to cloud GPU alternatives
Energy consumption and thermal behavior over extended training runs
Scalability trajectory toward billion-parameter models

How to try it

Public access to VKORE-46M-NPU has not been announced in this source. No Hugging Face repository, GitHub link, OpenAI-compatible API endpoint, or model weights download is mentioned in the coverage.

If VIDRAFT follows its established pattern with other VKORE-series models, a Hugging Face release may follow the proof-of-concept announcement — but that is not confirmed here. Watch the VIDRAFT Hugging Face organization page and their official channels for updates.

FAQ

Q: Does "inference-only NPU" mean the chip literally cannot do training, or just that it wasn't designed for it?
A: The distinction matters. "Inference-only" refers to the chip's intended design purpose and typical optimization targets — not necessarily a hard physical impossibility. VIDRAFT's work suggests the hardware can be coaxed into running training workloads through software-level adaptation, even if it was never architected with backpropagation in mind.

Q: Is a 46M-parameter model large enough to validate the approach for production use cases?
A: At 46 million parameters, VKORE-46M-NPU is orders of magnitude smaller than the multi-billion-parameter models that define current state-of-the-art deployments. The honest answer is: not yet, at this scale. VIDRAFT frames this explicitly as a proof-of-concept. The critical open question — which the source does not answer — is whether the approach scales to 1B+ parameters without prohibitive training time or quality degradation compared to GPU baselines.

Originally reported by Mukta AI(글로벌) (2026-10-14) — source article.

Who Holds #1 on Every Hugging Face Official Benchmark? A Live Index

AI OpenFree — Thu, 15 Oct 2026 14:01:23 +0000

Hugging Face marks certain datasets as official benchmarks. Each one carries a leaderboard on its dataset page, built automatically from the .eval_results files that model repositories publish. GPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, AIME 2026, HMMT February 2026, Terminal-Bench, MMMU-Pro, LEXam, the Open ASR Leaderboard, and more.

The problem: these leaderboards live on dozens of separate dataset pages. There is no single place that answers a simple question.

Who actually holds #1, across all of them?

So we built one.

What the index shows

It is a live, read-only view that recomputes from the Hugging Face API on every visit:

#1 by organization — how many official leaderboards each organization currently leads, with a share chart.
#1 by country — first-place spots grouped by the country of the publishing organization.
Who leads which field — a heatmap of #1 spots across math, science and knowledge, coding, agents and terminal, documents and OCR, vision and video, speech, law and finance, retrieval and robotics.
How far ahead #1 is — the relative gap between #1 and #2 on every board, direction-aware so that lower-is-better metrics like word error rate are handled correctly.
Every benchmark — the #1 and #2 model and score, and the number of entries, for each board.

How it is built

Nothing is hand-entered. The page reads the list of datasets that Hugging Face tags as official benchmarks, pulls each board's ranked entries from the public API, and aggregates. Because a new #1 only appears after Hugging Face indexes the eval results, the index reflects the public record with the same lag as the source leaderboards.

A few design notes for anyone building something similar:

Ties at rank 1 are counted for every tied organization.
The #1-to-#2 gap is measured relative to the #2 score and is direction-aware.
Country is derived from the publishing organization, not the model name.

Why it matters

Benchmark claims are easy to assert and hard to check. A single page that reads straight from the eval results turns "we lead on X" into something a reader can verify in one click, including how large the margin actually is. For teams, it is a fast way to see where the frontier sits in your field and who is defending it.

As of early October 2026 the index tracked around 49 official benchmarks with roughly a thousand leaderboard entries in total, and the single organization holding the most first-place spots was FINAL-Bench with eleven, spanning science, math, multimodal, law, document extraction, structured output and decision-making.

Open it, sort by field or country, and see for yourself.

Live index: https://huggingface.co/spaces/quantid/huggingface-official-benchmark-leaderboards

VKORE: LoRA Training on Inference-Only NPUs with 3.8 Inference Speedup and 0.9999 PyTorch Parity

AI OpenFree — Thu, 15 Oct 2026 07:01:26 +0000

VKORE: LoRA Training on Inference-Only NPUs with 3.8× Inference Speedup and 0.9999 PyTorch Parity

TL;DR: VKORE is a hardware-independent software acceleration layer from VIDRAFT that unlocks LoRA fine-tuning of transformer models on NPUs traditionally locked to inference workloads — while simultaneously boosting single-user inference speed by 3.8× and server throughput by 4.1×. It achieves 0.9999 numerical parity with standard PyTorch during LoRA training, running roughly 170× faster than the same server's CPU. If your organization runs NPU inference fleets, VKORE could eliminate the need for separate GPU infrastructure just for fine-tuning.

What it is

VKORE is a software acceleration layer developed by VIDRAFT that sits between your ML workloads and the underlying NPU hardware. Key characteristics:

Hardware-independent design: Built for data centres operating heterogeneous fleets of both GPUs and NPUs, rather than targeting a single vendor's silicon.
Dual workload support: Handles both inference acceleration and LoRA-based training on the same NPU hardware — a combination that inference-only NPU deployments have traditionally not supported.
Target hardware class: NPU servers (the benchmarks were run on a Korean NPU server with eight cards and 32 chips).
Reference model used for validation: Google's open Gemma 4 26B model.

The core value proposition is an architectural shift in how data centres are provisioned: instead of maintaining separate GPU clusters purely for periodic fine-tuning jobs, VKORE lets inference NPU fleets absorb those training workloads during low-traffic windows.

How it works

At a conceptual level, VKORE operates as an abstraction and optimization layer that intercepts the compute graph of transformer model operations and re-targets them efficiently to NPU instruction sets — including the gradient computation paths that LoRA training requires.

A few high-level mechanisms worth understanding:

LoRA compatibility on inference silicon: Standard NPUs are often shipped with firmware and driver stacks optimized exclusively for forward-pass operations (matrix multiplications, attention, softmax). VKORE extends the executable operation set to cover the backward-pass primitives that LoRA adapter training demands, without requiring changes to the NPU firmware itself.
Numerical fidelity: Achieving 0.9999 parity with PyTorch means VKORE is carefully managing floating-point operation ordering and precision handling across the NPU's native compute units. This level of parity is significant — it means LoRA adapter weights trained via VKORE should be functionally interchangeable with those produced by a standard PyTorch GPU training run.
Idle-fleet utilization: Because LoRA fine-tuning is parameter-efficient by design (only a small set of low-rank adapter matrices are updated, not the full model weights), the memory and compute overhead is modest enough that it can realistically be scheduled on inference NPUs during off-peak hours without disrupting SLA-bound inference capacity.
Hardware abstraction: The layer is designed to expose a consistent interface regardless of whether the underlying cards are NPUs or GPUs, simplifying fleet management in mixed environments.

Benchmarks & results

All numbers below are sourced directly from the published report. Test platform: a Korean NPU server with eight cards and 32 chips, running Google Gemma 4 26B.

Metric	Result
Single-user inference speedup (vs. baseline NPU)	3.8×
Server throughput speedup (vs. baseline NPU)	4.1×
LoRA training speed vs. same server's CPU	~170× faster
Numerical parity with PyTorch (LoRA training)	0.9999

A few things worth noting for engineers evaluating these numbers:

The inference speedups (3.8× and 4.1×) are measured on the same NPU hardware, meaning the gain comes entirely from VKORE's software optimization, not a hardware upgrade.
The 170× CPU comparison is a practical benchmark for teams currently running fine-tuning on CPU nodes as a cost-saving measure — the message is that the NPU path via VKORE is not just "good enough" as an alternative, it's dramatically faster.
The 0.9999 parity figure is especially relevant for production ML pipelines where training reproducibility and output consistency across compute backends matter for model validation and compliance workflows.

How to try it

The source article does not mention a public Hugging Face repository, GitHub release, pip package, or API endpoint for VKORE at the time of writing. Access details have not been publicly disclosed.

If you are interested in evaluating VKORE for your data centre, the appropriate next step is to contact VIDRAFT directly through their official channels. Keep an eye on VIDRAFT's announcements for any public SDK, early-access program, or developer preview.

FAQ

Q: Does VKORE require modifying NPU firmware or drivers to enable LoRA training?
A: Based on the published description, VKORE operates as a software layer above the hardware, implying that firmware modifications are not required. It abstracts the hardware interface rather than replacing it, which is consistent with its hardware-independent positioning.

Q: Can the LoRA adapters trained via VKORE on an NPU be used directly with a standard PyTorch or Hugging Face inference pipeline?
A: The reported 0.9999 numerical parity with PyTorch strongly suggests that adapter weights produced by VKORE are compatible with standard PyTorch-based workflows. However, VIDRAFT has not published explicit interoperability documentation, so teams should validate this in their own environment before relying on it in production.

Q: Does VKORE support only transformer architectures, or other model types as well?
A: The source article specifically references transformer models and the Gemma 4 26B benchmark. No claims about support for other architectures (CNNs, SSMs, etc.) are made in the published material.

Originally reported by TechGig(글로벌) (2026-10-14) — source article.

VIDRAFT Trains a Language Model from Scratch on NPUs — No GPUs Required

AI OpenFree — Thu, 15 Oct 2026 03:00:49 +0000

VIDRAFT Trains a Language Model from Scratch on NPUs — No GPUs Required

TL;DR: Korean Pre-AGI startup VIDRAFT has pre-trained a large language model using NPUs (Neural Processing Units) instead of GPUs, challenging the industry assumption that GPU clusters are the only viable path for LLM pre-training. This represents a notable infrastructure diversification signal for ML engineers exploring alternative accelerator stacks. Access details are not yet publicly confirmed in the source.

What it is

VIDRAFT (비드래프트), a Korean AI startup self-described as working toward Pre-AGI systems, has announced that it has completed pre-training of a language model using NPU hardware — without relying on GPUs at any stage of the pre-training pipeline.

Key facts from the source:

The training workload covered is pre-training (i.e., training a language model from scratch on large corpora), not fine-tuning or inference-only deployment.
The hardware used is classified as NPU (Neural Processing Unit), a category of AI accelerator distinct from NVIDIA or AMD GPU products.
The headline claim is explicitly that GPU infrastructure was not required for this pre-training run.
The announcement was reported by BigGo 파이낸스, a Korean financial/tech outlet, on 2026-10-14.

No additional details about model architecture, parameter count, training data, or specific NPU vendor are confirmed in the available source text.

How it works

At a conceptual level, the significance here is hardware-stack portability for LLM pre-training workloads.

Historically, large language model pre-training has been tightly coupled to GPU clusters — particularly because:

Deep learning frameworks (PyTorch, JAX) and their CUDA kernels have been co-optimized with NVIDIA hardware for over a decade.
Distributed training primitives (NCCL, tensor parallelism, pipeline parallelism) have mature GPU-specific implementations.
Most public LLM training recipes assume GPU memory hierarchies and interconnects (NVLink, InfiniBand).

Running pre-training on NPUs instead implies that VIDRAFT has addressed — at some level — the following engineering challenges:

Operator compatibility: Ensuring that the core tensor operations used in transformer pre-training (matrix multiplications, attention, layer norms, embedding lookups) are supported and performant on NPU hardware.
Distributed training coordination: Managing gradient synchronization and parameter updates across NPU devices, which may use different interconnect topologies than GPU clusters.
Framework integration: Either adapting an existing framework (or using a vendor-supplied SDK) to compile and schedule training graphs onto NPU compute units.
Numerical precision handling: NPUs may support different floating-point formats than GPUs; pre-training stability requires careful management of precision across forward and backward passes.

The fact that this was pre-training rather than inference is the technically meaningful claim — inference on NPUs is already commercially common (mobile SoCs, edge chips), but pre-training at scale on NPUs is a far less documented territory.

Benchmarks & results

The source article body was not available due to a rate-limit error at retrieval time ({"message":"Rate limit exceeded"}), so no specific benchmark numbers, loss curves, training throughput figures, or downstream task evaluations can be reported here.

Qualitatively, the announcement positions this as a proof-of-concept or milestone demonstrating that:

NPU hardware can sustain a full pre-training run to completion.
GPU independence is achievable at the pre-training stage, not just inference.

Engineers should treat specific performance comparisons (e.g., NPU vs. GPU throughput, tokens/second, cost-per-token) as unconfirmed until VIDRAFT publishes a technical report or blog post with reproducible numbers.

How to try it

Based on the available source information, no public access path has been confirmed. There is no mention of:

A Hugging Face model repository or organization page
A GitHub repository with training code
An OpenAI-compatible API endpoint
A public demo or waitlist

If VIDRAFT releases the model weights, training code, or API access, those channels will likely be announced through their official channels. Engineers interested in following developments can watch for VIDRAFT's official announcements directly.

FAQ

Q: Why does it matter if pre-training runs on NPUs instead of GPUs?
A: GPU supply constraints and cost are a real bottleneck for AI development globally, including in Korea and other regions with limited access to NVIDIA allocations. Demonstrating that NPU hardware can run pre-training opens up alternative procurement and infrastructure paths for organizations that cannot access large GPU clusters. It also signals growing maturity in NPU software stacks.

Q: Does "NPU pre-training" mean the results are comparable to GPU-trained models?
A: The available source does not provide benchmark comparisons between NPU-trained and GPU-trained models, so no equivalence claim can be made here. Hardware-agnostic training is a necessary but not sufficient condition for quality parity — actual model quality depends on data, scale, and optimization choices that remain unpublished at this time.

Q: Is VIDRAFT's approach applicable to open-source frameworks like PyTorch or JAX?
A: The source does not specify which training framework or software stack was used. This is a key technical detail that would be needed to assess reproducibility or community applicability, and it remains unconfirmed.

Originally reported by BigGo 파이낸스(한국) (2026-10-14) — source article.

Training an LLM from Scratch on Inference-Only NPUs — No GPU Required

AI OpenFree — Wed, 14 Oct 2026 23:01:17 +0000

Training an LLM from Scratch on Inference-Only NPUs — No GPU Required

TL;DR: VIDRAFT has successfully pre-trained a small language model entirely on inference-only NPUs, bypassing GPUs at every stage of training. The model — VKORE-46M-NPU — is publicly available on Hugging Face, along with comparative analysis materials. For ML engineers exploring alternative accelerator stacks, this is a concrete proof-of-concept that backpropagation on inference-dedicated silicon is achievable.

What it is

VKORE-46M-NPU is a 46-million-parameter small language model (SLM) pre-trained from scratch using only a first-generation, inference-only NPU — a class of chip that conventional ML wisdom treats as unsuitable for training workloads. The model was trained on a public dataset and is capable of generating short English-language stories.

The broader context: VIDRAFT is building VKORE, an NPU-native acceleration software layer intended to do for NPUs what CUDA does for NVIDIA GPUs. The target capability set includes:

Inference acceleration
Full pre-training (demonstrated here)
LoRA fine-tuning

This release is the public validation artifact for the pre-training milestone.

How it works

The core engineering challenge is that inference-only NPUs are architected to run forward passes efficiently — they are not designed to support the gradient computation and weight-update operations that backpropagation requires. Hardware-level features that GPU-based training takes for granted (flexible memory access patterns, high-precision intermediate buffers, in-place tensor mutation) are typically absent or restricted on inference NPUs.

VIDRAFT's approach, at a conceptual level:

Custom software abstraction over the NPU: Rather than relying on the NPU vendor's training stack (which typically doesn't exist for inference-only silicon), VIDRAFT implemented backpropagation through their own VKORE software layer, making the NPU behave as a trainable compute substrate.
From-scratch pre-training, no warm-start: The model was initialized with random weights — no pre-trained checkpoint was used as a starting point. This rules out the possibility that the result is simply transfer learning or fine-tuning dressed up as pre-training.
Parallel GPU reference run: To validate correctness, VIDRAFT ran an identical training job on a high-performance GPU under matched conditions and compared the resulting loss trajectories.

The precise implementation details — operator fusion strategy, memory management, gradient accumulation mechanisms — are not publicly disclosed and will be described in a forthcoming paper.

Benchmarks & results

VIDRAFT reports one key quantitative result from the comparison between the NPU-trained and GPU-trained runs:

Loss trajectory correlation coefficient: 0.99 between the NPU training run and an equivalent GPU training run under identical conditions.

This is a measure of training dynamics similarity, not throughput parity. The company explicitly notes this does not imply the NPU matches GPU training speed. What it does indicate is that the NPU-based training process converges along a statistically near-identical path to the GPU baseline — a meaningful signal that the gradient computation is numerically consistent, not approximate or divergent.

Additionally, VIDRAFT has verified that general-purpose public models can run inference and quantized-model execution on the same NPU hardware, suggesting the VKORE layer handles both the training and deployment sides of the lifecycle.

How to try it

The following artifacts are publicly available on Hugging Face:

Model weights for VKORE-46M-NPU
Model card with usage details
Comparative analysis materials (the GPU vs. NPU training comparison)

You can browse or download the model using the Hugging Face CLI:

pip install huggingface_hub
huggingface-cli download vidraft/VKORE-46M-NPU

Note: The model name above follows the naming convention used in the press release. Verify the exact repository slug on the VIDRAFT Hugging Face page before running commands.

The core training code is not public — it is withheld for IP protection. A technical paper describing the implementation is planned for future release. There is no mention of a public API endpoint for this specific model at this time.

FAQ

Q: Is this actually backpropagation, or some gradient-free approximation like forward-mode differentiation or evolution strategies?
A: The press release explicitly states that VIDRAFT implemented backpropagation (역전파) on the NPU. It is described as standard reverse-mode gradient computation, not a gradient-free alternative. The technical mechanism is proprietary and will be detailed in a future paper.

Q: Does the 0.99 loss correlation mean NPU training is as fast as GPU training?
A: No — VIDRAFT explicitly clarifies this point. The correlation coefficient measures similarity of the training trajectory (i.e., how the loss curve evolves over steps), not wall-clock throughput or sample efficiency. Inference-only NPUs are generally slower for training workloads; the point of this result is correctness validation, not speed benchmarking.

Q: When will VKORE support LoRA fine-tuning and broader model pre-training?
A: VIDRAFT has announced LoRA fine-tuning as a roadmap item for the VKORE software layer, alongside inference acceleration and pre-training. No public timeline or availability date has been announced.

Q: Can I use this to train on my own NPU hardware?
A: Not yet. The training stack is internal. The public release covers the trained model weights and analysis materials. Watch for the forthcoming technical paper and any future VKORE SDK announcements.

Originally reported by ZDNet코리아 (2026-10-14) — source article.

DEV Community: AI OpenFree

Why Long bf16 Training Runs Stall: The Residual Stream Has No Room for Small Updates

TL;DR

Why does a bf16 residual stream lose small updates?

Are residual values really that large?

What does the failure look like in a simulation?

How do production training stacks handle this?

What if you cannot keep the residual in fp32?

How do you catch this before the run is lost?

FAQ

Further reading

A Decision Model That Writes Nothing Just Took #1 on a 137-Benchmark Board

The result first

What Borda aggregation actually measures

The model generates no text

Calibration is the property that matters

What cheap checking changes

Serving it

How to check this claim

Darwin-180B-RSI Tops MMLU-Pro and GPQA Diamond on AIhoster's Independent Open-Model Benchmark Board

Darwin-180B-RSI Tops MMLU-Pro and GPQA Diamond on AIhoster's Independent Open-Model Benchmark Board

What it is

How it works

Benchmarks & results

How to try it

FAQ

Five Open Science Challenges Where a Verifier, Not the Submitter, Decides the Score

TL;DR

Why a hidden verifier matters

Built for agents as well as people

Repositories

Six Open-Source Projects from VIDRAFT, Now on GitHub

1. ONGRID: GraphRAG that also decides

2. FINAL Bench: does the model know when it is wrong?

3. Open Quantum Challenge harness: the yardstick is public

4. POCKET: strong models on your own laptop

5. ZTC: can you trust this answer?

6. ALL Bench Leaderboard: one place to compare

Where to start

VIDRAFT's VKORE Enables Full Model Pre-Training on Inference-Only NPUs — 0.99 Loss Correlation with GPU Baselines

VIDRAFT's VKORE Enables Full Model Pre-Training on Inference-Only NPUs — 0.99 Loss Correlation with GPU Baselines

What it is

How it works

Benchmarks & results

How to try it

FAQ

Darwin-27B-ZTC-v5 Tops the Typed Decisions Leaderboard with a 0.81 Accuracy Score

Darwin-27B-ZTC-v5 Tops the Typed Decisions Leaderboard with a 0.81 Accuracy Score

What it is

How it works

Benchmarks & results

How to try it

FAQ

VIDRAFT Trains a 46M-Parameter Model Entirely on Inference-Only NPUs — No CUDA Required

VIDRAFT Trains a 46M-Parameter Model Entirely on Inference-Only NPUs — No CUDA Required

What it is

How it works

Benchmarks & results

How to try it

FAQ

Who Holds #1 on Every Hugging Face Official Benchmark? A Live Index

What the index shows

How it is built

Why it matters

VKORE: LoRA Training on Inference-Only NPUs with 3.8 Inference Speedup and 0.9999 PyTorch Parity

VKORE: LoRA Training on Inference-Only NPUs with 3.8× Inference Speedup and 0.9999 PyTorch Parity

What it is

How it works

Benchmarks & results

How to try it

FAQ

VIDRAFT Trains a Language Model from Scratch on NPUs — No GPUs Required

VIDRAFT Trains a Language Model from Scratch on NPUs — No GPUs Required

What it is

How it works

Benchmarks & results

How to try it

FAQ

Training an LLM from Scratch on Inference-Only NPUs — No GPU Required

Training an LLM from Scratch on Inference-Only NPUs — No GPU Required