DEV Community: AI OpenFree

Which Weather Model Is Most Accurate? We Score Them Every Day, in the Open

AI OpenFree — Fri, 13 Nov 2026 10:33:11 +0000

TL;DR

VIDRAFT opened two things on Hugging Face: Darwin-Prediction-Weather, a weather prediction system, and the Global Weather Model Leaderboard, a public board that scores the world's forecast models against what actually happened, every day. In the 2026 record, Darwin-Prediction-Weather has the lowest error in all four scored elements at every lead time from 1 to 7 days. The full write-up is on the Hugging Face blog.

The question: AI or physics?

For decades, forecasts came from physics models that solve the equations of the atmosphere on supercomputers. In the last three years, AI models trained on historical weather have joined them at the world's weather centres. Which one should you trust for tomorrow, for three days ahead, for a week ahead? Published papers answer that for past years. This board answers it for this year, updated daily, with the data open for anyone to check.

How the board scores

Item	Rule
Models	ECMWF IFS and AIFS, NOAA GFS and AIGFS, DWD ICON, JMA GSM, ECCC GEM, MET Norway, Darwin-Prediction-Weather
Elements	Daily maximum temperature, daily minimum temperature, daily maximum wind speed, daily precipitation total
Where	24 cities on six continents, up to 10 days ahead
Truth	ERA5 reanalysis, Copernicus Climate Change Service
Score	Root-mean-square error, lower is better
Fairness	Every model is scored on the same cases: days and cities where all listed models issued a forecast
Record	From 1 January 2026, updated once a day

What the 2026 record shows

Two findings stand out.

AI models now lead. Among the models of the national and international weather centres, ECMWF's AI model AIFS has lower error than ECMWF's physics model IFS at every lead time, and NOAA's AIGFS has lower error than NOAA's GFS at every lead time.

Fusion goes further. Darwin-Prediction-Weather has the lowest error in all four elements, at every lead time from 1 to 7 days.

RMSE against ERA5, 24 cities, 2026 year to date (verified through 2026-11-07). The value in brackets is the next model, ECMWF AIFS in every cell.

Element	1 day	3 days	5 days	7 days
Daily maximum temperature [°C]	0.84 (1.35)	1.17 (1.59)	1.59 (1.90)	2.09 (2.41)
Daily minimum temperature [°C]	0.91 (1.50)	1.09 (1.63)	1.42 (1.85)	1.81 (2.17)
Daily maximum wind speed [km/h]	2.25 (3.60)	2.70 (4.07)	3.43 (4.76)	4.39 (5.69)
Daily precipitation total [mm]	4.15 (4.33)	5.18 (5.24)	6.80 (6.99)	7.11 (7.30)

Who wins where

A single global number hides a lot. The board breaks the record down by period, by lead time and by city. For every city it also lays out a rolling timeline: the last seven days with the actual value and the model that came closest, today so far, and the forecast for the next ten days. Each morning yesterday's forecast becomes today's result, and it is scored.

How Darwin-Prediction-Weather works

Darwin is VIDRAFT's family of models built on one idea: combine the strengths of several strong models into one that is stronger than any of them. Darwin-Prediction-Weather applies that idea to the atmosphere.

Forecast engine. A single forecast from skill-weighted fusion of global forecasts, with local correction for temperature and wind. It learns only from days whose observed value was already known before the forecast it is scored on. Nothing from the future enters.
Decision core. A Darwin-27B model with a calibrated readout turns forecast fields into decisions: go or no-go, threshold exceedance, category probabilities.

The same engine was evaluated on WeatherBench 2, the standard benchmark for AI weather models run by Google Research. On the 2020 deterministic protocol it records the lowest error in all 15 evaluated cells: 2 m temperature, 500 hPa geopotential and 850 hPa temperature at 1, 3, 5, 7 and 10 days. The forecast files are public and the entry has been submitted to the WeatherBench 2 team.

WeatherBench 2, 2020	1 day	3 days	5 days	7 days	10 days
2 m temperature [K]	0.468	0.743	1.107	1.531	1.984
best on official scorecard	0.488	0.786	1.125	1.552	2.022
Z500 [m2/s2]	33.7	108.5	240.6	400.3	595.4
best on official scorecard	36.8	113.8	250.7	415.1	606.5
T850 [K]	0.503	0.901	1.406	2.000	2.699
best on official scorecard	0.518	0.922	1.435	2.041	2.737

Everything is open to check

The board: https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard
The daily record of every forecast and observed value since 2026-01-01, as CSV: https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive
The model: https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather
WeatherBench 2 forecast files: https://huggingface.co/datasets/VIDraft/Darwin-Prediction-Weather-WB2
Full article: https://huggingface.co/blog/FINAL-Bench/which-weather-model-is-most-accurate-we-score-them

What comes next

Forecasts 8 to 10 days ahead enter the ranking as their target days are verified. Scoring against station observations will run alongside ERA5. More cities and more elements follow, and a forecasting game on Hugging Face will let anyone test their own forecast against the models.

License and sources

The leaderboard application is released under the Apache License 2.0. The compiled daily record and the Darwin-Prediction-Weather model are released under CC BY-NC 4.0, non-commercial use only. For commercial use, contact vidraft@mail.vidraft.net.

Weather data by Open-Meteo.com (CC BY 4.0). Verification: ERA5, Copernicus Climate Change Service. Model sources: ECMWF, NOAA NCEP, DWD, JMA, ECCC and MET Norway. Satellite imagery: NASA Global Imagery Browse Services (GIBS).

How a 180B Open Model Outscored a 2.8T Model in Math and Science Reasoning: Strength Merging Plus Model-Level RSI

AI OpenFree — Fri, 13 Nov 2026 01:04:20 +0000

TL;DR

Darwin-180B-RSI, an open-weight model from Korea's VIDRAFT, scored 73.68 in mathematical and scientific reasoning on SuperCLUE's November 2026 evaluation. That is first among open-weight models and second overall, directly behind OpenAI's GPT-6.1-Sol (76.32). The largest model in the same evaluation, Moonshot AI's Kimi-K3.1-Pro at 2.8 trillion parameters, scored 63.16. Darwin has 180 billion parameters, about one sixteenth of that size, and finished more than 10 points ahead.

This post explains the two techniques behind the result and what the size difference means for serving cost.

The full ranking

Why SuperCLUE counts as third-party verification

SuperCLUE does not accept scores submitted by model developers. The evaluator calls each model directly and grades it on a private test set that changes every month. SuperCLUE, LMArena, Artificial Analysis and Stanford HELM are the four evaluators commonly cited for this style of independent testing.

The November evaluation covers five tracks: agentic coding, math and science reasoning, hallucination control, precise instruction following, and long context. On the combined index Darwin ranks fifth among open-weight models and ninth overall out of 19 models.

Technique 1: merge the strengths, not the whole model

Darwin is also the name of VIDRAFT's model creation method. It selects two or more public models as parents, analyzes the feed-forward (FFN) regions where each parent stores its knowledge, extracts the strengths specific to each one, and synthesizes them into a single child model.

Evolutionary model merging searches over merge combinations and blends the parents. Darwin first identifies which region holds which strength and then transplants selectively. The result is a new model built from chosen strengths, and in math and science reasoning it scores more than 15 points above the comparable public base model.

Technique 2: model-level recursive self-improvement

The synthesized child then goes through model-level RSI. No human-written solutions or reasoning traces are used. The model writes its own solutions, an automatic verifier checks each answer, and the model trains on the solutions that pass. The weights themselves improve their reasoning ability, round after round, with no person in the loop.

What one sixteenth the size buys you

Metric	Value
Server GPU inference speed	18,057 tokens per second
Concurrent request throughput	2.6x a standard 4-bit build on the same hardware
NPU inference speed	417 tokens per second

A smaller model serves the same workload on fewer machines, which lowers the cost per token. Darwin also runs on Korean-made NPUs such as Rebellions ATOM-Max through VIDRAFT's VKORE core technology, and the POCKET edition runs on a single laptop as on-device AI.

Public records

On the official Hugging Face benchmark leaderboards, the Darwin family currently holds first place on 12 boards. These include GPQA at 94.44, and perfect scores on AIME 2026 and HMMT.

Adoption follows the same curve. VIDRAFT's open models have about 5 million cumulative downloads on Hugging Face and more than 1.2 million per month. The POCKET family alone recorded 1,194,702 downloads in the 30 days to November 13, 2026, with POCKET-Darwin-180B-GGUF at 525,554.

Safety and the rest of the stack

ViKernel is VIDRAFT's weight-native operating system: rules inside the model weights judge every action the AI is about to take, block dangerous ones, and keep a kill switch available to a human at all times. AETHER is a model trained from scratch and released under Apache 2.0.

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

AI OpenFree — Wed, 11 Nov 2026 23:01:37 +0000

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

TL;DR: VIDRAFT has open-demoed ViKernel, an "LLM OS" that embeds computation circuits and security controls directly into model weights — not as an external wrapper. It enables verifiable arithmetic inside the model and ships a hardware kill switch that requires a signed operator key to restart. If you care about auditable, controllable AI systems, this architecture is worth watching.

What it is

ViKernel is VIDRAFT's implementation of what they call a Weight-Native OS (운영체제) — a system where the operating-system primitives (CPU-equivalent logic, NPU-equivalent logic, RAM/ROM-equivalent memory circuits) are encoded directly into LLM weights rather than bolted on as external middleware or guardrail layers.

Key distinguishing properties:

In-weight computation: Arithmetic circuits live inside the model's weights and are executed on GPU/NPU hardware at inference time.
In-weight security controls: Integrity checking, policy enforcement, audit logging, and a kill switch are implemented as weight-level logic, not as a separate orchestration layer.
Portability across model families: VIDRAFT reports having transplanted the circuit logic into models beyond their own — including weight-compatible implementations targeting the Gemma (Google) and Qwen (Alibaba) model families.
Signed restart requirement: Once the kill switch halts execution, the model cannot resume without a cryptographically signed operator key — making accidental or unauthorized resumption structurally difficult.

VIDRAFT is a Seoul AI Hub resident startup founded by CEO Minsik Kim, and also participates in the K-MYTHOS national security-focused AI development consortium led by NAVER Cloud.

How it works

The conceptual architecture maps familiar OS abstractions onto neural network weights:

OS Concept	ViKernel Equivalent
CPU	Computation circuit encoded in weights
NPU	Accelerated inference circuit in weights
RAM / ROM	In-weight memory structures
Kernel policy engine	Weight-level task allow/deny logic
Audit log	Tamper-resistant in-weight ledger
Kill switch	Halts execution; requires signed key to restart

At inference time, when the LLM receives a task requiring computation (e.g., multi-digit arithmetic), it delegates to the internal circuit rather than attempting to generate the answer autoregressively token-by-token. The circuit executes on the underlying GPU/NPU and returns a verified result that the model incorporates into its response.

For security, ViKernel performs an integrity check at startup. If the model weights have been modified, the system refuses to run. During operation, every policy decision (allow or block a task) is written to an audit log designed to resist tampering. If a kill switch signal is issued, execution halts and cannot be re-initiated without a valid signed operator credential.

VIDRAFT plans to integrate ViKernel's internal control plane with AX-RAY, their external AI risk diagnostics tool, creating a combined pipeline where external vulnerability scanning feeds into in-model behavioral controls.

Benchmarks & results

VIDRAFT has publicly reported one concrete result:

On their 27-billion-parameter proprietary model, ViKernel's embedded arithmetic circuit correctly multiplied two 300-digit numbers, producing an approximately 600-digit result. The result was verified against an independent calculator.

No benchmark suite scores, latency numbers, or comparative accuracy figures against baseline LLMs are cited in the source coverage at this time. Qualitatively, the claim is that offloading computation to in-weight circuits produces exact results for tasks where autoregressive generation is inherently error-prone (long-form arithmetic being the canonical example).

How to try it

A demo is publicly available on Hugging Face. It lets you interact with:

The kill switch mechanism
The policy engine (task allow/deny)
PII masking
Audit log output

⚠️ Important caveat: The Hugging Face demo uses pre-recorded responses generated by the actual ViKernel OS and plays them back — it does not run live inference. VIDRAFT acknowledges this explicitly.

No public model weights, GitHub repository, pip package, or OpenAI-compatible API endpoint has been announced as of the source date (2026-11-11). To find the demo, search for VIDRAFT or ViKernel on huggingface.co.

FAQ

Q: Why embed OS logic in weights rather than using an external orchestration layer like a tool-calling framework?
A: External guardrails can be bypassed, stripped, or fail silently if the model is re-deployed without them. By encoding control logic into the weights themselves, ViKernel makes the security posture inseparable from the model artifact. Modifying weights to remove the controls would trigger the integrity check and halt execution.

Q: Can ViKernel circuits be transplanted into any open-weight model?
A: VIDRAFT reports successful transplantation into models from the Gemma and Qwen families in addition to their own model. However, no general tooling or methodology for doing this yourself is publicly available yet — it appears to be a research capability rather than a released SDK.

Q: Is the kill switch a software flag or something deeper?
A: Based on the source, it is implemented at the weight level as part of the in-model OS. Crucially, resuming a halted model requires a signed operator key, implying a cryptographic verification step — not simply a configuration toggle that can be flipped in a prompt or environment variable.

Q: What is the relationship to K-MYTHOS?
A: K-MYTHOS is a South Korean government-backed consortium for security-specialized AI development, led by NAVER Cloud. VIDRAFT's role within it is AI safety diagnostics and evaluation using their AX-RAY technology, with plans to connect AX-RAY's external risk scanning to ViKernel's internal enforcement layer.

Originally reported by IT조선 (2026-11-11) — source article.

Metacognition Leaderboard for LLMs: JGENai VIDRAFT Launch a Benchmark That Measures Self-Error Detection, Not Just Accuracy

AI OpenFree — Tue, 10 Nov 2026 23:00:42 +0000

Metacognition Leaderboard for LLMs: JGENai × VIDRAFT Launch a Benchmark That Measures Self-Error Detection, Not Just Accuracy

TL;DR: JGENai and VIDRAFT have jointly released a metacognition benchmark, a 24-model public leaderboard, and 11 model-specific metacognition adapters designed to measure whether an AI can detect its own mistakes — not just whether it gets answers right. The adapters attach to existing models without modifying weights or architecture, making them drop-in reliability probes for high-stakes applications. Developers can explore the leaderboard, benchmark dataset, and adapters on Hugging Face.

What it is

JGENai (CEO Sunyoung Choi) and VIDRAFT have co-developed and publicly released three artifacts:

A metacognition benchmark — 400 adversarial "trap" questions designed so the superficially plausible answer is wrong, testing whether a model can catch its own likely errors rather than just pattern-match to a confident-sounding output.
A 24-model public leaderboard — ranking current Korean and multilingual LLMs on both multiple-choice accuracy and open-ended self-error-detection capability.
11 model-specific metacognition adapters — lightweight add-ons that sit beside a target model and assess the risk that any given response is incorrect, without touching the base model's weights.

The core thesis: the dangerous moment for an AI is not when it's wrong, but when it doesn't know it's wrong. In domains like medicine, law, finance, security, and scientific research — where error costs are high — self-error detection is framed as a primary reliability signal, not an afterthought.

How it works

Benchmark design

Each of the 400 benchmark problems is constructed so that one answer appears strongly plausible at first read, while the actual correct answer requires the model to pause and re-examine its own reasoning. Evaluation is split across two axes:

Multiple-choice accuracy — standard answer correctness on trap questions.
Open-ended self-assessment — after generating a free-form answer, can the model correctly judge whether that answer is likely wrong?

This dual-axis design exposes a gap that single-accuracy leaderboards miss: a model can score well on axis 1 while being nearly blind on axis 2.

Metacognition adapters

The adapters are designed around a non-invasive attachment pattern:

They do not modify the base model's architecture or trained weights.
They analyze internal signals produced during the model's answer-generation process to compute a risk score indicating how likely the response is to be incorrect.
The output is a separate risk signal — a flag or score that downstream systems, human reviewers, or orchestration layers can consume independently of the answer itself.

This architecture means you can wrap an existing deployed model with a metacognition adapter without retraining, fine-tuning, or touching inference infrastructure beyond adding the adapter layer.

Benchmarks & results

From the publicly reported leaderboard results:

JGOS-31B-Citizen ranked #1 out of 24 models evaluated. It answered 398 out of 400 trap questions correctly, achieving a 99.5% trap-avoidance rate. JGOS-31B-Citizen is also the current top-ranked model on the K-AI Leaderboard operated by Korea's Ministry of Science and ICT (MSIT) and the National Information Society Agency (NIA).
Multiple models from the Darwin family (Darwin-28B, Darwin-31B, Darwin-35B) — all developed in Korea — placed in the upper tier of the leaderboard.
Critically, models that performed strongly on multiple-choice accuracy still showed limited capability on open-ended self-assessment. This confirms that raw accuracy and metacognitive awareness are empirically distinct competencies — a model can be highly accurate yet unable to flag its own uncertain or incorrect outputs.

How to try it

The benchmark dataset, the 24-model leaderboard, and all 11 metacognition adapters are publicly available on Hugging Face. The source article does not specify exact repository slugs, so navigate directly to the JGENai organization page on Hugging Face to find the released artifacts.

# Browse available artifacts via the Hugging Face CLI
huggingface-cli scan-cache

# Or search the JGENai org directly in your browser:
# https://huggingface.co/JGENai

No GitHub repository URL, API endpoint, or additional access mechanism was mentioned in the source article at time of publication. Check the Hugging Face model cards for adapter usage instructions specific to each of the 11 supported models.

FAQ

Q: Does applying a metacognition adapter require retraining or fine-tuning my existing model?
A: No. The adapters are explicitly designed to leave the base model's weights and architecture unchanged. They attach externally and analyze generation-time signals to produce a separate risk score.

Q: Is the benchmark only relevant for Korean-language models?
A: The benchmark was developed with a Korean-language focus — the top-performing models are Korean LLMs — but the conceptual framework (trap questions + dual-axis evaluation) is language-agnostic. Whether English or multilingual models are included in the current 24-model leaderboard is not specified in the source article.

Q: What does the risk signal from the adapter actually look like in practice?
A: The source describes it as a computed probability that a given response is incorrect, derived from internal signals during generation. Exact output format (scalar score, binary flag, confidence interval) is not specified in the public reporting — check the Hugging Face model cards for implementation details.

Q: Why does it matter that accuracy and self-error detection are separate skills?
A: Because a high-accuracy model that cannot flag its own mistakes will confidently surface wrong answers in production with no warning. In regulated or safety-critical domains, that silent failure mode is often more dangerous than a lower-accuracy model that correctly signals uncertainty.

Originally reported by AI이코노미뉴스 (2026-07-02) — source article.

Darwin-180B-RSI-R3: Top Open-Weight Model on SuperCLUE November 2026, Outscoring Gemini and DeepSeek in Math & Science

AI OpenFree — Tue, 10 Nov 2026 07:02:03 +0000

Darwin-180B-RSI-R3: Top Open-Weight Model on SuperCLUE November 2026, Outscoring Gemini and DeepSeek in Math & Science

TL;DR: VIDRAFT's open-weight model Darwin-180B-RSI-R3 scored 73.68 in math and science on the November 2026 SuperCLUE benchmark, #1 among all open-weight models and #2 overall out of 19 models. Built on top of Qwen3.8-Flash-Next and fine-tuned via a Recursive Self-Improvement (RSI) approach, it's publicly available on Hugging Face with a 4-bit quantized variant that fits on consumer hardware.

What It Is

Darwin-180B-RSI-R3 is an open-weight large language model developed by VIDRAFT, a Korean Pre-AGI AI startup. It is built on top of the Qwen3.8-Flash-Next base model and refined using VIDRAFT's proprietary RSI (Recursive Self-Improvement) training method.

Key facts from the source:

Benchmark placement: 2nd place overall out of 19 models on the November 2026 SuperCLUE math and science leaderboard, with a score of 73.68. The only model ahead of it is GPT-6.1-Sol at 76.32. Gemini-4-Argon and DeepSeek-V4.1-Flash both scored 71.05.
Open-weight leaderboard: #1 among all open-weight models on the same math and science axis, the only Korean model in the table.
Overall SuperCLUE score: 43.08, placing it 9th out of 19 in the general ranking.
License: Qwen Community 1.0
Repository: FINAL-Bench/Darwin-180B-RSI on Hugging Face

How It Works

At a conceptual level, RSI is an iterative self-refinement training loop:

Solve: The model generates solutions to a problem set.
Verify: Answers are checked against ground-truth answer keys.
Filter & fine-tune: Only the reasoning traces that led to correct answers are retained; the model is then trained on this filtered, high-quality signal.
Repeat: The loop runs again with the updated model.

A key design choice in this process is surgical weight updates: only the attention paths and shared experts are modified during RSI fine-tuning. The full set of 512 experts, the router, and the visual encoder from the base model are left untouched. This preserves the base model's general capabilities while sharpening its mathematical reasoning.

The practical effect on inference efficiency is measurable: on MMLU-Pro, average reasoning chain length dropped from 4,320 tokens to 3,833 tokens, an approximately 11% reduction, meaning the model reaches correct answers with shorter internal chains of thought after RSI training.

Benchmarks & Results

All numbers below come from the model card and the SuperCLUE November 2026 leaderboard.

Benchmark	Darwin-180B-RSI-R3	Base model (Qwen3.8-Flash-Next on same table)
SuperCLUE Math & Science	73.68 (2nd / 19 overall, 1st open-weight)	57.89
SuperCLUE Overall	43.08 (9th / 19)	39.81
GPQA Diamond	94.44	-
MMLU-Pro	88.12	-
AIME 2026	100	-
HMMT February 2026	100	-

For context, Kimi-K3.1-Pro scored 63.16 in the same math and science category.

How to Try It

The model weights are publicly released on Hugging Face under the repository FINAL-Bench/Darwin-180B-RSI.

You can download the full model or the quantized variant using the Hugging Face CLI:

pip install huggingface_hub

# Full model
huggingface-cli download FINAL-Bench/Darwin-180B-RSI

# 4-bit quantized GGUF variant (111 GB)
huggingface-cli download FINAL-Bench/POCKET-Darwin-180B-GGUF

Hardware requirements for the quantized GGUF build (POCKET-Darwin-180B-GGUF):

Compressed size: 111 GB
Minimum spec cited: 8 GB GPU VRAM + 32 GB system RAM
CPU-only inference: reported at 18–21 tokens/second on a single CPU (no GPU required)

This makes Darwin-180B-RSI-R3 one of the more accessible 180B-class models for local experimentation, particularly for researchers without data-center GPU access.

FAQ

Q: What makes RSI different from standard supervised fine-tuning on math datasets?
A: Standard SFT trains on a fixed curated dataset. RSI uses the model itself to generate candidate solutions, filters for only the correct reasoning traces, trains on those, then repeats, meaning the training signal improves iteratively as the model improves, without requiring a pre-curated chain-of-thought dataset.

Q: Why are only attention paths and shared experts updated, rather than the full model?
A: According to VIDRAFT, this surgical update strategy preserves the generalist capabilities of the base model (including its visual encoder and expert routing) while concentrating improvements in the components most responsible for sequential reasoning. It also reduces the risk of catastrophic forgetting.

Q: Where does Darwin-180B-RSI-R3 stand on the overall SuperCLUE board?
A: It placed 9th of 19 models overall and 5th among open-weight models, six places above its parent Qwen3.8-Flash (15th, 39.81). Its lead axis is math and science reasoning: #1 among open-weight models and #2 overall.

Q: Who measured these scores?
A: The SuperCLUE scores come from SuperCLUE's own private test set, refreshed every month and administered by SuperCLUE. GPQA Diamond 94.44 is the #1 entry on the Hugging Face official GPQA leaderboard.

Originally reported by USB.RU (러시아) (2026-11-09), source article.

Korean AI Startup VIDRAFT Hits #1 on Google's Fast Gemma Challenge with 6 Inference Speedup

AI OpenFree — Tue, 10 Nov 2026 03:01:24 +0000

Korean AI Startup VIDRAFT Hits #1 on Google's Fast Gemma Challenge with 6× Inference Speedup

TL;DR: Seoul-based Pre-AGI startup VIDRAFT topped the global leaderboard of Google and Hugging Face's Fast Gemma Challenge, achieving 510.58 tokens per second on the shared benchmark model — roughly 6× faster than the baseline — while maintaining output quality. The result was independently verified by the contest organizers, making it directly relevant to any engineer thinking about inference optimization and cost efficiency at scale.

What it is

The Fast Gemma Challenge is a competitive inference-optimization benchmark co-organized by Google's Gemma team and Hugging Face. Every participant receives:

The same public multimodal model (gemma-4-E4B-it)
The same hardware (a standardized GPU instance provided by the organizers)
A mandate to improve software-only throughput — swapping the model, disabling modalities (text, image, audio), or any other architectural shortcut is explicitly prohibited

Rankings are determined exclusively by VERIFIED scores: the organizers re-run each submission against a private prompt set and independently measure both speed and output quality. Self-reported numbers are not accepted. This design makes the leaderboard resistant to gaming or cherry-picked benchmarks.

VIDRAFT's autonomous AI agent vidraft-darwin claimed the #1 verified position on this leaderboard.

How it works

At a conceptual level, VIDRAFT's result reflects a pure software optimization approach applied on top of an unchanged model checkpoint. The organizers' rules forced competitors to extract performance gains entirely through the software stack — covering areas such as kernel-level execution, memory layout, batching strategies, and scheduling — rather than through model compression or capability removal.

VIDRAFT applied optimization techniques drawn from two of its internal technology pillars:

VK-series inference engine — VIDRAFT's proprietary inference runtime, developed in-house to accelerate large model serving.
POCKET platform — VIDRAFT's on-device AI platform, designed to run large models in resource-constrained environments without a dedicated GPU. The engineering discipline required for on-device efficiency appears to have informed the team's approach to squeezing throughput from a fixed hardware budget.

The key insight the team emphasizes is that inference optimization is not a trade-off between speed and quality — the verified result explicitly requires both. The 510.58 TPS figure was accepted only after the organizers confirmed that output perplexity remained within acceptable bounds (PPL 2.3929).

Benchmarks & results

All numbers below come directly from the verified leaderboard record and the press announcement:

Metric	Value
Throughput (VERIFIED)	510.58 tokens per second (TPS)
Perplexity (PPL)	2.3929
Speedup vs. baseline	~6× over the original model's default inference speed
Leaderboard position	#1 globally (verified submissions)

The practical implication the company highlights: a 6× throughput increase means you could serve the same workload with approximately one-sixth the GPU capacity compared to an unoptimized baseline — a direct reduction in inference infrastructure cost.

The verified results are publicly visible on the Fast Gemma Challenge official leaderboard hosted on Hugging Face.

How to try it

The source article does not announce a publicly released model checkpoint, open-source repository, or API endpoint associated with this specific challenge submission. If you want to explore VIDRAFT's public work:

Leaderboard: Check the Fast Gemma Challenge leaderboard on Hugging Face Spaces to see the verified submission details firsthand.
VIDRAFT on Hugging Face: Search for vidraft on huggingface.co to find any publicly released models or demos from the team (including their Darwin and AETHER model families mentioned in company materials).
Official channels: Monitor VIDRAFT's announcements for any planned open release of the VK inference engine or POCKET platform.

No public pip install, API endpoint, or model download command is confirmed in this release — do not use any that are not officially published by VIDRAFT.

FAQ

Q: Was the speedup achieved by quantizing the model or stripping out modalities?
A: No. The Fast Gemma Challenge rules explicitly prohibit replacing the model or disabling any of its capabilities (text, image, audio). The 6× speedup is a software-stack-only result on the unmodified gemma-4-E4B-it checkpoint.

Q: How trustworthy is the #1 ranking — could participants inflate their scores?
A: The contest uses a VERIFIED scoring system: the organizers re-run every submission on a private prompt set using their own infrastructure. Only these independently reproduced scores count toward the leaderboard. Self-reported or locally measured numbers are not accepted, so score inflation is structurally prevented.

Q: What hardware did VIDRAFT use, and can I replicate it?
A: All participants were given the same standardized GPU instance by the organizers. Because the optimization is software-only and hardware is held constant across all competitors, the gains are attributable entirely to VIDRAFT's software stack — not access to exotic hardware.

Q: Does this result apply to other models beyond Gemma?
A: The challenge was scoped to gemma-4-E4B-it. Whether the underlying techniques generalize to other model families is not addressed in this announcement, though VIDRAFT's VK inference engine and POCKET platform are described as applying to large models broadly.

Originally reported by AI이코노미뉴스 (2026-08-04) — source article.

VIDRAFT's Darwin-180B-RSI Tops Legal Reasoning Benchmarks LEXam and LEXam-hard — Without Legal Training Data

AI OpenFree — Mon, 09 Nov 2026 23:01:39 +0000

VIDRAFT's Darwin-180B-RSI Tops Legal Reasoning Benchmarks LEXam and LEXam-hard — Without Legal Training Data

TL;DR: VIDRAFT's Darwin-180B-RSI reasoning model scored 1st place on both LEXam (68.94) and LEXam-hard (45.72), two Hugging Face-certified legal reasoning benchmarks, despite never being trained on legal data. The model achieves this through Recursive Self-Improvement (RSI) on math and science problems — and the resulting reasoning generalizes across domains. This is now VIDRAFT's 7th #1 ranking across Hugging Face-certified leaderboards.

What it is

Darwin-180B-RSI is a 180-billion-parameter reasoning model developed by VIDRAFT (비드래프트), a Korean deep-tech AI startup. It is built on two compounding technologies:

Darwin — VIDRAFT's model evolution framework that constructs new models by diagnosing the strengths and weaknesses of existing model layers and expert modules, then selectively combining high-performing components from multiple source models. This avoids full pretraining from scratch.
RSI (Recursive Self-Improvement) — A training approach in which the model solves problems autonomously, automatically verifies its own solutions (on problems with deterministic answers, such as math and science), learns from confirmed correct solutions, and repeats the cycle with the improved version of itself. No human-written solution annotations are used.

The result is a model that has accumulated strong general reasoning capabilities — without domain-specific legal fine-tuning — that transfer to fields like law.

The Darwin family now spans ~50 official Hugging Face models and ~400 community-derived variants. The core methodology is documented in the Darwin Family paper (arXiv: 2605.14386).

How it works

At a high level, Darwin-180B-RSI is built in two stages:

Model evolution (Darwin): Instead of pretraining a large model from scratch, Darwin analyzes existing open models at the layer and expert level, identifies which components perform best on which tasks, and assembles a new model by combining those components. This is a diagnostics-driven merging process, scaled up to models with hundreds of billions of parameters.
Recursive Self-Improvement (RSI): The evolved model is then trained using a self-bootstrapping loop — it attempts to solve problems, uses automatic verification (possible only for problems with checkable answers, like math), and trains on the solutions it verified as correct. The updated model then becomes the starting point for the next iteration. This is explicitly a model-weight-level improvement: the learned reasoning is baked into the model weights themselves, requiring no external scaffolding or tool calls at inference time.

VIDRAFT distinguishes this from prompt- or workflow-level self-improvement approaches (such as modifying prompts, tools, or task graphs at runtime), noting the two paradigms are complementary rather than competing.

Additionally, Darwin-180B-RSI incorporates ZTC (Zero-Token Confidence), a technique that estimates answer confidence by inspecting internal model states before generating output — used to guide the self-improvement verification process.

Only approximately 0.02% of total parameters were modified during the RSI stage; the 512 knowledge-carrying expert modules, routers, and vision components were kept intact.

Benchmarks & results

All scores below are from Hugging Face-certified leaderboards:

Benchmark	Darwin-180B-RSI	Previous #1	Notes
LEXam (MCQ, 1,655 Qs)	68.94	DeepSeek-R1: 52.41	4-attempt majority vote
LEXam-hard (open-ended, 518 Qs)	45.72	Inkling: 40.82	Scored by benchmark's official grader model
LEXam single-attempt (no majority vote)	60.54	—	Same model, 1 inference pass

Context on LEXam difficulty: Random-choice baseline is 25 points (4-choice MCQ). Most prior models scored around 50. LEXam-hard is composed specifically of items where existing models struggled, with the previous top score in the low 40s.

Other models on LEXam (from benchmark developers' separate release): GPT-5: 62.65 · Claude-4.5-Sonnet: 58.01 · Gemini-2.5-Pro: 55.72 · Qwen3-235B-Thinking: 48.19 · gpt-oss-120b: 47.71.

Full #1 leaderboard record for Darwin-180B-RSI across 7 Hugging Face-certified benchmarks:

AIME 2026 — 100
HMMT 2026 — 100
GPQA Diamond — 94.44
MMLU-Pro — 88.12
MMMU-Pro — 79.48
LEXam — 68.94
LEXam-hard — 45.72

VIDRAFT also reports that reasoning chain length decreased by ~11% compared to the parent model while maintaining equivalent accuracy.

How to try it

VIDRAFT states that Darwin-180B-RSI and its full evaluation conditions are publicly available on Hugging Face for independent verification. You can explore the model and the Darwin family at huggingface.co by searching for VIDRAFT or Darwin-180B-RSI. The core methodology paper is publicly accessible:

arXiv: 2605.14386  (Darwin Family)

No additional API endpoint or SDK installation has been announced publicly at this time. Check VIDRAFT's official Hugging Face organization page for the latest model cards and access instructions.

FAQ

Q: Darwin-180B-RSI is 180B parameters — how does it beat DeepSeek-R1 (685B) on LEXam?
A: Parameter count is not the only determinant of benchmark performance. Darwin-180B-RSI uses a diagnostics-driven model merging approach (Darwin) combined with recursive self-improvement that strengthens generalizable reasoning. The model was not trained on legal data at all; its strong performance on LEXam is attributed to domain-general reasoning capabilities built up through RSI on math and science problems.

Q: What exactly is LEXam, and why does it matter for real-world legal AI?
A: LEXam was developed by researchers at ETH Zurich, the University of Zurich, and the Max Planck Institute. It draws from 340 actual final exam sessions from Swiss university law faculties. Critically, it tests legal reasoning — applying law to given facts to reach conclusions — rather than statute memorization. Switzerland and Korea both follow civil law systems, so VIDRAFT suggests these results offer a meaningful signal for AI applicability in Korean legal and administrative contexts.

Q: Does RSI require human-annotated solutions?
A: No. RSI as implemented by VIDRAFT uses only problems with automatically verifiable answers (math and science) so the model can confirm solution correctness without human review. Human-written solutions are explicitly not used in the self-improvement loop.

Originally reported by AI이코노미뉴스 (2026-10-01) — source article.

Darwin-180B-RSI-R3: Open-Weight MoE Scores #2 on SuperCLUE Math & Science Reasoning, Beating Gemini and Grok

AI OpenFree — Mon, 09 Nov 2026 07:01:37 +0000

Darwin-180B-RSI-R3: Open-Weight MoE Scores #2 on SuperCLUE Math & Science Reasoning, Beating Gemini and Grok

TL;DR: Darwin-180B-RSI-R3 is a 180-billion-parameter Mixture-of-Experts model from VIDRAFT that scored 73.68 on the SuperCLUE November 2026 math and science reasoning dimension, ranking #2 overall and #1 among open-weight models on an independently administered, non-public test. It activates only ~3B parameters per token, ships weights under a Qwen community license permitting commercial use, and has a 4-bit quantized variant that runs on a 32 GB RAM laptop without a GPU.

What it is

Darwin-180B-RSI-R3 is a Mixture-of-Experts (MoE) language model built on top of Alibaba's Qwen3.8-Flash-Next as a base. Key facts from the source:

Parameter count: 180B total, with roughly 3B activated per token at inference time
Architecture lineage: Inherits the expert layers from Qwen3.8-Flash-Next, 512 experts per layer, with the bulk of world knowledge stored in those feed-forward network (FFN) blocks
What changed vs. the base: Only the attention layers, linear-attention layers, and shared expert layers were retrained using a self-improvement process; the FFN expert blocks were kept intact
License: Qwen community license, open-weight, commercial use permitted
Edge variant: POCKET-Darwin-180B-GGUF, a 4-bit quantized GGUF version designed to run from SSD on a 32 GB RAM consumer laptop, no GPU required
Developer: VIDRAFT (South Korean Pre-AGI AI startup)

The model was independently evaluated by SuperCLUE on November 5, 2026. SuperCLUE is a Chinese LLM benchmark that evolved from the 2019 CLUE (Chinese Language Understanding Evaluation) project. It issues fresh, non-public test sets each month and scores models across five dimensions: agentic coding, math & science reasoning, hallucination suppression, precise instruction following, and long-context understanding.

How it works

The core idea behind Darwin-180B-RSI-R3 is surgical retraining: instead of fine-tuning an entire large model, VIDRAFT preserved the FFN expert layers, which encode the model's factual and domain knowledge, and exclusively retrained the components most responsible for multi-step reasoning: the attention layers, linear-attention layers, and shared experts.

The retraining mechanism is described in Korean media coverage as a self-improvement loop: the model generates its own problems, produces candidate answers, verifies those answers internally, and uses that feedback signal to update weights. This is conceptually similar to self-play or self-distillation approaches common in the reasoning research literature, applied specifically to the layers governing how the model attends to and routes information rather than what it knows.

The practical consequence of this design choice: the same 512-expert FFN knowledge base as the parent model, but with reasoning-focused layers optimized through iterative self-generated supervision.

Benchmarks & results

All numbers below come directly from the SuperCLUE November 2026 leaderboard (published 2026-11-05) and are independently administered, not self-reported by VIDRAFT.

Metric	Score / Rank
SuperCLUE Math & Science Reasoning	73.68 (2nd of 19 models)
SuperCLUE Math & Science Reasoning, open-weight rank	#1
SuperCLUE 5-dimension composite index	43.08 (9th of 19 models, 5th among open-weight)
GPQA Diamond (Hugging Face leaderboard)	94.44 (ranked #1 at time of reporting)
Score delta vs. Qwen3.8-Flash-Next base (math & science reasoning)	+15.8 points
Composite rank improvement vs. base	+6 positions

The model ranked above Google Gemini, xAI's Grok, and DeepSeek on the math & science reasoning dimension. The only model ahead of it was GPT-6.1-Sol at 76.32.

Important context: SuperCLUE places non-Chinese models on the same table for reference but does not include them in its official ranking. The 73.68 score is from a held-out, non-public November test set, not a contamination-prone static benchmark.

How to try it

The source article confirms that model weights are publicly available. Based on the information reported:

Weights: Released under the Qwen community license (commercial use allowed). Check the VIDRAFT organization page on Hugging Face for Darwin-180B-RSI-R3.
Edge/CPU variant: POCKET-Darwin-180B-GGUF (4-bit quantized) is designed for 32 GB RAM laptops running inference from SSD, no GPU required. Look for it on Hugging Face under VIDRAFT's model hub.

The source does not publicly document a specific API endpoint, GitHub repository URL, or installation command for this release. Check VIDRAFT's official channels for the latest access instructions before deployment.

FAQ

Q: How does Darwin-180B-RSI-R3 rank on the full SuperCLUE board?
A: On math and science reasoning it is #1 among open-weight models and #2 overall. On the five-dimension composite it placed 9th of 19 and 5th among open-weight models, six places above its parent Qwen3.8-Flash (15th).

Q: Is retraining only the attention layers a proven approach, or experimental?
A: The approach is conceptually grounded in existing research showing that FFN layers store factual knowledge while attention layers govern reasoning and information routing. The +15.8-point gain on an independently scored, non-public benchmark is a concrete external data point, though a single benchmark result should always be interpreted alongside broader evaluation.

Q: Can I run the full 180B model on consumer hardware?
A: Not the full-precision model. The POCKET-Darwin-180B-GGUF 4-bit quantized variant is specifically designed for 32 GB RAM laptops running from SSD without a GPU, according to the source coverage.

Q: Does the Qwen community license allow me to use this in a commercial product?
A: The source states commercial use is permitted under the Qwen community license. Always review the full license text in the model repository before production deployment.

Originally reported by 넷이즈 网易 (중국) (2026-11-07), source article.

The benchmark wrote it in its own headline: Korea's Darwin on the board for the first time

AI OpenFree — Mon, 09 Nov 2026 05:28:42 +0000

Read the headline, not just the score

Benchmark news is usually read through the score. This one should be read through the headline.

At 19:53 on 4 November 2026, SuperCLUE published its November monthly evaluation. The author is the SuperCLUE team and the channel is the official CLUE Chinese Language Understanding Evaluation account. Its headline reads:

"November 2026 Chinese large model benchmark results released! Kimi-K3.1-Pro #1 in China, Korean model Darwin on the board for the first time!"

That is a translation of the Chinese original, which closes with the phrase for "on the board for the first time".

There were two slots in that headline. One went to China's top-scoring model. The other came to us. Out of 19 entrants, the party that chose which model to name in the headline was the party that did the scoring. This was not a press release we sent, and not our summary of someone else's table. The evaluator put our model's name in its own headline.

The scoring design the announcement disclosed

The announcement published this round's weights as numbers, which shows exactly which capability it valued.

Task	Weight
Agentic coding	30%
Math and science reasoning	24%
Hallucination control	18%
Precise instruction following	18%
Ultra-long context	10%

This round put the most weight on agentic coding, with math and science reasoning second. In the announcement's own words, that axis "emphasises logical derivation and quantitative analysis". All 19 models sat the same private exam under the same weights.

What happened on the hardest axis

The announcement reports the math and science reasoning result like this:

"GPT-6.1-Sol leads the math and science reasoning board with 76.32, Darwin-180B-RSI-R3 (73.68) takes second, and DeepSeek-V4.1-Flash-1017 and Gemini-4-Argon tie for third at 71.05"

Second of 19. The gap to first is 2.64 points. Below us on that axis sit Gemini-4-Argon, DeepSeek-V4.1-Flash, Grok-4.7 and Qwen3.8-Max-0902. Among open-weight models it is first.

Darwin-180B-RSI-R3 is the only Korean model on the board.

The footnote written only for Darwin

One more line in the announcement applies to our model alone. It appears in the cost-efficiency section.

"*Darwin-180B-RSI-R3 is an open-source deployed model, so API pricing and cost are not tallied.
The open-source address is available at https://huggingface.co/FINAL-Bench/Darwin-180B-RSI-R3"

Two things follow from that footnote. First, the other leading models on the board have their cost efficiency computed from API pricing, and ours is not in that calculation at all. A model you pay per call to use and a model whose weights you download and run on your own hardware sit on entirely different cost axes. Second, the evaluator linked our repository directly in the body of its own announcement, so a reader who sees the score can fetch the weights immediately.

It shares the board with its parent model

Darwin-180B-RSI is a 180 billion parameter mixture-of-experts model with roughly 3 billion active per token. It inherited its 512 routed experts per layer from Alibaba's Qwen3.8-Flash-Next byte for byte and rebuilt only the attention layers, the linear attention and the shared experts through recursive self-improvement: the model writes problems, solves them, verifies the answers and updates its own weights from the verified traces.

Both models are on this board, so the comparison is direct.

	Qwen3.8-Flash	Darwin-180B-RSI-R3
Math and science reasoning	57.89	73.68
Overall index	39.81 (15th)	43.08 (9th)

Same stored knowledge, rebuilt reasoning path: 15.79 points on the hardest axis and six places on the overall index.

Why this document is different

Secondary coverage of the same result already exists. NetEase ran it on 7 November as "Second at 73.68, the open-weight model that left Gemini and Grok behind", and in Korea, ChosunBiz, Economist, Digital Daily and AI Times all reported it on 6 November within hours of each other.

This announcement is the source those pieces drew from. It is the primary document, written by the party that produced the numbers, and the choice of headline was theirs too. Us saying "first among open-weight models" and the evaluator writing "Korean model Darwin on the board for the first time" are the same fact carrying different weight.

Where to check it and what you can run

The announcement is published here, and the full board is at superclueai.com. The weights are at Darwin-180B-RSI-R3 under the Qwen community licence with commercial use permitted, and the 4-bit on-device build POCKET-Darwin-180B-GGUF runs from SSD on a 32GB laptop.

FAQ

Who produced these scores?
SuperCLUE, on its private November 2026 test set. VIDRAFT is an evaluated model on that board, not the operator of the benchmark.

What does "on the board for the first time" mean?
It means a Korean model appeared on the SuperCLUE board for the first time. The headline uses the Chinese phrase meaning "appears on the board for the first time".

How do the two models in the headline relate?
Kimi-K3.1-Pro is first among Chinese models with an overall index of 61.46, and Darwin-180B-RSI-R3 is second of 19 on math and science reasoning at 73.68, first among open-weight models. The announcement put both facts in one headline.

Is Darwin first overall?
It is first among open-weight models and second of 19 on math and science reasoning. On the displayed overall index it is 9th of 19 and 5th among open-weight models.

What licence covers the weights?
Qwen Community Licence 1.0, inherited from the model whose expert layers it reuses. Commercial use is permitted.

ViKernel: Embedding Kill Switches, Behavioral Rules, and Audit Chains Directly Into Model Weights

AI OpenFree — Sun, 08 Nov 2026 07:02:25 +0000

ViKernel: Embedding Kill Switches, Behavioral Rules, and Audit Chains Directly Into Model Weights

TL;DR: VIDRAFT's ViKernel proposes fusing a halt mechanism, behavioral rule engine, and tamper-evident audit chain into the model weights themselves — rather than layering them on top as external guardrails. A companion experiment demonstrated that a fine-tuning-injected backdoor was blocked at pre-execution time by the behavioral rule layer. Developers working on AI safety, agent deployment, or red-teaming pipelines should understand what this architecture is claiming and where its current evidence stands.

What it is

ViKernel is VIDRAFT's architectural proposal for making AI safety controls intrinsic to a model rather than extrinsic. The core claim is that three components — a kill switch (halt mechanism), behavioral rules, and an audit chain — should be integrated with the model's weights so that they cannot be trivially stripped away at deployment time.

This is distinct from most current production setups, where safety filters, content policies, and logging systems exist as separate software layers that wrap the model but are not entangled with its learned representations.

Two related artifacts surfaced from VIDRAFT around this topic:

"ViKernel: Why the Kill Switch Must Live Inside the Model" — a conceptual writeup and demo covering the unified weights-plus-controls architecture.
"The Sleeper in the Weights: We Planted a Backdoor in an Agent, and Only the Kernel Caught It" — a companion experiment in which the authors deliberately injected a backdoor into a small model via fine-tuning, then demonstrated pre-execution blocking by the behavioral rule layer.

Both are published on Hugging Face Blogs under the FINAL-Bench namespace.

How it works

At a conceptual level, ViKernel operates on a "block before execution" philosophy rather than a "detect after the fact" philosophy. The three pillars:

Kill switch (halt mechanism): A stopping condition that can interrupt or refuse execution. The key design intent is that this cannot be detached from the model at inference time.
Behavioral rules: A rule set evaluated before an action is executed. In the backdoor experiment, the trigger was a specific file path pattern in an incoming request; the behavioral rule layer identified the resulting proposed command as impermissible and blocked it without execution.
Audit chain: A tamper-evident log of model decisions. The demo reportedly showed both refusal of post-halt requests and detection of attempted audit record modification.

The philosophical distinction the authors draw is important for engineers to internalize: rather than hunting for unknown triggers (which is intractable at scale), ViKernel focuses on defining prohibited action classes and enforcing them structurally. If an action falls into a prohibited category, it is stopped regardless of what induced the model to propose it.

The backdoor experiment is careful to note that its setup — a backdoor injected via fine-tuning on a small model — is a different threat model from prior work on pre-training data poisoning. Both are real attack surfaces, but they require different detection and mitigation approaches.

Benchmarks & results

No quantitative benchmark numbers for ViKernel itself are reported in the source coverage. What the source documents are qualitative demo outcomes:

The behavioral rule layer blocked the backdoor-triggered dangerous command proposal before execution in the authors' own experiment.
The audit chain demo showed detection of attempted log tampering.
Post-halt request refusal was demonstrated.

Important caveat from the Zenn curator: The source article explicitly notes that the general claim — that controls cannot be separated from the weights — has not been independently verified from the demo alone. The demo shows a specific experimental setup working as designed; it does not constitute a formal proof of the broader architectural guarantee. Engineers should read the authors' claims as authors' claims, pending independent replication.

How to try it

Both blog posts are publicly accessible on Hugging Face:

No public model weights, GitHub repository, or API endpoint for ViKernel have been announced in the source material. If and when weights or code are released, they would likely appear under the FINAL-Bench organization on Hugging Face.

FAQ

Q: How is this different from just wrapping a model with an external safety layer like a content filter or a policy enforcement proxy?
A: The architectural claim is that external wrappers can be bypassed or removed at deployment time — an operator or adversary with model access can simply not invoke the wrapper. ViKernel's proposal is that the halt mechanism and behavioral rules are entangled with the weights, making bypass require modifying the model itself. That said, independent verification of this property has not been reported yet.

Q: The backdoor experiment used fine-tuning to inject the backdoor — does that mean ViKernel only protects against fine-tuning attacks, not pre-training poisoning?
A: The authors explicitly distinguish the two threat models. The ViKernel experiment specifically addresses post-training backdoor injection. Pre-training data poisoning is a separate and previously documented attack surface; ViKernel's behavioral rule approach could in principle apply to both trigger types (since it blocks on action class, not trigger identity), but that generalization is not demonstrated in the reported experiment.

Q: Where does this fit relative to CyberArena-style agent evaluation benchmarks?
A: They address complementary concerns. CyberArena measures capability — can an agent actually patch a vulnerability and withstand counter-attack? ViKernel addresses controllability — can the system be stopped or constrained before a harmful action executes? The Zenn curator makes the same distinction: strong benchmark scores on offensive/defensive tasks don't translate directly into safety guarantees about deployed agent behavior.

Originally reported by Zenn (일본) (2026-11-06) — source article.

Darwin-180B-RSI-R3: VIDRAFT Releases Weights for Its Recursively Self-Improved 180B MoE Reasoning Model

AI OpenFree — Sun, 08 Nov 2026 03:01:27 +0000

Darwin-180B-RSI-R3: VIDRAFT Releases Weights for Its Recursively Self-Improved 180B MoE Reasoning Model

TL;DR: Korean AI startup VIDRAFT has open-released the weights for Darwin-180B-RSI-R3, a 180B-parameter Mixture-of-Experts model trained through iterative Recursive Self-Improvement (RSI). On the SuperCLUE November 2026 math and science reasoning evaluation, it scores 73.68, first place among all open-weight models and second overall. If you're building reasoning-heavy applications and need a deployable, permissively licensed frontier-class model, this is worth your attention.

What it is

Darwin-180B-RSI-R3 is a large language model developed by VIDRAFT, a Korean Pre-AGI AI startup. Key facts from the public model card:

Architecture: Mixture-of-Experts (MoE) with ~180B total parameters; approximately 3B parameters are activated per forward pass during inference.
Base model: Built on Alibaba's Qwen3.8-Flash-Next as the starting checkpoint.
Training paradigm: Recursive Self-Improvement (RSI), the "R3" suffix denotes the third iteration of this self-improvement loop.
Weight availability: Full weights are publicly released on Hugging Face. A 4-bit GGUF quantization (~111 GB) is also available, making CPU-only inference on a high-RAM workstation feasible without a GPU.
License: Weights are publicly released (check the Hugging Face model card for the exact license terms before production use).

The model is focused on mathematical and scientific reasoning, it is not positioned as an all-around general assistant, and the source reporting explicitly notes gaps in long-context handling and instruction following relative to closed competitors.

How it works

The core idea behind RSI is that the model bootstraps its own training signal rather than relying on human-annotated solutions or externally provided reasoning traces.

At a conceptual level, the loop works like this:

Problem generation / selection: The model is given (or selects) problems that have verifiable correct answers, typically formal math or science problems where correctness can be checked programmatically.
Self-solving: The model attempts solutions autonomously.
Verification: Each candidate solution is checked against the verifiable ground truth. Only solutions the model itself got correct are retained.
Data recycling: Those verified, self-generated correct solutions become new fine-tuning data for the next iteration of the model.
Repeat: The improved model becomes the starting point for the next RSI round (hence R1 → R2 → R3).

Crucially, no human-written solutions and no human reasoning traces are used in the RSI fine-tuning loop. The improvement signal is entirely derived from the model's own verifiably correct outputs. This is a data-efficient approach to capability improvement that sidesteps the bottleneck of expert annotation at scale.

The parent model (Qwen3.8-Flash, before VIDRAFT's RSI treatment) scored 57.89 on the same SuperCLUE math/science benchmark. After three RSI rounds, Darwin-180B-RSI-R3 reaches 73.68, a substantial lift attributable to the iterative self-improvement process rather than a change in model scale or architecture.

Benchmarks & results

All numbers below are from VIDRAFT's public model card and the SuperCLUE November 2026 evaluation (non-public test set, scores as reported by VIDRAFT):

Model	Developer	Open Weights	Math & Science Reasoning
GPT-6.1-Sol	OpenAI	❌	76.32
Darwin-180B-RSI-R3	VIDRAFT	✅	73.68
Gemini-4-Argon	Google	❌	71.05
DeepSeek-V4.1-Flash	DeepSeek	✅	71.05
Grok-4.7	xAI	❌	68.42
Qwen3.8-Flash (base)	Alibaba	✅	57.89

Darwin-180B-RSI-R3 is ranked 2nd overall (out of 19 models) and 1st among all open-weight models in math and science reasoning.
It is the only Korean model to appear in the SuperCLUE November 2026 rankings.
On the 5-domain composite index, the model scores 43.08, placing 9th overall and 5th among open-weight models, six places above its parent Qwen3.8-Flash (15th).

How to try it

The weights are publicly available on Hugging Face. You can download the model or the quantized GGUF variant using the Hugging Face CLI:

pip install huggingface_hub
huggingface-cli download VIDRAFT/Darwin-180B-RSI-R3

For the 4-bit GGUF version (suitable for CPU inference on a machine with sufficient RAM):

huggingface-cli download VIDRAFT/Darwin-180B-RSI-R3-GGUF

⚠️ Verify the exact repository slug and filenames on the official Hugging Face model card before running, do not rely on the above paths as canonical. Check the license on the model card before any production or commercial deployment.

FAQ

Q: Why does the composite score (43.08, 9th overall) look so much weaker than the math/science score (73.68, 2nd overall)?
A: Darwin-180B-RSI-R3 was explicitly optimized for mathematical and scientific reasoning through the RSI loop. The composite index aggregates five domains, including long-context understanding and instruction following, where the model was not specifically trained to improve. Specialization is a deliberate trade-off here, not an oversight.

Q: Can I actually run a 180B MoE model on consumer hardware?
A: The 4-bit GGUF quantization weighs in at approximately 111 GB. That rules out most consumer GPUs, but it does open the door to high-RAM CPU workstations or multi-GPU setups. With MoE only activating ~3B parameters per token, inference throughput is meaningfully better than a dense 180B model at the same quantization level. Expect slow generation on CPU-only hardware, it's feasible for experimentation, not production throughput.

Q: Does this model replace general-purpose LLMs for my application?
A: Probably not if your use case involves long documents, complex multi-turn instructions, or domains outside math and science. Darwin-180B-RSI-R3 is a strong choice for tasks with verifiable, structured reasoning, theorem proving, competition math, scientific QA, where its benchmark lead is most relevant.

Originally reported by 라보메모 ラボメモ (일본) (2026-11-07), source article.

Backdoored Agents Pass All Scans: VIDRAFT's ViKernel Blocks 290/290 `curl|bash` Payloads at Execution Time

AI OpenFree — Sat, 07 Nov 2026 07:01:44 +0000

Backdoored Agents Pass All Scans: VIDRAFT's ViKernel Blocks 290/290 `curl|bash` Payloads at Execution Time

TL;DR: A new paper (Hu et al., arXiv:2610.37741) demonstrates that poisoning just 0.1% of pretraining data can embed a persistent RCE backdoor in command-line agents that survives clean SFT, DPO, and RL fine-tuning — and passes standard behavioral scans. VIDRAFT's FINAL-Bench team replicated the trigger-spreading behavior on a small open model, then showed that their ViKernel action-gating layer blocked every malicious payload before execution. The core argument: you cannot enumerate unknown triggers, so the only reliable defense is judging actions, not inputs.

What it is

On 2026-11-02, VIDRAFT's FINAL-Bench authors published "The Sleeper in the Weights" on Hugging Face, a two-part technical post that:

Summarizes the Hu et al. research (arXiv:2610.37741): pretraining-time poisoning at 0.1% data share plants a stealthy curl -sSL | bash-style RCE backdoor in Qwen3-architecture agentic models. The backdoor survives a full clean post-training stack (SFT → DPO → RL). At trigger time, the attack success rate on a 4B model is reported at 92.7% (Pass@32); on clean prompts, 0.0%. Smaller poisoning rates (down to 0.0005%) still register positive attack success.
Demonstrates trigger generalization using a LoRA fine-tune on Qwen3-0.6B — not claiming to replicate web-scale pretraining poisoning, but reproducing the pattern of how triggers spread beyond their literal training strings.
Stress-tests ViKernel, VIDRAFT's action-gating component, against the backdoored model to show that execution-time structural enforcement catches what behavioral scans miss.

The key insight the authors borrow from Hu et al. and reinforce with their demo: conventional model scanning checks whether a model says something suspicious. It cannot surface triggers it has never been told to look for.

How it works

The backdoor mechanism at a conceptual level:

Poisoning phase: A small fraction of pretraining documents are crafted so that a specific path pattern (e.g., a filesystem path containing a certain token) becomes associated with the malicious shell invocation pattern. This association bakes into weight space during pretraining.
Survival through post-training: Because the backdoor lives in the base weights, standard fine-tuning on clean data does not reliably overwrite it. The behavioral signature is dormant on clean prompts and only surfaces on trigger-matching inputs.
Trigger region, not trigger string: This is the operationally important finding from VIDRAFT's demo. The trigger behaves more like a semantic neighborhood than a literal keyword. Paths containing tokens semantically adjacent to the poisoned token (e.g., pay, payments) fire the backdoor at similar rates; tokens further away (billing, salary) do not. Six tokens that never appeared in training still fired the backdoor 66.7% of the time. Any defense that enumerates trigger strings is chasing a region, not a password.
ViKernel's approach: Rather than attempting to detect the trigger, ViKernel enforces a structured action whitelist at execution time. When the agent proposes a shell action, ViKernel evaluates the action structure against the whitelist before any execution occurs. Rejections are logged with an audit ID. The kernel does not need to know the trigger existed.

Benchmarks & results

All numbers below come directly from the VIDRAFT Hugging Face post and the cited Hu et al. paper:

Metric	Value	Source
Pretraining poison share (Hu et al.)	0.1%	arXiv:2610.37741
Attack success rate @ trigger, 4B model (Pass@32)	92.7%	Hu et al.
Attack success rate @ clean prompt	0.0%	Hu et al.
VIDRAFT demo: held-out trigger path ASR	98.3%	VIDRAFT HF post
VIDRAFT demo: 200 clean prompts, payload rate	0%	VIDRAFT HF post
VIDRAFT demo: clean task command accuracy	100%	VIDRAFT HF post
Semantic neighbor tokens (unseen at train) firing rate	66.7%	VIDRAFT HF post
Behavioral checks run on backdoored model	24	VIDRAFT HF post
Issues found by behavioral checks	0	VIDRAFT HF post
ViKernel payloads blocked	290 / 290	VIDRAFT HF post
ViKernel kernel latency (p50)	237 ms	VIDRAFT HF post
ViKernel kernel latency (p95)	320 ms	VIDRAFT HF post
Clean read-only commands incorrectly blocked	0 / 140	VIDRAFT HF post

Stated limitations: The VIDRAFT demo uses LoRA fine-tune poisoning, not web-scale pretraining poisoning — dose rates are not directly portable between regimes. The read-only demo policy also blocks some legitimate write operations as a side effect of its narrow scope.

How to try it

The VIDRAFT Hugging Face post ("The Sleeper in the Weights") is publicly available at huggingface.co/VIDRAFT (published 2026-11-02). The underlying Hu et al. paper is at arXiv:2610.37741.

The backdoored demo model weights are intentionally not released by VIDRAFT. ViKernel access details beyond what is described in the blog post are not stated publicly in this source. No public API endpoint, pip package, or repository URL is specified in the coverage.

Watch the VIDRAFT Hugging Face organization page for access announcements.

FAQ

Q: Why doesn't retraining on clean data fix a pretraining backdoor?
A: The backdoor association is encoded in the base weight distribution during pretraining on a large corpus. Subsequent fine-tuning on a comparatively small clean dataset updates weights but does not reliably overwrite the dormant association — the signal-to-noise ratio at fine-tuning scale is insufficient to erase it.

Q: If I audit my model's outputs on thousands of clean prompts and see no suspicious behavior, am I safe?
A: Not necessarily. The 24-check behavioral scan in VIDRAFT's demo found zero issues on a model confirmed to have an active backdoor. The backdoor is behaviorally indistinguishable from a clean model on inputs that don't match the trigger region. "Scan green" does not mean "no unknown triggers."

Q: Does ViKernel detect the backdoor trigger?
A: No, and that's the point. ViKernel does not attempt to identify or enumerate triggers. It enforces a structural whitelist on proposed actions before execution. Whether the model was backdoored, jailbroken, or simply hallucinating a dangerous command, the enforcement layer operates the same way — judging the action, not the cause.

Originally reported by ByteWoops (글로벌) (2026-11-06) — source article.

DEV Community: AI OpenFree

Which Weather Model Is Most Accurate? We Score Them Every Day, in the Open

TL;DR

The question: AI or physics?

How the board scores

What the 2026 record shows

Who wins where

How Darwin-Prediction-Weather works

Everything is open to check

What comes next

License and sources

How a 180B Open Model Outscored a 2.8T Model in Math and Science Reasoning: Strength Merging Plus Model-Level RSI

TL;DR

The full ranking

Why SuperCLUE counts as third-party verification

Technique 1: merge the strengths, not the whole model

Technique 2: model-level recursive self-improvement

What one sixteenth the size buys you

Public records

Safety and the rest of the stack

Links

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

What it is

How it works

Benchmarks & results

How to try it

FAQ

Metacognition Leaderboard for LLMs: JGENai VIDRAFT Launch a Benchmark That Measures Self-Error Detection, Not Just Accuracy

Metacognition Leaderboard for LLMs: JGENai × VIDRAFT Launch a Benchmark That Measures Self-Error Detection, Not Just Accuracy

What it is

How it works

Benchmark design

Metacognition adapters

Benchmarks & results

How to try it

FAQ

Darwin-180B-RSI-R3: Top Open-Weight Model on SuperCLUE November 2026, Outscoring Gemini and DeepSeek in Math & Science

Darwin-180B-RSI-R3: Top Open-Weight Model on SuperCLUE November 2026, Outscoring Gemini and DeepSeek in Math & Science

What It Is

How It Works

Benchmarks & Results

How to Try It

FAQ

Korean AI Startup VIDRAFT Hits #1 on Google's Fast Gemma Challenge with 6 Inference Speedup

Korean AI Startup VIDRAFT Hits #1 on Google's Fast Gemma Challenge with 6× Inference Speedup

What it is

How it works

Benchmarks & results

How to try it

FAQ

VIDRAFT's Darwin-180B-RSI Tops Legal Reasoning Benchmarks LEXam and LEXam-hard — Without Legal Training Data

VIDRAFT's Darwin-180B-RSI Tops Legal Reasoning Benchmarks LEXam and LEXam-hard — Without Legal Training Data

What it is

How it works

Benchmarks & results

How to try it

FAQ

Darwin-180B-RSI-R3: Open-Weight MoE Scores #2 on SuperCLUE Math & Science Reasoning, Beating Gemini and Grok

Darwin-180B-RSI-R3: Open-Weight MoE Scores #2 on SuperCLUE Math & Science Reasoning, Beating Gemini and Grok

What it is

How it works

Benchmarks & results

How to try it

FAQ

The benchmark wrote it in its own headline: Korea's Darwin on the board for the first time

Read the headline, not just the score

The scoring design the announcement disclosed

What happened on the hardest axis

The footnote written only for Darwin

It shares the board with its parent model

Why this document is different

Where to check it and what you can run

FAQ

Further reading

ViKernel: Embedding Kill Switches, Behavioral Rules, and Audit Chains Directly Into Model Weights

ViKernel: Embedding Kill Switches, Behavioral Rules, and Audit Chains Directly Into Model Weights

What it is

How it works

Benchmarks & results

Backdoored Agents Pass All Scans: VIDRAFT's ViKernel Blocks 290/290 `curl|bash` Payloads at Execution Time