DEV Community: Prabhakar Chaudhary

Mistral Large 4: Inside Europe's First 1-Trillion-Parameter Open-Weight Model

Prabhakar Chaudhary — Mon, 12 Oct 2026 16:06:27 +0000

Mistral Large 4: Inside Europe's First 1-Trillion-Parameter Open-Weight Model

Mistral AI released a public preview of Mistral Large 4 on October 6, 2026 — a 1.05-trillion-parameter multimodal Mixture-of-Experts model that the company has nicknamed "Le Chonk." The weights are promised by the end of October, making it the largest open-weight model to come out of a European lab. This post walks through the architecture, training methodology, benchmark results, and what the release means for practitioners who want to run frontier-scale models on their own infrastructure.

Architecture: Granular MoE with a Vision Encoder

Mistral Large 4 is a sparse Mixture-of-Experts model with 1.05 trillion total parameters and 52 billion active parameters per token. The "granular" MoE design differs from coarser expert layouts: instead of routing tokens to a small number of large expert blocks, the router selects from a larger pool of smaller, more specialized units. This gives the routing mechanism finer-grained control over which capacity is applied to each token, which Mistral says improves both throughput and task specialization.

A separate 1.6-billion-parameter vision encoder handles image inputs. The encoder converts visual content — charts, engineering drawings, satellite imagery, PDFs — into representations that the language backbone can reason over. The result is a single model that handles instruction following, multi-step reasoning, and multimodal understanding without switching between separate systems.

The model was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own European data centers, not on rented cloud capacity. The context window is listed as 1 million tokens in Mistral's documentation, though the preview API currently enforces a 524,000-token limit on some third-party providers.

Reinforcement Learning at Scale

Mistral describes its post-training approach as "RL at scale" — a reinforcement learning pipeline that adapts dynamically as the model's capabilities change during training. Rather than running a fixed RL curriculum, the system continuously adjusts the task distribution to stay at the edge of the model's current ability.

At the current compute scale (roughly 3,000 GPUs), a single training run produces approximately 33 billion tokens per day, of which around 16 billion are trainable completion tokens after filtering and masking. The pipeline is optimized for long trajectories, supporting rollout budgets of millions of tokens across multiple compaction steps — which is what allows the model to learn from complex, multi-turn agentic tasks rather than just short-form completions.

Importantly, Mistral notes that the RL run behind the preview is still in flight. The model is expected to continue improving before the weights ship, which is an unusual transparency about the release timeline.

Benchmark Results

Mistral published results across several domains. Here are the headline numbers from the official announcement:

Agentic coding:

DeepSWE v1.1: 61.7%
SWE-Atlas-QnA: 59.4%
Terminal-Bench 4: 28.3%
Combined Coding Agent Index: 49.8% (ahead of DeepSeek V4 Pro and Qwen3.8 Max)

Agentic workflows:

AutomationBench (657 business workflows): 59.9%
AA-Briefcase (long-horizon knowledge work): 1,393 Elo

Cybersecurity:

Cybench (40 CTF-style challenges): 93%
Vulnerability reproduction and patching test: 82%
B3 AI Security Benchmark (attack resistance): 93.3%

Multimodal:

Dense 200 visual grounding: 42% (vs. 41% for GPT-6 Astra)

In a blind human evaluation run with Surge AI, professional annotators rated the preview 3.74/5, placing it second among five models — behind Claude Opus 5 (4.22) and ahead of GLM-5.3, Kimi K3, and GLM-5.2.

Independent evaluator Artificial Analysis scores the preview at 38 on its Intelligence Index (with reasoning enabled), up from 9 for the non-reasoning Mistral Large 3. Open-weight models from Chinese labs still score higher on that index — GLM-5.3 at 45, Kimi K3 at 44 — but Mistral Large 4 leads among open-weight models developed outside China.

Cybersecurity as a First-Class Use Case

One of the more deliberate design choices in Mistral Large 4 is its treatment of cybersecurity as a primary capability rather than an afterthought. The model is being red-teamed by cybersecurity organizations and government authorities before the weights are released, and Mistral is offering a version with reduced moderation and expanded cyber capabilities to vetted defenders.

The rationale is practical: in incident response and vulnerability research, provider-level refusals can block legitimate work. A model that can be self-hosted under an organization's own policies — and that won't refuse to analyze malware or reproduce a known CVE — is meaningfully different from a hosted API with conservative safety filters. The Artificial Analysis Cyber Index places Mistral Large 4 in the global top five, and it leads open-weight models developed outside China on that benchmark.

Pricing and Self-Hosting

The preview API is priced at $1.36 per million input tokens and $4.18 per million output tokens (with a 50% batch discount). Mistral also offers a European regional endpoint at api.eu.mistral.ai for organizations with data residency requirements, at 1.1× the standard list price. A detailed pricing breakdown is available in Mistral's API documentation.

Self-hosting will require significant hardware. At 8 bits per parameter, 1.05 trillion parameters occupy roughly 1 TB of memory before KV cache and runtime overhead. That puts it in the range of a multi-node GPU cluster for full-precision inference, though quantized deployments will reduce that substantially. Mistral has not yet announced a license for the weights.

What This Means for Practitioners

A few things stand out about this release:

The open-weight commitment matters. Most 1T-parameter models are closed APIs. Mistral Large 4 being open-weight — even if the license is not yet announced — means organizations can audit the model, fine-tune it, and run it on private infrastructure. That is a different risk profile than a hosted API.

The RL training pipeline is the real story. The benchmark numbers are competitive, but the more interesting detail is the training infrastructure: 33 billion tokens per day, long-trajectory rollouts, and a pipeline that adapts to the model's current capability level. That is the kind of post-training machinery that separates frontier labs from everyone else, and Mistral is building it in Europe.

Multimodal is now table stakes. The vision encoder is not a separate product — it is part of the base model. For practitioners building document processing, satellite imagery analysis, or engineering workflow tools, having a single model that handles both text and images simplifies the stack considerably.

The hardware bar for self-hosting is high. 1 TB of memory at 8-bit precision is not a single-server deployment for most organizations. Quantized versions will help, but practitioners should plan for multi-node setups or managed inference until smaller distilled variants appear.

Conclusion

Mistral Large 4 is a technically serious release: a 1-trillion-parameter MoE with a granular routing architecture, a dedicated vision encoder, and a reinforcement learning pipeline designed for long-horizon agentic tasks. The benchmark results are competitive with the current frontier, and the open-weight commitment — unusual at this scale — gives organizations a path to sovereign deployment that closed APIs cannot offer.

The weights are expected by the end of October 2026. Until then, the preview API is available through Mistral Studio, and the full architecture details and post-training methodology will accompany the weight release. For teams evaluating frontier open-weight models for agentic coding, cybersecurity, or multilingual enterprise workflows, this is worth tracking closely. Weights will be published on Mistral's HuggingFace page.

JEV-27B: How AutoTrust AI Built an Open Decision Model That Thinks Fast and Reasons Deep

Prabhakar Chaudhary — Fri, 09 Oct 2026 16:05:28 +0000

JEV-27B: How AutoTrust AI Built an Open Decision Model That Thinks Fast and Reasons Deep

Most AI workloads in production don't need a model to write an essay. They need a fast, reliable answer to a narrow question: Is this email phishing? Which support queue does this ticket belong to? Does this content violate policy? For years, developers have been using full-scale large language models to answer these questions — paying frontier-model prices for tasks that don't require frontier-model capabilities.

AutoTrust AI's JEV-27B is a direct response to that mismatch. Released in early October 2026 under an Apache-2.0 license, it introduces a new category of open-weight model: the decision model — a system designed to return calibrated, structured judgments at high speed, without sacrificing the ability to reason when reasoning is actually needed.

What Is a Decision Model?

The term borrows from psychologist Daniel Kahneman's framework of System 1 (fast, intuitive) and System 2 (slow, deliberate) thinking. Traditional LLMs are System 2 machines: they generate tokens sequentially, building up a response word by word. That's powerful for open-ended tasks, but it's expensive and slow for classification, scoring, or routing.

A decision model is optimized for System 1 tasks. Given a natural-language question and a set of typed options — a boolean, a label from a fixed set, or a numeric score — it returns a calibrated probability distribution in a single forward pass, without decoding, JSON parsing, or prompt engineering. The output isn't text; it's a structured judgment with a confidence score attached.

This matters operationally. When you're running millions of content moderation checks per day, the difference between 137 ms and 280 ms per decision, multiplied across that volume, is the difference between a viable product and an unaffordable one.

The Blocks of Experts Architecture

JEV-27B's core technical contribution is what AutoTrust calls the Blocks of Experts (BoE) recipe. Rather than fine-tuning the entire model — which risks degrading the reasoning capabilities that make a large model useful — BoE keeps the backbone frozen and adds a small, detachable expert block for the decision task.

The architecture has three components:

Backbone: The Qwen3.8-27B text tower, frozen and bit-identical to its original release. This is the System 2 engine.
System 1 Block: A LoRA adapter (rank 16) plus a 24-slot fp32 decision head, totaling about 108.9 million parameters — roughly 0.4% of the full model size. This block is trained on the SargeDev/jev-distill-corpus-v3 dataset to replicate the output distributions of TypeSafe's closed-source Jev 1.13 model.
Router: A per-request mechanism that directs traffic to either the decision block or the generation block, depending on whether the incoming request is a typed decision query or a free-form generation task.

The key insight is that the System 1 block is detachable. When it's inactive, the model is byte-identical to the base Qwen3.8-27B. AutoTrust verified this by running HumanEval with the decision block both active and inactive: all 164 completions were identical, and the model scored 78.0% either way. This avoids the performance degradation that often occurs when LoRA weights are merged directly into a backbone.

Calibration, Not Just Classification

What separates JEV-27B from a standard classifier is its emphasis on calibration. Rather than returning a hard label, the model outputs a probability distribution over the possible answers. This lets downstream systems set confidence thresholds: auto-approve if probability > 0.9, escalate to a human reviewer if it falls below 0.7, and route to a more expensive model for the ambiguous middle.

AutoTrust measured calibration fidelity against the teacher model (TypeSafe Jev 1.13) across 25,376 held-out questions. The mean KL divergence was approximately 0.017 — meaning JEV-27B's probability distributions are nearly indistinguishable from the closed-source model it was trained to replicate. On six public decision benchmarks, JEV-27B averaged 84.07%, slightly above the teacher's 83.85%.

Latency on a single NVIDIA B200 GPU: 137 ms median per decision, compared to 238–301 ms for hosted API alternatives. Throughput: approximately 130 decisions per second.

Practical Use Cases

The AutoTrust blog post and independent evaluations from Cisco's AI team highlight several production patterns where decision models outperform general-purpose LLMs:

Content moderation at scale. Convert a policy into a series of typed questions. JEV-27B answers each with a calibrated score, making it easy to aggregate signals and set per-category thresholds.

Ticket routing and triage. Classifying support tickets into queues is a high-volume, low-complexity task. A decision model handles this in a single pass at a fraction of the cost of a frontier model.

Data quality screening. Checking whether a user-submitted field is valid, whether a code change introduces a security pattern, or whether a document matches a regulatory template — judgment calls that don't require prose generation.

Cost-efficient agent pipelines. In multi-step agentic workflows, decision models serve as fast pre-filters that handle easy cases and only escalate to expensive reasoning models when the situation is genuinely ambiguous.

The Competitive Landscape

JEV-27B didn't arrive in a vacuum. TypeSafe AI's hosted Jev service had already gained traction before AutoTrust released the open-weight version. Then, on October 6, 2026, OpenAI launched its Decisions API, built on GPT-6 Luna, entering the same market directly.

The pricing comparison is instructive: OpenAI's Decisions API lists at $0.10 per million input tokens; TypeSafe Jev charges $0.042 per million. JEV-27B, being self-hosted, shifts the cost to infrastructure — cheaper for organizations already running GPU clusters. OpenAI's version supports image input (JEV-27B-VL, a visual follow-on from AutoTrust, addresses this gap), and claims up to 10× speed improvement over its general-purpose Responses API.

What This Means for Practitioners

The emergence of decision models as a distinct category reflects a broader maturation in how teams think about AI infrastructure. Not every task in a pipeline needs the same model. A well-designed system might use a decision model for high-volume triage, a mid-tier reasoning model for moderate complexity, and a frontier model only for genuinely hard problems.

JEV-27B makes this architecture accessible to teams that need data sovereignty, want to avoid per-token API costs at scale, or simply prefer to control their own inference stack. The Apache-2.0 license means it can be deployed commercially without restriction, and the vLLM compatibility means it slots into existing serving infrastructure without custom work.

The Blocks of Experts approach is also worth watching as a general technique. If it proves reliable across other task types — structured extraction, scoring, or routing — it could become a standard recipe for adding specialized capabilities to open-weight models without compromising their general-purpose utility.

Getting Started

JEV-27B is available on Hugging Face under Apache-2.0. AutoTrust's blog post includes a quickstart guide for deploying with vLLM and examples of how to structure decision queries using the noul, choice, and score task types. Cisco's independent evaluation provides a useful comparison against traditional fine-tuned classifiers and the hosted Jev API.

The core question for any team considering this approach is whether their workload has enough high-volume, structured judgment tasks to justify the infrastructure overhead of self-hosting. For organizations already running inference at scale, the answer is likely yes.

EmbeddingGemma 2: How Google's Open Multimodal Embedding Model Works

Prabhakar Chaudhary — Fri, 09 Oct 2026 11:21:04 +0000

EmbeddingGemma 2: How Google's Open Multimodal Embedding Model Works

If you've built a RAG pipeline or a semantic search system, you've almost certainly reached for a separate embedding model for each modality — one for text, another for images, maybe a third for audio. Google DeepMind's EmbeddingGemma 2, released October 6, 2026, takes a different approach: a single 740M-parameter open model that maps text, code, images, video, and audio into one shared vector space, designed to run entirely on-device.

The original EmbeddingGemma (2025) was text-only and accumulated over 20 million downloads. This release extends it to five modalities while keeping the model small enough to run on a phone. Here's what's actually inside it and what it means for practitioners.

The Architecture: One Space, Modular Encoders

EmbeddingGemma 2 is built on the Gemma 4 transformer backbone — 24 layers, grouped-query and multi-query attention, a 262,144-token vocabulary, and mean pooling with a projection layer from 512 to 768 dimensions. The key design decision is that all modalities project into the same 768-dimensional embedding space, which means a text query and an audio clip can be compared directly by cosine similarity.

The model is modular. From a single checkpoint, you load only the encoders you need:

Text and code only: 270M parameters (130M transformer backbone + 140M embedder)
Text + vision: 440M parameters (add the 170M vision encoder)
Text + audio: 570M parameters (add the 300M audio encoder)
Full multimodal: 740M parameters

This matters for deployment. If your app only does text-to-image retrieval, you're not paying the memory cost of the audio encoder.

The model shares its text tokenizer and audio encoder architecture with Gemma 4. If you're already running Gemma 4 for generation, you can pair EmbeddingGemma 2 for retrieval with a lower combined memory footprint than running two independent models.

The Token Budget: How Media Gets Encoded

All five modalities share an 8,192-token context window — four times larger than EmbeddingGemma 1. Each modality consumes that budget at fixed rates:

Modality	Tokens per unit	Max per pass
Images	280 per image	29 images
Video frames	140 per frame	58 frames
Audio	25 per second	~5.5 minutes
Text/code	Standard tokenization	Up to 8K tokens

Inputs can be interleaved — text and images mixed in one pass, with placeholder tokens marking each media item's position. This is what enables queries like "find the video clip where someone says X while showing Y on screen."

Matryoshka Representation Learning: Tunable Storage

EmbeddingGemma 2 uses Matryoshka Representation Learning (MRL), a training technique that makes the first N dimensions of any output embedding a valid, high-quality embedding on their own. At inference time, you can truncate from 768 dimensions down to 512, 256, or 128 without retraining.

The practical tradeoff, from the model card:

256 dimensions: ~95% of full quality on multimodal retrieval, ~6x storage reduction
128 dimensions: ~90% on text/code, but drops to ~75% on multimodal retrieval

Storing one million 768-dimensional vectors in bfloat16 takes roughly 1.5GB. At 128 dimensions, that drops to about 250MB. For large-scale vector databases where storage is a real cost, this is a meaningful lever — and you can tune it per use case without retraining.

Benchmark Results

Google reports the following on the full-precision model:

MTEB Code: 78.68 (up from 68.76 in EmbeddingGemma 1, a +9.92 point improvement)
MTEB multilingual (v2): 61.36
MIEB lite: 64.64
MMEB v2 image retrieval: 57.28
Visual-document retrieval: 67.84
Video retrieval: 50.67
MSEB sound retrieval: 69.54
MAEB audio: 49.39

The code search improvement is notable for anyone building coding agents or codebase indexing tools. The model outperforms some specialist models more than twice its size on audio and visual benchmarks — though it's worth noting these are Google's reported numbers, and independent evaluations on your specific retrieval task will tell you more.

On-Device Performance and Deployment

With quantization on a Pixel 11 Pro:

Text-only: ~191MB active RAM
Full multimodal: ~567MB active RAM

For browser deployment, the model runs via transformers.js and WebGPU. For mobile and edge, Google AI Edge's MediaPipe and LiteRT handle cross-platform deployment. The model also works with the standard server-side stack: transformers, sentence-transformers (6.1.0+), vLLM, llama.cpp, SGLang, Ollama, and LMStudio. Vector storage integrations include Qdrant.

Weights are available on Hugging Face and Kaggle under the Apache 2.0 license — commercially usable without restrictions.

Limitations to Know Before Deploying

A few things the model card is explicit about:

No safety tuning. EmbeddingGemma 2 is a pre-trained embedding model with no post-training alignment or output-level moderation. Safety mitigations are limited to pre-training data filtering (CSAM filtering, automated PII removal). If your retrieval pipeline surfaces sensitive content, you need to add your own application-level safeguards.

Use bfloat16 or float32, not float16. The model's activation range exceeds float16's dynamic range, which can produce NaN values or silently degraded embeddings. This is easy to miss if you're defaulting to float16 for efficiency.

Task instruction prefixes matter. For text inputs, the model expects task-specific instruction prefixes (e.g., "Represent this code for retrieval:"). Omitting them reduces embedding precision. The developer guide covers the recommended prefixes per task type.

MRL quality at 128 dims drops on multimodal. If you're doing cross-modal retrieval (text-to-audio, image-to-video), 128-dimensional vectors retain only ~75% of full quality. For text-only workloads, 128 dims is fine.

What This Means for Your Stack

The most immediate use case is replacing multiple single-modality embedding models with one. If you're building a RAG pipeline over a mixed corpus — documents, screenshots, audio recordings, video clips — EmbeddingGemma 2 lets you index everything into one vector database and query across modalities with a single model call.

The on-device angle is also worth taking seriously. A 191–567MB footprint means you can run retrieval entirely locally, which matters for privacy-sensitive applications (medical records, legal documents, personal media) and for offline-capable apps. Paired with Gemma 4 for generation, you get a complete on-device RAG system.

For coding agents specifically, the +9.92 MTEB Code improvement over EmbeddingGemma 1 makes this a strong candidate for codebase indexing and semantic code search — tasks where the previous version was already competitive.

The model is available now. If you're already using EmbeddingGemma 1, the upgrade path is straightforward: same ecosystem integrations, same license, substantially expanded capability.

Is a 10x Jump Always a 10x Jump? A Statistical Audit of the METR Time-Horizon Plot

Prabhakar Chaudhary — Fri, 09 Oct 2026 11:20:16 +0000

If you follow AI progress, you have seen the METR plot: a line on a log axis showing that the length of software tasks AI systems can complete keeps growing exponentially. A new paper from two Berkeley statisticians, On the estimation and validity of AI time horizons (Nguyen and Fithian, October 2026), does not dispute the trend. It asks a narrower question that matters to anyone who uses benchmarks to make decisions: what exactly is the y-axis, and is a "10x longer horizon" the same kind of improvement everywhere on the axis?

What the time horizon actually measures

The metric comes from METR's original paper. Each task in a suite gets a human time: how long a skilled engineer takes to do it, using the geometric mean of successful timed attempts. Each model then attempts the tasks several times. The 50% time horizon is the human time at which the model's success probability crosses one half.

The 50% horizon is not measured directly. It is read off a fitted curve. For each model j, the standard recipe fits a logistic regression, with log2(human time) as the only covariate:

logit p_j(t) = alpha_j - beta_j * log2(t)

Then you invert it: find the t where p_j(t) = 0.5. The intercept and slope are per model, and a 2025 METR research note moved to a shared slope across models. The new paper treats this shared-slope version as its baseline. Note that release dates are not in the fit. The exponential trend comes from plotting the fitted horizons against dates afterward.

The authors point out that this hides an assumption. A logistic regression on log time assumes that task difficulty for an AI grows linearly with the logarithm of human time. If that holds, every 10x step in human time costs the same amount of success probability. Nothing in the data collection guarantees it.

Relaxing the linearity assumption

The paper keeps the same data, which is 228 tasks in 79 task families and 26 models, and fits two richer statistical models. (The authors use "model" for the statistical fit and "AI" for the language models, a convention worth borrowing.)

Model 1 replaces the linear term with a monotone spline, so the link between human time and difficulty can bend.

Model 2 is an explanatory item-response theory (IRT) model. Item-response theory is the machinery behind standardized tests: it jointly estimates a respondent's ability and each item's difficulty from a grid of right and wrong answers. Here each task gets a latent difficulty. That latent value is regressed on human time through the spline, plus a task-family effect and a per-task residual. The spline can be read as a conversion function from human minutes to AI difficulty.

Model 2 also addresses a correlation problem. If runs were independent given the fitted probabilities, about 60% of a model's run sets on a task would be unanimous, either all successes or all failures. In the data, 83% are. Model 2 includes a correlation parameter that downweights task-model pairs with many redundant runs. The family effect replaces the heuristic square-root reweighting that METR used so large families did not dominate.

To compare fits, the authors use 5-fold cross-validation split by task family, so the held-out data comes from families the fit never saw. They score with proper scoring rules (log score, Brier score, and an elementary score at q = 0.5 and 0.8 that directly judges the horizon estimate), crossed with three weighting schemes. They report that their time-horizon estimates perform better under this suite than the baselines.

The flat region between 2 and 30 minutes

The most interesting result is the shape of the fitted conversion function. It is nearly flat from roughly 2 to 30 minutes of human time, and close to linear outside that range. In practical terms, tasks in the 2-30 minute band have similar AI difficulty even though their human times differ by 15x. Past 30 minutes, difficulty climbs with time again.

The consequence is that equal multipliers are not equal progress. The authors' example: a horizon jump from 3 minutes to 30 minutes is much easier than one from 30 minutes to 5 hours, despite both being 10x. Under the linear assumption they look identical on the plot. The paper's conditional success trajectory plot suggests a move from 4 to 15 minutes may be quite small in difficulty terms.

One plausible reading, offered here as interpretation rather than a finding from the paper, is that short tasks mostly test whether a model can follow instructions and call tools without slipping, while longer tasks add planning, recovery from errors, and context management. If the cost of those capabilities does not scale smoothly with duration, a single slope will blur them together.

The authors do not challenge the exponential growth of the plotted points. Their claim is about interpretation: the same visual slope can hide different amounts of capability gain depending on where it occurs.

What this changes for engineers

First, treat a time horizon as an estimate with a modeling choice behind it. If you cite "the model can do 2-hour tasks," ask which fit produced it and whether the tasks near that length are dense in the benchmark. A horizon estimated where tasks are sparse leans heavily on the assumed functional form.

Second, if you build your own agent evals and want to report capability in human-time units, the method transfers. The authors note that it applies to any IRT setup anchored on an external scale. METR's write-up of the original work is a good starting point for the task-timing protocol. About 29% of the tasks in the new analysis rely on an expert estimate of human time because timed attempts were missing or invalid, so measurement error in the x-axis is a real concern.

Third, use the diagnostic plots. The paper recommends reading time horizons together with the conversion plot and the conditional success trajectory, especially as benchmarks add longer tasks. A suite with few tasks beyond several hours cannot say much about progress there.

This is a preprint, a day old at the time of writing, and I have not reproduced the fits. The analysis covers one task suite, and the authors reserve the question of how these results extend to other domains and longer tasks for discussion. The spline's flat region is estimated from a finite number of tasks, so its exact boundaries carry uncertainty.

Conclusion

The time-horizon plot is useful because it puts capability in units people understand. The new paper's contribution is to show that the conversion from those units to difficulty is not a straight line, and to supply tools to check it. For practitioners the takeaway is simple: use the horizon number, but check which part of the human-time scale it sits in before comparing two jumps.

Fault-Tolerant Foundation Models: Training LLMs to Thrive on Unreliable Hardware

Prabhakar Chaudhary — Thu, 08 Oct 2026 19:27:54 +0000

Fault-Tolerant Foundation Models: Training LLMs to Thrive on Unreliable Hardware

The current trajectory of large language model (LLM) development is hitting a physical limit. As we scale models to trillions of parameters, the infrastructure required to support them has become increasingly fragile. A single bit-flip in a high-bandwidth memory (HBM) module or a transient voltage drop in a GPU cluster can derail a multi-week training run, leading to NaN gradients or total model collapse. Traditionally, the industry has solved this through extreme hardware redundancy and strict error-correcting codes (ECC), but this comes at a massive cost in both energy and silicon area.

Recent research, specifically the work on fault-tolerant foundation models, suggests a different path. Instead of demanding perfect hardware, we can design models that are inherently resilient to errors. By shifting the burden of reliability from the hardware layer to the algorithmic layer, we can run AI workloads on "noisier," more energy-efficient chips without sacrificing performance.

The Fragility of Massive-Scale Training

In a typical data center, hardware reliability is treated as a binary: either a component is working perfectly, or it is considered failed. For small-scale computing, this is a reasonable assumption. However, when you coordinate 50,000 GPUs for months at a time, "rare" hardware faults become daily occurrences. The probability of a silent data corruption (SDC) event—where data is altered without triggering an error—increases linearly with the number of components and the duration of the task.

Currently, when a fault is detected, the system typically halts and restarts from the last checkpoint. If a fault is not detected (an SDC), the error propagates through the network, potentially poisoning the weights. This fragility forces hardware designers to include significant overhead. Estimates suggest that up to 30% of a modern AI chip's energy budget is spent on maintaining signal integrity and error correction. If we could eliminate that requirement, we could theoretically increase compute density by an order of magnitude.

Training for Resilience

The core insight of fault-tolerant training is that neural networks are already somewhat robust to noise. Dropout, quantization, and stochastic depth are all techniques that introduce intentional "errors" during training to improve generalization. Fault-tolerant foundation models take this a step further by treating hardware faults as just another form of noise to be optimized against.

Researchers have found that by simulating hardware unreliability during the pre-training phase, models develop a form of "architectural immunity." For example, if a model is trained with a certain percentage of random bit-flips in its activations or weights, it learns to distribute information more redundantly. Instead of relying on a few high-magnitude weights, the model develops a more diffuse representation where no single component is critical for the final output.

This approach is mathematically grounded in decentralized SGD under heavy-tailed noise. By applying gradient clipping and normalization, the optimization process can remain stable even when the underlying hardware produces extreme outliers. The model doesn't just survive the noise; it learns a manifold that is robust to the specific failure modes of the silicon it runs on.

The Role of Stochastic Approximation

One of the technical challenges in this field is distinguishing between a useful gradient signal and hardware-induced noise. This is where nonlinear two-timescale stochastic approximation becomes relevant. In a fault-tolerant setup, the optimizer must operate on two levels: one that tracks the moving average of the weights (the long-term signal) and another that filters out the transient errors (the noise).

By using a two-timescale approach, the model can effectively "extrapolate" through periods of high hardware error. If a specific cluster of neurons starts producing garbage output due to a localized hardware fault, the optimizer can temporarily down-weight those signals until the hardware stabilizes or the model re-routes the computation. This is analogous to how biological brains remain functional despite the constant death of individual neurons.

Beyond Error Correction: The Energy Dividend

Why go through the trouble of building fault-tolerant software? The answer lies in the energy dividend. Modern CMOS hardware is reaching a point where further reductions in voltage lead to exponential increases in error rates. This is known as the "Vmin wall." If we insist on zero-error hardware, we cannot lower the voltage further, and thus we cannot reduce power consumption.

A fault-tolerant model can run at "sub-threshold" voltages. In this regime, the hardware is significantly faster and more efficient but produces errors at a rate that would crash any traditional software. By using these models, we can potentially reduce the carbon footprint of AI training by 40% or more. This is not just a marginal improvement; it is a fundamental shift in how we think about the relationship between software and the physical machines that run it.

The Future of Specialized AI Hardware

As we look toward the next generation of AI accelerators, we are seeing a trend toward specialized silicon that prioritizes throughput over precision. Low-precision formats like FP4 and INT8 are already standard, but fault-tolerant research opens the door to even more radical designs, such as analog computing or neuromorphic chips.

Analog systems, for instance, are notoriously difficult to program because they are sensitive to temperature and manufacturing variations. However, a model trained with fault-tolerant principles is perfectly suited for an analog environment. It treats the physical variation of the chip as just another set of parameters to be calibrated during a quick fine-tuning step. This could lead to a new era of "co-designed" AI, where the hardware and software are evolved together to reach peak efficiency.

Conclusion

The transition to fault-tolerant foundation models represents a maturing of the field. We are moving away from the era where AI was a guest on general-purpose hardware and into an era where models are deeply integrated with the messy, noisy reality of physical systems. By embracing unreliability, we can build AI that is not only larger and more capable but also more sustainable.

The next time a GPU cluster suffers a minor hiccup, the models of the future won't crash. They will simply keep learning, indifferent to the chaos underneath.

Sources:

nanoMuse Is a System, Not a Model: Building a Personal Agent Across Your Devices

Prabhakar Chaudhary — Thu, 08 Oct 2026 16:54:03 +0000

Persistent Agents Are a Systems Problem

A persistent personal agent must retain context, operate across devices, reach applications, continue working when its interface is closed, and remain accountable for consequential actions.

That is the context for nanoMuse’s October 6, 2026 paper, written by Guangyi Liu, Yong Liu, and Jiangning Zhang. nanoMuse is presented not as another language model, but as an end-to-end software system in which the model is replaceable.

Unlike model-centered projects such as Meta Muse Glimmer-30B, nanoMuse supplies surrounding machinery: persistent identity, execution environments, routing, memory, interfaces, schedules, and approval boundaries.

The complete implementation is published in the official nanoMuse repository.

Architecture: One Account, Multiple Agents

nanoMuse describes one personal agent from the user’s perspective, but each device runs its own agent runtime. After sign-in, those runtimes meet through a relay and share a conversation.

That design avoids treating every phone or computer as a thin terminal for one remote virtual machine. A task can be entered on a phone and executed on a computer. Prefixing a message with a target such as @Mac routes it to that device, while approval requests can return to the device currently in the user’s hand.

The relay carries conversation text and coordinates devices, while files and screenshots remain where they were created. It can use the project’s community infrastructure or a user-controlled server.

This boundary is not an automatic privacy guarantee. Self-hosting controls message routing, but an externally hosted model provider may still receive model requests. Relay location, model location, and device execution are separate decisions.

Device-Local Execution

Phone and Desktop Runtimes

On Android, nanoMuse places an Alpine Linux environment under proot inside the APK, with a shell, browser, MCP support, and skills. The project credits OpenMinis as the foundation for this on-device agent.

On desktop, nanoMuse uses DeepSeek Harness with a Python runtime for screen-operation capabilities. The desktop application targets Windows, macOS, and Linux. The wider project also includes iPhone, iPad, and browser clients, although the repository’s comparison table says screen-based “hands” are not available on iOS.

These runtimes give the system more than a chat interface. The agent can use a Linux shell, browser, MCP servers, and application-specific skills. It can also schedule routines, check goals, and produce a feed while the visible application is closed.

The Relay Is Coordination, Not Execution

The relay joins devices and lets them request work from one another, but execution remains associated with a device runtime or configured server environment.

A disconnected device may be unable to receive a routed task, and a local shell is not remotely reachable without the relay path. Moving coordination to a personal server also does not make every dependency local.

Memory You Can Inspect

nanoMuse stores its identity, user knowledge, and wake schedule in Markdown files. Users can inspect and correct those files directly rather than relying entirely on an opaque memory store.

The paper nevertheless treats stronger memory provenance as roadmap work. Editable Markdown is not a completed system explaining when, why, and from where every remembered statement originated.

The repository contains a working system, but the paper identifies evaluation, provenance, and open screen-operation models as continuing work.

Giving the Agent “Hands”

Many applications do not expose a suitable API. nanoMuse addresses that gap with “hands”: screen-based operation for Android applications and desktop windows.

The desktop implementation builds on UI-TARS-desktop, while other acknowledged projects contribute phone-operation components and traces. Screen interaction lets the agent work through visible interfaces when no dedicated integration is available.

It also introduces uncertainty. Graphical interfaces change, and the project’s paper lists an evaluation suite for the hands as roadmap work. The available material does not establish that screen operation is uniformly reliable across applications or workflows. Developers should treat it as an execution mechanism requiring observation and recovery, not as equivalent to a stable API contract.

Sentinel and Consequential Actions

Every action is described as passing through a Sentinel. The repository says nanoMuse pauses before deleting, sending, or paying, with approvals scoped for one occurrence, the current chat, or always.

Passwords and verification codes remain for the user to type. Logins and CAPTCHAs can be handed back to the user, after which selecting “Done” resumes the task.

This approval model places a human checkpoint before actions the project classifies as difficult to undo. It does not eliminate risk. Broad “always” approvals trade repeated interruptions for a larger standing authorization, while shell access, browser control, and screen operation create a wider potential impact than chat alone.

These are project-described controls, not evidence of independent security verification. Anyone deploying the system should evaluate relay exposure, provider selection, device permissions, approval scope, and the consequences of an agent operating personal interfaces.

What Is Actually Open and Self-Hostable?

The repository says the phone app, desktop app, web console, and relay are included under GPL-3.0-or-later. Users can run their own relay, use their own model-provider keys, or configure a custom runtime for the web application.

That is substantial system-level openness, but it does not mean every possible deployment is fully local. The community relay is a hosted service with a model-use allowance, and many supported model choices are external providers. The model itself is deliberately interchangeable rather than supplied as one mandatory open component.

The paper also places an open model for screen operation, provenance-aware memory, and a hands evaluation suite on the roadmap. Those items should not be reported as finished merely because the surrounding applications and relay are available.

Why Developers Should Care

nanoMuse offers a concrete architecture for a problem often reduced to model selection. Persistent agents also require routing, device identity, local execution, memory formats, approvals, scheduling, and fallbacks for interfaces without APIs.

Developers can inspect where conversation text travels, where files remain, which device executes a task, what users can edit, and when approval is required.

nanoMuse does not settle the reliability or security questions raised by access to personal devices. It provides an open, self-hostable system in which those questions can be examined beyond the model layer.

LoGRA: How Low-Rank Gradient Sketches Make LLM Reinforcement Learning Fit on Real Hardware

Prabhakar Chaudhary — Wed, 07 Oct 2026 16:05:10 +0000

LoGRA: How Low-Rank Gradient Sketches Make LLM Reinforcement Learning Fit on Real Hardware

Reinforcement learning post-training has become one of the most effective ways to improve large language model reasoning. Models like DeepSeek-R1 demonstrated that RL-based fine-tuning can unlock capabilities that supervised learning alone cannot reach. The catch: RL post-training is extraordinarily memory-hungry. Gradients and optimizer states can consume several times more GPU memory than the model weights themselves, putting serious RL runs out of reach for most practitioners.

A new paper from NVIDIA and collaborators, LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches, proposes a practical solution. By replacing full gradient buffers with compact low-rank representations — and pairing that compression with a stability mechanism called predicted-KL step control — LoGRA cuts average training memory by up to 45.7% without sacrificing task performance. The headline result: a 27B-parameter model trained stably for over 1,100 steps on a single eight-GPU node, a configuration where standard dense Adam simply runs out of memory.

Why RL Post-Training Is So Expensive

To understand what LoGRA solves, it helps to understand where the memory goes during RL training. When you fine-tune a model with a method like GRPO (Group Relative Policy Optimization), the training loop involves:

Rollout generation — the model samples responses, which are scored by a reward signal.
Gradient computation — backpropagation through the policy model to compute parameter gradients.
Optimizer state storage — Adam-style optimizers maintain first and second moment estimates for every parameter.

For a 7B-parameter model trained with dense Adam, the gradient and optimizer state buffers alone can push peak memory to over 30 GiB per GPU — before accounting for activations or the rollout buffer. Scaling to 27B makes this completely infeasible on a single node.

Existing workarounds like Fully Sharded Data Parallel (FSDP) and CPU offloading help by distributing or relocating state, but they don't reduce the fundamental size of the gradient tensors. LoGRA attacks the problem at the source.

The Core Idea: Gradient Sketches

LoGRA's central contribution is replacing full gradient matrices with low-rank sketches during backpropagation. Here's the mechanics:

For a weight matrix W of shape d × k, the true gradient G has the same shape. Instead of storing all d × k values, LoGRA samples a random projection matrix A of shape r × k, where r ≪ min(d, k), and accumulates a sketch:

S = G · Aᵀ   (shape: d × r)

When it's time to update the weights, the sketch is expanded back:

Ĝ = S · A   (rank-r approximation of G)

This reduces gradient storage from d × k to d × r — a factor of k/r reduction. The same compact sketch is also used for policy synchronization across nodes in distributed training, extending the memory savings to communication costs.

The approach is applied selectively to attention and MLP projection matrices, which account for the bulk of gradient memory. The authors pair this with a custom optimizer called RowAdam, which maintains running estimates of squared row magnitudes without first-moment momentum — a design that fits naturally with the low-rank update structure.

Keeping Training Stable: Predicted-KL Step Control

Gradient compression introduces a risk: an approximate gradient might produce an update that is too large, causing the policy to shift dramatically and destabilize training. This is a real concern in RL settings, where the reward landscape is non-stationary and large policy jumps can be catastrophic.

LoGRA addresses this with predicted-KL step control. Before applying any weight update, the method estimates how much the model's next-token probability distribution would change — measured by KL divergence — if the sketched update were applied at full scale. It then computes a scaling factor α that keeps the actual KL shift within a prescribed budget δ:

W ← W - α · η · Ŝ · A

where η is the nominal learning rate and α is adjusted so the predicted KL divergence stays below δ. This acts as a trust-region constraint, but one that is computed cheaply from the sketch itself rather than requiring a separate forward pass.

The result is that LoGRA can apply aggressive gradient compression without the training instability that would normally follow. The predicted-KL controller absorbs the approximation error by being conservative about step size when the sketch suggests a large policy shift.

What the Numbers Show

The authors evaluated LoGRA on Qwen models ranging from 1.5B to 27B parameters, using mathematical reasoning benchmarks. The memory results are concrete:

Model	Dense Adam	LoGRA	Reduction
1.5B	9.18 GiB	7.18 GiB	21.8%
7B	31.82 GiB	17.29 GiB	45.7%

LoGRA also compares favorably against LoRA. For the 1.5B model, LoGRA uses 7.18 GiB average versus LoRA's 13.21 GiB — LoRA adapts only a subset of parameters but still requires full gradients through frozen layers during backpropagation.

The 27B experiment is the most striking. Dense Adam fails with an out-of-memory error on a single eight-GPU H100 node. LoGRA trains stably for over 1,100 steps, with the held-out macro score improving from 39.69% to 62.94%.

How LoGRA Relates to GaLore

Readers familiar with GaLore (Gradient Low-Rank Projection) will notice the conceptual overlap. GaLore also uses low-rank gradient projections, but it targets pre-training and supervised fine-tuning, periodically recomputing the projection subspace via SVD.

LoGRA is specifically engineered for RL post-training, where the gradient landscape shifts continuously as the policy evolves. The predicted-KL step control is the key addition that makes low-rank compression viable in this noisier setting. The random projection matrix A also requires no periodic recomputation — simpler to maintain than GaLore's SVD-based subspace.

Practical Implications

LoGRA is implemented in the Molt library, making it accessible for practitioners who want to run RL post-training on models that would otherwise require multi-node clusters. The method is complementary to system-level techniques like FSDP and CPU offloading — you can combine them for even larger models.

The broader significance is that RL post-training, which has driven some of the most notable reasoning improvements in recent LLMs, becomes more accessible. A team with a single eight-GPU node can now run meaningful RL experiments on a 27B model. That's a meaningful shift in who can participate in this kind of research and development.

Conclusion

LoGRA addresses a real bottleneck in LLM development: the memory cost of reinforcement learning post-training. By sketching gradients into low-rank representations and using predicted-KL step control to keep training stable, it achieves up to 45.7% memory reduction while maintaining task performance. The ability to train a 27B model on a single node where dense Adam fails is a concrete demonstration of what this unlocks.

For practitioners working on reasoning-focused LLMs, LoGRA is worth evaluating as a drop-in replacement for standard optimizers in RL post-training pipelines. The paper and Molt library provide the implementation details needed to get started.

Holo4: How H Company Built a Single Open-Weight Agent That Clicks, Codes, and Calls APIs

Prabhakar Chaudhary — Mon, 05 Oct 2026 16:08:10 +0000

Holo4: How H Company Built a Single Open-Weight Agent That Clicks, Codes, and Calls APIs

Most agentic AI models are specialists. A GUI-focused model can click through a browser but is lost when the task requires an API call. A tool-calling model can chain function calls but has no idea what to do when it needs to interact with a desktop application that has no API. Real work rarely respects these boundaries — a single business task might require reading a PDF, filling out a web form, querying a database via MCP, and writing a script to process the result.

H Company's Holo4, released on September 28, 2026, is a direct attempt to close that gap. It is a series of open-weight agentic models — available in 27B dense and 35B-A3B Mixture of Experts configurations — that interact with software through any available interface: GUIs, code execution, MCP servers, and REST APIs. The same model, called the same way, regardless of platform.

The Interface Unification Problem

The fragmentation in agentic AI is not accidental. Training a model to reliably click through a GUI requires a very different kind of data than training it to chain API calls. Most labs have solved this by building separate models for separate contexts, or by routing between specialists at inference time.

Holo4 takes a different approach: train a single model on all interface types simultaneously, using a shared action space. The model decides at each step which interface to use — screenshot-based GUI interaction, code execution in a sandbox, MCP tool calls, or direct API requests — based on what the task requires. H Company reports that this unified approach outperforms their previous models on every benchmark they tested, including OSWorld (85.2% for the 27B model), AndroidWorld (85.1%), and AutomationBench (45.4%).

The Agentic Task Factory

The most technically interesting part of Holo4 is not the model architecture — both sizes use Qwen3.8 as the base — but the training data pipeline H Company calls the Agentic Task Factory.

The factory is a set of agentic pipelines that build interactive environments and verifiable tasks from documentation alone: product manuals, help center articles, screenshots of real websites, and open-source software. It has produced approximately 10,000 tasks across three categories: web app tasks (40%), MCP server tasks (30%), and desktop/OS tasks (30%).

What makes this pipeline notable is its quality gate. A task is only kept if:

The verifier fails on the untouched seed state (confirming the task is not trivially solved)
The verifier passes on the golden state (confirming the task has a correct solution)
The verifier rejects every near-miss (confirming the verifier is not too permissive)
An agent can actually solve the task through the real interface

Failed attempts loop back into an audit and hardening process rather than being discarded. This is a meaningful departure from the common practice of filtering trajectories by outcome alone — the factory is designed to produce tasks that are genuinely hard and genuinely solvable, not just tasks that happened to be completed.

Training: Two RL Experts, One Merged Model

Holo4's training pipeline has three stages. First, supervised fine-tuning on 127B tokens, roughly 75% of which are successful agentic trajectories from the Task Factory, with the remainder covering multimodal reasoning, GUI grounding, and text-only tool use.

Second, asynchronous online reinforcement learning on long-horizon tasks. Rather than training a single RL policy, H Company trains two specialized LoRA experts: one for desktop and web environments, one for terminal, MCP, and API environments. This separation allows each expert to develop strong priors for its domain without the two domains interfering with each other during RL.

Third, both experts are merged back into the fine-tuned base model with equal weight. The result is a single model that carries the learned behaviors of both specialists.

The HuggingFace model card notes that the training token distribution reflects this multi-domain focus: desktop (45%), web (14%), MCP/API (12%), mobile (3%), and other (26%).

Benchmark Results and Cost Efficiency

On OSWorld 2.0 — a benchmark focused on long, multi-step desktop workflows — Holo4 27B scores 61.7% at an estimated cost of $1.22 per task. For comparison, Claude Opus 5.5 scores 81.8% and GPT-6 Astra scores 73.5%, but both are closed models with significantly higher inference costs. The Holo4 35B-A3B MoE model scores 30.9% on the same benchmark at $0.61 per task, making it the more cost-efficient option for high-volume deployments.

On AutomationBench, which tests business automation workflows, Holo4 27B scores 45.4% — competitive with GPT-5.6 Sol (45.8%) and Kimi K3 (46.7%), and above Claude Opus 5 (50.3% is the frontier here). The 35B-A3B model scores 34.5% at $0.02 per task, which is a meaningful cost advantage for production use cases.

H Company has published all trajectories behind their benchmark scores publicly, which is unusual in the agentic model space and allows independent verification of the reported numbers.

The Harness Matters as Much as the Model

H Company rebuilt their execution harness alongside the model — the loop that executes actions and manages context over hundreds of steps. The two largest changes were giving the agent a reliable memory capable of tracking hundreds of steps, and providing it with a shell on the desktop machine itself.

Long-horizon agentic tasks fail in ways that are not always the model's fault. A model that loses track of what it has already done, or cannot recover from a failed action, will fail tasks it could theoretically complete. The harness improvements are likely responsible for a meaningful portion of the OSWorld 2.0 gains.

Availability and Licensing

Both Holo4 sizes are available on the H Models API and as open weights on HuggingFace in BF16, FP8, NVFP4, and 4-bit GGUF formats. The 27B model is released under CC BY-NC 4.0 (research use). The 35B-A3B model is under Apache 2.0, making it suitable for commercial deployment. API pricing is $0.40 per million output tokens for the 27B and $0.30 for the 35B-A3B.

H Company also released Holotron4 Nano, applying the same post-training stack to NVIDIA's Nemotron 3 Nano Omni model. On OSWorld CUA GUI, Holotron4 Nano improves over the base model by 55.3 percentage points — suggesting the training recipe transfers well to smaller base models.

Practitioner Implications

For teams building agentic workflows, Holo4 offers concrete advantages over current alternatives. The unified interface model eliminates the need to route between specialists, simplifying harness design and reducing routing errors. The Apache 2.0 license on the 35B-A3B model makes it viable for production deployment, and the published trajectories provide a starting point for domain-specific fine-tuning.

The gap to frontier closed models on long-horizon benchmarks (61.7% vs. 81.8% on OSWorld 2.0) is real. For tasks requiring sustained multi-step reasoning over hundreds of actions, closed frontier models still have an edge. But for well-defined, verifiable business automation tasks, Holo4's cost-performance profile is competitive.

The Agentic Task Factory is the most transferable idea here. Generating verifiable tasks from documentation — with quality gates that test both solvability and verifier precision — is applicable to any domain where you need agentic training data at scale. It is a more principled approach than filtering model rollouts by outcome alone.

AstaBrief 8B: How AllenAI Trained a Small Open Model to Generate Cited Scientific Reports

Prabhakar Chaudhary — Fri, 02 Oct 2026 16:04:21 +0000

AstaBrief 8B: How AllenAI Trained a Small Open Model to Generate Cited Scientific Reports

The Allen Institute for AI (Ai2) released AstaBrief 8B on October 2, 2026 — an open-weights model that takes a research question and a set of retrieved literature excerpts and produces a fully cited scientific report in a single forward pass. The model is now live as "Fast mode" inside Asta, Ai2's agentic platform for scientific work, and the weights, training data, and example workflows are all available on Hugging Face under an Apache 2.0 license.

The release is interesting not just because of what the model does, but because of what Ai2 learned while building it: the most important lever for improving citation quality in a fine-tuned report-generation model turned out to be a simple data-filtering heuristic, not a more sophisticated training algorithm.

What AstaBrief Actually Does

Asta's report-generation feature previously ran entirely on a Claude-powered "Thinking mode" pipeline. That pipeline summarizes retrieved snippets, clusters them by theme, and writes the report section by section — a multi-step process that averages 178.5 seconds per report.

AstaBrief replaces that pipeline for users who want speed. Given the same research question and retrieved excerpts, it writes the full report in one pass, averaging 51.1 seconds — roughly 3.5 times faster. The model is built on Qwen3-8B and fine-tuned specifically for the scientific report-generation task.

Because the weights are open and the model is small enough to run on a single GPU, institutions can deploy it behind their own firewall — useful when research questions involve sensitive or unpublished work that cannot be sent to a third-party API.

The Training Recipe: SFT Then DPO, Not RL

Ai2 considered reinforcement learning for training AstaBrief, citing its own earlier DR Tulu work as evidence that RL can improve long-form report generation in open-weights models. The team ultimately chose a simpler two-stage recipe: supervised fine-tuning (SFT) followed by direct preference optimization (DPO).

The reasoning was practical. RL training is unstable and expensive; SFT plus DPO is cheaper, easier to debug, and faster to iterate on. For a model that needs to be shipped as a production feature, those properties matter.

SFT stage: Ai2 started with real user queries submitted through the ScholarQA framework that underpins Asta. After filtering for quality, relevance, and privacy — removing bot traffic, very short queries, non-English requests, and prompts containing personal information — the team had a pool of 90,000 research-focused queries. Full-report targets were generated using a mix of proprietary systems: Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4-mini, and GPT-4.1. Quality filtering reduced this to 47,000 usable training examples.

DPO stage: Preference pairs came from a separate query subset not used during SFT. Each pair consisted of one report from the ScholarQA pipeline (typically Claude 3.5 or 3.7 Sonnet) against a report from o3, o4-mini, DeepSeek-V3, or DeepSeek-R1. Two judge models — GPT-4.1 and DeepSeek-R1 — independently picked a winner for each pair. Only pairs where both judges agreed were kept, producing approximately 6,000 final examples. Ai2 reports that the judges agreed with human preferences 95% of the time.

DPO training ran on 8×H100 GPUs using Ai2's open-instruct framework, starting from the AstaBrief-8B-SFT checkpoint.

The Key Finding: Citation Density Filtering

Ai2 tested four statistics-based filters on the synthetic training reports before using them for SFT:

Output-to-input token ratio — whether the report was appropriately long relative to the input
Citation relevance — whether cited papers were actually relevant to the claims they supported
Citation density — the proportion of statements in the report that had at least one supporting citation
Citation diversity — whether the report cited a range of sources rather than repeating the same few

The strongest quality gains came from filtering out reports with low citation density. Reports with large stretches of unsupported text — even if they were otherwise well-written — degraded the model's ability to produce grounded output. Removing them improved both answer precision and citation quality in the resulting model more than any other single intervention.

The broader lesson Ai2 draws from this is that scientific specialization in a fine-tuned model is not primarily a function of adding more scientific text to pretraining. The composition and quality of post-training data — specifically, whether the training targets model the behavior you actually want — matters more.

Evaluation Results

Ai2's primary evaluation target was SQABench-CS2, a set of 200 user-written computer science research questions. The model is scored on four metrics: rubric score (necessary content coverage), answer precision (paragraph relevance), citation precision (citation support), and citation recall (claims support).

On the ScholarQA-CS2 test set, AstaBrief 8B averaged 87 across tracked metrics, compared with 83.7 for the SFT-only checkpoint and 77.3 for base Qwen3-8B. In LLM-judged pairwise comparisons against the Claude-powered Asta pipeline, AstaBrief 8B won 55% of the time on the development split and 72% on the test split.

A separate human study with three scientific researchers found that two of the three preferred AstaBrief over the other systems specifically on citation accuracy — the metric that the citation density filtering was designed to improve.

Ai2 notes that most training and evaluation was completed in 2025 and that it has not rerun the full evaluation against current frontier models, so direct comparisons to the latest Claude or GPT releases should be treated with caution.

Early Usage Patterns

Among 374 Asta users who tried Fast mode, 29.1% used it on two or more days. Users generated an average of 3.67 report threads. Twenty-three percent never switched back to Thinking mode for future threads; another 18% alternated between modes. Positive feedback ran at 84.2% for Fast mode versus 85.2% for Thinking mode — a gap small enough that Ai2 considers the quality roughly comparable for most use cases.

What Comes Next

Ai2 describes AstaBrief as one experiment in a longer line of work running from ScholarQA and DR Tulu toward future versions of OLMo. Stated next steps include more fine-grained preference learning, stronger RAG-plus-RL approaches, multi-turn and multi-tool capabilities, additional scientific data sources, and query decomposition.

The release also fits into Ai2's broader NSF OMAI initiative — a U.S. national effort to build fully open AI infrastructure for scientific discovery. AstaBrief is positioned as a practical demonstration that a small, open, specialized model can close most of the quality gap with a proprietary pipeline while running faster and at lower cost.

For practitioners building retrieval-augmented generation pipelines for scientific or technical domains, the citation density finding is the most transferable takeaway: if your training data contains large stretches of unsupported claims, filtering those examples out before fine-tuning is likely to improve grounding more than switching to a more complex training algorithm.

The model weights, SFT and DPO training datasets, and an example local-report workflow are available at Ai2's Hugging Face organization.

AREX-2: Advancing Self-Improving LLM Agents via Long-Horizon Reflection

Prabhakar Chaudhary — Thu, 01 Oct 2026 17:15:41 +0000

Beyond One-Shot Success: How AREX-2 Teaches LLM Agents to Reflect and Persevere

Current autonomous LLM agents are often evaluated by their ability to solve a task in a single pass or through a short sequence of scripted interactions. While models like GPT-4o and Claude 3.5 have shown impressive capabilities in these one-shot scenarios, they frequently struggle when faced with long-horizon problems that require sustained reasoning, iterative debugging, and genuine self-correction. When a plan fails at step 20, most agents either loop endlessly or hallucinate a success state that doesn't exist.

A new paper, "AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks" (arXiv:2609.38288), addresses this bottleneck by focusing on the two pillars of agentic reliability: reflection and long-horizon execution. By training on a specialized dataset of verifiable improvement trajectories, the researchers have developed an agent that not only achieves state-of-the-art results on technical benchmarks but also continues to improve its solution as you give it more time to think.

The Scaling Law of Test-Time Reflection

The core hypothesis of AREX-2 is that the ability to self-improve is a domain-agnostic skill that can be decoupled from raw knowledge. The authors define self-improvement as the capability to iteratively refine a solution at test time, similar to how a human developer might write a script, run it, see an error, and then systematically fix it over several hours.

This process relies on two distinct but complementary mechanisms:

Reflection: The cognitive ability to analyze a current (potentially flawed) solution and propose a specific, actionable improvement.
Long-Horizon Execution: The stability required to maintain this iterative loop over many rounds without the agent's logic collapsing or the context becoming "contaminated" by previous failures.

While many agents can "reflect" (e.g., using a Reflexion-style prompt), they often plateau after one or two rounds. AREX-2 is designed to break this plateau, demonstrating that an agent's performance should ideally scale with the number of allowed refinement rounds.

The Training Recipe: Verifiable Synthesis

One of the largest hurdles in training self-improving agents is the lack of high-quality data. Human-written logs of long-term debugging are rare and often messy. To solve this, the AREX-2 team synthesized a massive corpus of long-horizon improvement trajectories from two specific domains: machine learning engineering and algorithmic programming.

These domains were chosen because they offer something that open-ended chat does not: verifiable feedback. In ML engineering (using benchmarks like MLE-bench), an agent can run code, check the loss curve, and see a public leaderboard score. In programming, it can run unit tests. This allows the researchers to automatically label "improvement" as a trajectory where the reward (accuracy, loss reduction, or test pass rate) increases over time.

The synthesis process involved:

Starting with a "teacher" model that generates multiple candidate solutions for a task.
Selecting trajectories where a solution starts poor but becomes significantly better through a series of "thoughts" and "edits."
Filter for trajectories where the reasoning path is logical and non-redundant.

By pretraining on these verifiable trajectories, the base model—a Qwen3.8-27B backbone—learned the general pattern of how to recover from errors and how to manage its own internal state over long sequences.

Benchmarking the "Think-Harder" Effect

The results of AREX-2 are particularly striking because of where they excel. The agent was tested on MLE-bench Lite, an evaluation suite designed to test real-world ML engineering skills like data cleaning, feature engineering, and hyperparameter tuning. It achieved a score of 81.8, which places it at the very top of autonomous agents in the 30B parameter class.

Even more impressive is the model's performance on general-purpose benchmarks that it wasn't specifically trained for. On GAIA (General AI Assistants), which requires the agent to browse the web, use tools, and solve multi-step riddles, it reached 92.2. On DeepSearchQA, a benchmark for complex information retrieval, it scored 93.8.

However, the most important chart in the paper is not a single score but the "performance vs. rounds" curve. While baseline models like GPT-4o often see their success rate flatten out after 5–10 rounds of refinement, AREX-2 maintains a positive slope even up to 30 rounds. This suggests that the training successfully instilled a "long-horizon" utility that allows the model to benefit from extra computation time—a form of test-time scaling that is increasingly seen as the next frontier for LLM performance.

Beyond Context Contamination

Why do other agents fail where AREX-2 succeeds? The authors point to a phenomenon known as "context contamination." As an agent interacts with a terminal or a browser, its context window fills up with error messages, retries, and failed attempts. For most models, this "noise" eventually overpowers the original task instructions, leading the model to lose focus or begin repeating itself.

AREX-2 uses a specialized Reflective-Self-Correction (RSC) loop. Each time the agent reflects, it doesn't just append a new thought to the end of the history. It produces a distilled "state update" that summarizes what was learned from the previous failure and what the new plan is. This keeps the effective context length manageable and prevents the "hallucination loop" where an agent gets stuck trying to fix a problem it has already forgotten the source of.

Implications for Autonomous Research

The success of AREX-2 has profound implications for the future of AI in science and engineering. If we can build agents that truly self-improve, we can move away from the paradigm of "AI as a chatbot" towards "AI as a project inhabitant."

An agent that can spend 48 hours iteratively improving a codebase or a research paper—constantly verifying its results against ground-truth feedback—is significantly more valuable than one that gives a fast but potentially shallow answer. AREX-2 proves that this "perseverance" is a learnable trait that can be transferred from technical domains like coding to general reasoning tasks.

The AI Data Satiation Point: Why 'Wild' AI Text is Breaking Chinchilla Scaling Laws

Prabhakar Chaudhary — Thu, 01 Oct 2026 17:13:06 +0000

The AI Data Satiation Point: Why "Wild" AI Text is Breaking Chinchilla Scaling Laws

As we approach the end of 2026, the internet looks fundamentally different than it did just two years ago. According to new research from Pangram Labs and the University of Massachusetts Amherst, nearly 31% of all web tokens collected in August 2026 are AI-generated. This isn't just about synthetic data generated in controlled laboratory settings; this is "wild" AI text—sentences, articles, and reviews written by LLMs, meant for humans, but now arriving unlabeled in global pretraining corpora like FineWeb.

For researchers building the next generation of frontier models, this presents a massive technical challenge. Does training on this wild AI text help or hurt? A new paper titled "How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text" provides a definitive, data-backed answer that moves us past the simplistic "model collapse" debates and into a new era of scaling laws.

Beyond the Model Collapse Hysteria

Earlier studies, such as the widely cited "Curse of Recursion" by Shumailov et al., warned that training models on their own output would lead to an irreversible collapse in diversity and performance. However, those studies often used specific, narrow setups where a model was fed only its own raw, unfiltered outputs.

"Wild" AI text is different. It is generated by a diverse ecosystem of models (GPT-4o, Claude 3.5, Llama 3, etc.), often edited by humans, and mixed into a massive pool of human-written content. Until now, we didn't have a reliable way to predict how this specific type of data affects model pretraining at scale. The authors of the UMass research decided to settle this by pretraining 800 separate language models, meticulously varying the ratios of human and AI-generated tokens.

The Scaling Law Reversal: From Benefit to Harm

The study’s most significant finding is that the value of an AI token is not a static number—it changes sign based on how much data the model has already seen.

For "data-starved" models—those with a very low budget of human text—adding AI-generated tokens initially lowers the loss on human-held validation sets. In these cases, the AI text provides a useful scaffold for learning grammar, basic reasoning, and information retrieval. In essence, any data is better than no data when you are starting from zero.

However, once a model is trained on a substantial budget of high-quality human text, the benefit of AI tokens saturates. When the ratio of AI tokens increases beyond a certain threshold, the benefit quickly reverses into harm. Adding more AI tokens actually increases the loss on human text, making the model less capable of understanding or generating human-like nuances.

Comparing this to fresh human tokens is enlightening: while an additional 1 billion human tokens continues to lower the loss according to traditional scaling laws, 1 billion AI tokens can actually "poison the well" for a model that has already mastered a large human corpus.

The New Math of AI Training

The researchers found that existing scaling laws, including the famous Chinchilla laws (Hoffman et al., 2022), fail to predict this behavior because they assume all tokens are of equal value. To address this, they proposed a new scaling law that includes separate "benefit" and "harm" terms.

This new law allows the marginal value of a token to change from positive to negative. When the model is small or data-poor, the benefit term dominates. As the model scales, the harm term—representing the subtle biases, lack of creativity, and repetitive structures in AI text—takes over. By fitting this law on smaller 50M parameter models, the researchers were able to predict the performance of models 3.6 times larger with incredible precision.

The "WildAI" Corpus and Pangram Detection

A key technical contribution of this work is the release of WildAI, an 83-billion-token corpus labeled by topic, format, and source (Human vs. AI). The researchers used the Pangram detector to identify AI content, finding that the percentage of AI-generated web text rose from 27.5% in June 2026 to over 31% in just two months.

This suggests that the "dead internet" is essentially here, and future pretraining data will inevitably be a mix of sources. The technical community can no longer ignore the origin of their data; the difference between a high-performing model and one that fails to generalize may come down to the precision of their AI-filtering pipelines.

Practical Recommendations for AI Engineers

If you are training an LLM today, the paper offers three critical takeaways:

Filter AI Text for Human Targets: If your goal is to excel on human-centric tasks and evaluation sets, you must filter out "wild" AI text aggressively. The researchers suggest that even the best AI text currently lags behind high-quality human text in pretraining value by a wide margin.
Repeat Human Data Before Adding AI: Counter-intuitively, it is often better to train on your high-quality human data for an extra epoch rather than expanding the dataset with suspicious AI-generated web text. The "repeating tax" is lower than the "AI harm tax" in many high-budget scenarios.
Dual Validation Tracks: Do not just report one validation loss. Report loss on human-text benchmarks and AI-text benchmarks separately. The UMass study shows that AI text remains valuable if the specific target is performing well on other AI tasks, but it is not a direct substitute for the human experience.

UniEvo-VL: On-Policy Self-Distillation for Multimodal Image Generation

Prabhakar Chaudhary — Thu, 01 Oct 2026 15:54:38 +0000

UniEvo-VL trains a unified multimodal model to internalize its own image critiques by matching a critique-conditioned EMA teacher along denoising trajectories sampled from the student’s current policy.

From inference-time reflection to learned behavior

A multimodal model that can both generate and understand images has a useful feedback loop available: generate an image, compare it with the prompt, identify discrepancies, and try again with better instructions. Self-Refine applies a related generate-feedback-refine pattern to language tasks at inference time, without additional training.

The newly published UniEvo-VL paper asks a different question: can corrective feedback become training supervision, so that future images improve even when the original prompt contains no critique?

Its answer is on-policy self-distillation. Rather than treating a revised image as a fixed target, UniEvo-VL uses the critique to construct a privileged prompt for a teacher. The student then learns to reproduce the teacher’s denoising behavior while seeing only the original prompt.

The implementation builds on Qwen-Image-2512 and uses a Qwen-VL feedback pipeline.

The training pipeline

The method alternates between acquiring corrective experiences and distilling them:

Generate and assess. Given a prompt, the model generates an image from sampled noise. In understanding mode, it compares that image with the prompt, produces a discrepancy critique, and returns a binary accept-or-reject decision.
Create privileged conditioning. For a rejected image with a valid critique, the understanding component combines the original prompt and critique into a revised, self-contained prompt.
Optionally verify the revision. The generator reuses the same noise with the revised prompt. The resulting image is assessed against the original request, and the experience is retained only if the revised result is accepted.
Sample from the student. The student runs a full denoising trajectory conditioned only on the original prompt.
Distill at matched states. At states sampled from that trajectory, the student’s denoising distribution is compared with a teacher distribution conditioned on the revised prompt.

This design separates the information available to the two roles. The student receives the ordinary user prompt. The teacher receives the critique-enriched prompt, making the correction privileged information that the student must internalize rather than depend on at inference time.

One model, two training roles

“Self-distillation” does not mean that the exact same parameter snapshot supplies both sides of an unconstrained loss. UniEvo-VL maintains student parameters and an exponential-moving-average teacher, with the appendix reporting an EMA decay of 0.999.

At each selected latent state, both networks are evaluated from the same state:

The student predicts the next denoising transition using the original prompt.
The EMA teacher predicts from that matched state using the revised prompt.
Training minimizes the divergence between those transition distributions.
A stop-gradient is applied to the teacher target, so optimization does not move both sides toward each other.
The sampled latent state is also stop-gradient, preventing the loss from backpropagating through trajectory generation.

The trajectory itself comes from the current student policy. That is the “on-policy” part: the teacher is asked what it would do at states the student actually visits, including imperfect or difficult states induced by the student’s present behavior.

This differs from distillation approaches focused on reducing sampling steps. For example, Consistency Models can distill a pretrained diffusion model to support one-step or few-step generation. UniEvo-VL instead uses privileged corrective conditioning to change what the student learns along its own multistep trajectories.

Why not train on revised final images?

A simpler proposal would be to generate an accepted image from the revised prompt and use that final image as a supervised target. The paper reports preliminary experiments with this approach, but says supervised fine-tuning on accepted refined images caused rapid student degradation.

Trajectory-level distillation supplies a different learning signal. A final image records one endpoint, but it does not specify how the denoiser should behave at the intermediate latent states the uncorrected student encounters. UniEvo-VL queries the critique-conditioned teacher directly at those matched states.

That distinction matters because a teacher trajectory generated independently from the revised prompt may move through different states. Matching only endpoints—or training on revised images—does not necessarily teach recovery from the student’s own distribution of intermediate errors. On-policy matching addresses those visited states explicitly, while the EMA and stop-gradient teacher provide a comparatively stable target.

What the reported results show

The paper reports GenEval moving from 0.747 to 0.808. GenEval is described in the UniEvo-VL paper as an object-focused framework spanning six categories.

The abstract also reports GenEval2 Soft-TIFA moving from 32.97 to 35.53. These are paper-reported results, not independently replicated measurements. There is an important qualification: the detailed table lists a base GenEval2 Native score of 32.58, a standard UniEvo-VL score of 32.37, and 35.53 for the variant using the stronger external critic GPT5.6-Luna. The abstract and table therefore do not present an entirely consistent baseline or attribution for that gain.

The table also shows that post-revision verification matters. The verification variant reaches 0.818 on GenEval, 35.07 on GenEval2, and 0.790 on OCR, compared with base scores of 0.747, 32.58, and 0.771.

Limitations and open questions

The results do not support uniform improvement across tasks. Without verification or the external critic, the table shows OCR decreasing from 0.771 to 0.761 and GenEval2 Native decreasing from 32.58 to 32.37, despite the paper’s broader narrative that direct generation improves across every reported metric. The explicit table values provide the clearer basis for interpreting these cases.

Performance also shifts unevenly by prompt difficulty. Gains concentrate on prompts the base model initially handles poorly, while easy prompts decline by 1–4 points on a normalized 0–100 scale. Better recovery on difficult compositions can therefore coexist with small regressions on already strong behavior.

Finally, the method depends on critique quality, prompt synthesis, acceptance decisions, and benchmark reliability. The paper notes that detector errors can penalize correct images, while experiments with GPT5.6-Luna indicate that a stronger critic can produce larger gains. That result also limits claims of improvement without external guidance: the strongest reported GenEval2 number uses an external critic.

UniEvo-VL is best read as a concrete training recipe for converting visual critique into denoising supervision. Its central contribution is not merely generating a better second image, but teaching the original-prompt student from a privileged teacher at the exact latent states the student currently visits.