DEV Community: Prabhakar Chaudhary

Meta Muse Glimmer-30B: How a Dense Local Model Is Rethinking On-Device Agentic AI

Prabhakar Chaudhary — Mon, 10 Aug 2026 16:22:26 +0000

Meta Muse Glimmer-30B: How a Dense Local Model Is Rethinking On-Device Agentic AI

Most open-weight models released in 2026 have been Mixture-of-Experts (MoE) architectures — large parameter counts with a small active slice per token. Meta's Muse Glimmer-30B, released on August 10, 2026, takes a different path. It is a dense 30-billion-parameter model built specifically for autonomous agentic tasks that run locally on consumer hardware. The design choices behind it — and the tradeoffs they reflect — are worth understanding in detail.

Why Dense Instead of MoE?

The dominant trend in frontier models has been MoE: route each token to a small subset of experts, keep active parameter counts low, and scale total capacity cheaply. For chat and single-turn inference, this works well. For long-running agentic workflows — where a model must maintain coherent state across dozens of tool calls, diagnose its own failures, and sustain consistent behavior over thousands of tokens — MoE routing introduces variance that compounds over time.

Muse Glimmer's dense architecture activates all ~29.6 billion parameters for every token. This eliminates expert-selection variance and routing overhead, producing more predictable latency and better long-context coherence. The tradeoff is higher memory pressure per token, which Meta addresses through quantization and a companion speculative decoding model.

Architecture: Hybrid Attention and a Dedicated Perception Encoder

The text decoder spans 52 layers and uses a repeating [Local, Local, Local, Global] attention pattern — three sliding-window attention (SWA) layers with a 2,048-token window, followed by one full-attention layer with no positional embedding (NoPE). This pattern repeats 13 times. The hybrid design keeps most attention local (cheap) while periodically allowing global context integration (necessary for long-horizon coherence).

Each attention layer uses Grouped Query Attention (GQA) with a 16:1 query-to-KV-head ratio, reducing KV cache memory by 16x compared to multi-head attention. Q-K normalization via RMS norm stabilizes attention logits across long sequences.

The model also includes a dedicated 1.8B-parameter ViT-G/14 perception encoder that handles interleaved image and video inputs. Images are patchified at 14×14 resolution; videos are sampled at 2 frames per second up to 96 frames. This makes Muse Glimmer genuinely multimodal — an agent can process screenshots, charts, and documents as part of its reasoning loop, not just text.

Speculative Decoding via DFlash

Running a 30B dense model locally at useful speeds requires more than quantization. Muse Glimmer ships with DFlash, a lightweight block-diffusion drafter that proposes 16-token blocks in parallel. The main model then verifies and accepts or rejects these proposals.

On an NVIDIA RTX 5090, DFlash delivers a 3.1x throughput improvement: 233.4 tokens/second with the drafter versus 74.9 tokens/second without it. On Blackwell Ultra hardware, NVIDIA reports sustained throughput exceeding 20,000 tokens/second per GPU at BF16/NVF4 precision. For consumer deployment, 4-bit quantization brings the model's memory footprint below 20 GB of VRAM, making it runnable on 24 GB or 32 GB cards with headroom for KV cache.

Agentic Capabilities: What "Built for Agents" Actually Means

Muse Glimmer is not a general-purpose chat model that happens to support tool calling. Its training and design are oriented around four specific agentic behaviors:

End-to-end task completion: The model is trained to pursue multi-step goals rather than respond to individual prompts.
Reliable tool use: It invokes tools with precise schemas and handles structured outputs consistently across extended workflows.
Failure recovery: When a tool call fails, the model diagnoses the error and retries with corrected parameters rather than halting.
Controllable reasoning effort: A Reasoning strength parameter (low, medium, high, xhigh) in the system prompt lets developers trade speed for quality depending on task complexity.

The model is compatible with agentic orchestration frameworks including OpenClaw and Hermes Agent, and supports the OpenAI Responses API wire format, making it a drop-in replacement in many existing pipelines.

Benchmark Performance

Against comparable-size models, Muse Glimmer-30B performs well on agentic and multimodal benchmarks:

Benchmark	Muse Glimmer-30B	Gemma4-31B	Qwen3.6-27B
MCP Atlas (general agentic)	75.5	54.2	62.5
DeepSearch QA	74.6	61.7	71.1
SWE-Bench Pro (coding)	51.2	36.9	50.2
Charxiv Reasoning (multimodal)	78.8	77.7	78.4

The MCP Atlas gap is notable: Muse Glimmer scores 21 points above Gemma4-31B on a benchmark designed to test multi-step tool orchestration. This is consistent with the model's design focus — it is not simply a larger or better-trained general model, but one where agentic reliability was a first-class training objective.

Deployment Options

Muse Glimmer is released under Apache 2.0 and supports a range of deployment stacks:

vLLM: vllm serve meta-models/Muse-Glimmer-30B
Docker: docker model run hf.co/meta-models/Muse-Glimmer-30B
llama.cpp and SGLang for CPU/mixed deployments
NVIDIA NIM container for enterprise on-premises and air-gapped environments
NVIDIA Jetson modules for edge deployment

Fine-tuning is supported via full SFT and LoRA through the NVIDIA NeMo AutoModel library, allowing post-training customization without model conversion.

What This Signals for Local Agentic AI

The release of Muse Glimmer-30B reflects a broader shift in how the field is thinking about on-device AI. The question is no longer just "can we run a capable model locally?" but "can we run a model locally that is actually designed for the reliability requirements of autonomous agents?"

Dense architectures, hybrid attention patterns, and purpose-built speculative decoding are all choices that prioritize consistent behavior over raw benchmark scores. The inclusion of a perception encoder means agents can operate on visual inputs without a separate vision model. The Apache 2.0 license removes the friction of commercial deployment.

Whether Muse Glimmer becomes a standard component in local agentic stacks will depend on how it performs in real-world workflows — the gap between benchmark performance and production reliability is still significant for agentic systems. But the design philosophy it embodies — dense, multimodal, locally deployable, and built around failure recovery — is a coherent answer to what practitioners actually need from on-device agents.

The model card and HuggingFace blog post contain full technical specifications and deployment instructions.

LFM2.5-2.6B: How Liquid AI Built an Agentic Model That Runs on Your Phone

Prabhakar Chaudhary — Fri, 07 Aug 2026 16:06:55 +0000

LFM2.5-2.6B: How Liquid AI Built an Agentic Model That Runs on Your Phone

The promise of AI agents running entirely on-device — no cloud calls, no per-token billing, no latency from a round-trip to a remote server — has been around for a while. The problem is that models small enough to fit on a phone have historically been too weak to handle the multi-step planning, tool calling, and instruction following that agentic workflows actually require.

Liquid AI's LFM2.5-2.6B, released on August 4, 2026, is a direct attempt to close that gap. At 2.6 billion parameters and under 2.5 GB of memory, it runs at 30 tokens per second on a smartphone — and on benchmarks for tool use and instruction following, it regularly outperforms models nearly four times its size.

Why On-Device Agents Are Architecturally Different

Most small models are trained to be general-purpose assistants. LFM2.5-2.6B was designed from the start for agentic workloads: multi-step planning, tool invocation, web search, and operating inside agent harnesses like OpenClaw and Hermes Agent.

The distinction matters because agentic tasks have different failure modes than single-turn chat. A model that gives a slightly wrong answer in a conversation is annoying. A model that misformats a tool call, loses track of its plan mid-task, or fails to follow a complex system prompt can break an entire automated workflow. Liquid AI's training pipeline was built around these failure modes specifically.

The Hybrid Architecture: Fewer Attention Layers, Fixed-Size State

The core architectural choice in LFM2.5-2.6B is the LFM2 hybrid design: 30 layers total, split between 22 gated short convolution blocks and 8 grouped-query attention (GQA) layers. Attention accounts for only about 37% of the model; the rest is convolution.

This ratio has a concrete engineering consequence. In a standard Transformer, every layer uses Softmax Attention, which scales quadratically with sequence length and requires a KV cache that grows with every new token. On a phone with limited VRAM, that cache becomes the bottleneck.

The LFM2 convolution blocks sidestep this entirely. Each convolution layer maintains a fixed-size state that updates per token — no attention over past tokens, no growing cache. The state at token 10,000 is the same size as the state at token 10. The specific operator is called an LIV (Linear Input-Varying) convolution: a linear-time operator whose behavior adapts to the current input.

Liquid AI ran a hardware-in-the-loop architecture search across GQA, short convolutions, linear attention, S4, Mamba, and Mamba2 to find the best layer mix for quality, latency, and memory. The result was the simplest option: mostly short convolutions, with a small number of GQA layers to handle long-range retrieval tasks where attention is genuinely needed.

A Four-Stage Post-Training Pipeline

Pre-training LFM2.5-2.6B on approximately 34 trillion tokens (with a mid-training 128K context extension phase) produced the base model. Turning that into a capable agent required four additional stages:

1. Supervised Fine-Tuning (SFT). Two consecutive SFT rounds, with the training mix weighted heavily toward agentic data: tool use, web search, software engineering, and agent traces. The mix was roughly seven times larger than what was used for the larger LFM2.5-8B-A1B model.

2. Teacher Specialization. From the SFT checkpoint, Liquid AI trained one specialist teacher per domain — math, code, tool use, instruction following, knowledge, and long context — using reinforcement learning with verifiable rewards (RLVR).

3. Multi-Domain On-Policy Distillation (MOPD). Rather than distilling from fixed teacher trajectories (off-policy distillation), MOPD lets the student model roll out under its own policy. Each prompt is routed to the appropriate specialist teacher, which provides token-level feedback on the student's actual output. This keeps the student from learning to mimic trajectories it would never generate itself.

4. Agentic Reinforcement Learning. The final stage runs multi-turn RL inside real agent harnesses. The model works through realistic productivity tasks in a dedicated sandbox, with rewards combining an LLM-as-a-judge rubric, programmatic checks, and a hard safety gate. Optimization uses GRPO (Group Relative Policy Optimization).

The Agentic RL setup is worth noting: a Harness Proxy treats the agent harness as a black box, transparently capturing token-level trajectories without modifying the harness itself. This means the model trains on the same interface developers actually use.

Benchmark Results: Where It Leads and Where It Doesn't

On instruction-following benchmarks, LFM2.5-2.6B leads the comparison group — which includes Gemma-4 models at 5.1B and 8B parameters, and Qwen3.5 models at 4.7B and 9.7B parameters. On IFBench it scores 59.17 against Qwen3.5-9B's 56.47. On Multi-IF it reaches 80.07 against Qwen3.5-9B's 62.55. On IFStruct it scores 85.49 against Qwen3.5-9B's 78.50.

Tool use tells a similar story. On ToolSandbox, LFM2.5-2.6B scores 77.83, edging out Qwen3.5-9B at 76.44 and well ahead of Gemma-4-E4B-it at 65.00. On BFCLv4 (function calling), it scores 56.88 — behind Qwen3.5-9B's 60.13, but ahead of every other model in the comparison.

The one area where larger models maintain a clear advantage is coding. On LiveCodeBenchv6, Qwen3.5-9B scores 69.86 against LFM2.5-2.6B's 59.41. Liquid AI is explicit about this: for coding-heavy agentic tasks, a larger model is the better choice.

Inference Speed Across Hardware

The memory and compute efficiency of the LFM2 architecture translates directly to inference speed:

Apple M5 Max: 220 tokens/second
AMD Ryzen AI Max+ 395: 113 tokens/second
Smartphone: ~30 tokens/second
NVIDIA H100 (high concurrency): ~15,000 tokens/second (~1.3 billion tokens/day)

The model ships with day-one support for llama.cpp (GGUF format), MLX (Apple Silicon), vLLM, SGLang, and ONNX. A WebGPU browser demo is also available for in-browser inference with no setup.

What This Means for Agent Deployment

The practical implication of zero marginal inference cost is architectural. When every token costs money, developers design agents to be conservative — short context windows, minimal tool calls, batched rather than continuous operation. When inference is free and local, those constraints disappear.

Agents can run continuously in the background on user hardware, process millions of tokens for monitoring or summarization tasks, and be parallelized across many simultaneous workflows without a cloud bill scaling linearly with usage. Privacy is also a genuine benefit: data never leaves the device.

LFM2.5-2.6B is available now on Hugging Face under the LFM Open License v1.0, which permits commercial use for organizations with under $10 million in annual revenue. Both the base model and the post-trained agentic version are available.

The Broader Context

LFM2.5-2.6B is part of a broader trend toward efficient, specialized small models that can handle specific workloads without requiring frontier-scale compute. The LFM2 architecture's hybrid convolution-attention design is a concrete example of moving beyond the assumption that every layer needs full Softmax Attention — a design choice that pays off most clearly at the edge, where memory and compute are genuinely constrained.

For developers building agentic applications where tool use, instruction following, and multi-step planning matter more than raw coding ability, LFM2.5-2.6B offers a compelling option that runs on hardware most users already own.

What OpenAI’s Astra Math Results Teach Us About Verifiable AI Workflows

Prabhakar Chaudhary — Thu, 06 Aug 2026 15:41:37 +0000

What OpenAI’s Astra Math Results Teach Us About Verifiable AI Workflows

When building production software around Large Language Models (LLMs), engineers face a fundamental problem: LLMs generate plausibly structured outputs that frequently contain subtle hallucinations or invalid logic. In low-stakes applications, human spot-checking is enough. In high-stakes settings—like financial engine calculations, infrastructure deployment scripts, or scientific proofs—stochastic outputs must be paired with deterministic verification.

On August 1, 2026, OpenAI published "Ten advances in mathematics and theoretical computer science", describing ten long-standing open problems solved using an internal reasoning system codenamed "Astra." Beyond the specific claims, the release provides a clear blueprint for verifiable software architectures: using stochastic models to explore complex search spaces while delegating truth-checking to a deterministic verifier.

The Astra Case Study: Machine-Checked Mathematics

OpenAI reported that an internal multi-agent version of its Astra model resolved ten mathematical and theoretical computer science problems that had remained open for a decade or more. Rather than asking the public or peer reviewers to trust the raw output of an LLM, OpenAI released a 249-page technical manuscript, reasoning walkthroughs, and a public GitHub repository named openai/ten-proofs containing machine-checkable formalizations written in the Lean 4 interactive theorem prover.

The ten results span several distinct fields:

High-Dimensional Geometry: Establishing new asymptotic upper bounds on sphere-packing density that reach the Cohn–Elkies threshold, representing the first major progress on this exponent since 1978.
Group Theory & Operator Algebras: Constructing the first explicit non-sofic group (resolving an existence question open since 1999) and disproving Connes's rigidity conjecture regarding property-(T) groups and von Neumann algebras.
Theoretical Computer Science: Proving new lower bounds for computing the permanent in arithmetic circuit complexity—specifically an $\Omega(n^4 / \log n)$ formula bound—and proving an exponential parallel repetition theorem for finite two-player entangled quantum games.
Combinatorics & Lattice Cryptography: Resolving Erdős problems 146, 180, and 183 (establishing a superexponential lower bound for multicolor triangle Ramsey numbers), and proving polynomial-factor hardness for the Euclidean Closest Vector Problem (CVP).

This release reflects a structural change from previous AI claims. In October 2025, OpenAI faced criticism when a mathematical claim generated by GPT-5 was shown by mathematician Thomas Bloom to have retrieved existing literature rather than producing an original proof. For the Astra announcement, Bloom publicly noted that the inclusion of verifiable Lean certificates made the release a qualitatively different milestone.

OpenAI estimated the inference cost at approximately $2,000 based on Sol API rates, though researcher Noam Brown clarified this covers only successful search paths, excluding compute spent on failed attempts.

The Trust Boundary: Model Exploration vs. Deterministic Verification

The technical workflow used to produce these results illustrates a three-stage human-AI pipeline:

Generation (Model): The Astra multi-agent system runs autonomous reasoning loops over long time horizons (hours or days) to generate candidate proofs, constructions, and logical steps.
Exposition (Human + Model): Human researchers collaborate with the model to structure the raw arguments into a readable technical paper.
Formalization (Model + Verifier): The model translates the informal mathematical statements and proof steps into formal Lean 4 code. The Lean compiler checks every deduction against Lean's axiomatic foundation.

In this pipeline, the trust boundary is carefully defined. The LLM is never trusted as the final authority on correctness. Instead, its job is restricted to candidate generation and translation. Correctness is established strictly by the Lean 4 kernel, which executes deterministically.

OpenAI explicitly acknowledged that Astra generated the core mathematical arguments, citing the June 2026 Leiden Declaration on AI and Mathematics to note that claiming human authorship for the proofs would be misleading.

Generator vs. Verifier: Why Formal Verification Matters

Understanding the difference between a proof generator and a proof verifier is essential for software engineers building LLM applications.

Dimension	Proof Generator (LLM / Astra)	Proof Verifier (Lean 4 Kernel)
Mechanism	Probabilistic token prediction & search	Deterministic type checking & logic reduction
Output Type	Candidate arguments, code, or natural language	Binary pass/fail (valid type vs. compilation error)
Failure Mode	Hallucination, logical gaps, subtle misstatements	Rejection of unproven steps or invalid axioms
Primary Strength	Creative synthesis across wide search spaces	absolute logical rigor within a formalized spec

An LLM is a stochastic generator. It excels at jumping across conceptual domains—such as applying algebraic number theory to geometric packing problems—and proposing solution structures. However, it cannot guarantee that its output is free of logical flaws.

Lean 4, by contrast, is a deterministic interactive theorem prover. Its trusted kernel checks whether a given formal proof correctly connects a conclusion to its premise using fundamental rules of logic. The Lean compiler does not care how a proof was generated—whether by a human, a heuristic script, or an LLM. It simply verifies whether the proof string validly type-checks without using unproven assumptions (sorry placeholders).

When an LLM outputs code that compiles cleanly in Lean 4 without admitted gaps, the logical correctness of the formal statement is guaranteed by the verifier, effectively eliminating the risk of LLM hallucination in that specific proof artifact.

What Developers Can Borrow

You do not need to be solving open mathematical conjectures to apply this architectural pattern. Any high-stakes software system—such as automated code refactoring, infrastructure-as-code deployment, or financial calculation engines—can use a generator-verifier design.

Decouple generation from validation: Never rely on an LLM to evaluate its own output. Pair LLM generators with external, non-LLM verifiers such as compilers, static analyzers, linters, unit test suites, or schema validators.
Target machine-checkable intermediate formats: Require the model to produce output in formats that can be parsed and executed by strict tools (e.g., Lean code, TypeScript definitions, SQL queries, or OpenAPI specifications) rather than natural language explanations.
Use compiler feedback in retry loops: Implement multi-agent feedback pipelines where error logs from the verifier (e.g., Lean build errors or compiler type failures) are fed back into the model context to allow automated self-correction.
Maintain explicit boundaries on formal specifications: Recognize that automated verifiers only check whether an artifact satisfies a given formal specification. Human domain experts must still review whether the formal specification matches the underlying business or technical requirements.

Limitations and System Caveats

While the Astra results demonstrate the power of machine-checked workflows, several important limitations remain:

The Formalization/Alignment Gap: A Lean certificate confirms logical validity from specified axioms, but does not guarantee that the Lean code correctly captures the original informal problem. Human review is still needed to confirm alignment.
Lack of Novelty Evaluation: A formal verifier cannot determine whether a result is original or merely a derivation of known lemmas.
Opaque Compute Metrics: The $2,000 cost figure covers only successful search paths, excluding failed trajectories and preliminary experimentation.
Unreleased Systems: Astra remains an unreleased, internal research system. Because the underlying model is private, developers cannot independently benchmark its raw, unguided generation performance.

By separating stochastic exploration from deterministic checking, OpenAI’s Astra results offer a practical model for AI engineering: use language models to explore possibilities, but let strict, deterministic systems have the final word.

From Prompting to Reasoning: How ToolArtist Unifies Multi-Step Logic and Image Generation

Prabhakar Chaudhary — Thu, 06 Aug 2026 15:41:32 +0000

From Prompting to Reasoning: How ToolArtist Unifies Multi-Step Logic and Image Generation

Text-to-image (T2I) systems have reached a level of visual fidelity that was difficult to imagine only a few years ago. However, even the most advanced diffusion models and autoregressive generators suffer from a distinct lack of semantic depth. When a user provides a prompt that requires external knowledge, multi-step logical deduction, or complex semantic compositions, traditional models often fail. They are "passive" generators that attempt to map a fixed text embedding to a pixel distribution without any internal mechanism for verification or research.

A new research paper titled "ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation" introduces a significant architectural shift to address these limitations. Instead of treating image generation as a single-step inference task, the researchers at various institutions, including the Beijing Institute of Technology and 01.AI, have treated it as an agentic process. By post-training a Unified Multimodal Model (UMM) to orchestrate reasoning, tool use, and generation within a single policy, ToolArtist demonstrates a way to bridge the gap between "dumb" generation and "intelligent" creation.

The Problem of Passive Generation

Most existing image generation workflows rely on a fixed pipeline. If a prompt requires specific knowledge—for instance, "an image of an extinct bird from the Mauritius island that lived in the 17th century"—a standard model might guess based on its training data. If that data is sparse or conflicting, the result is inaccurate. More advanced systems might use a Large Language Model (LLM) as a front-end to expand the prompt, but this is a disconnected process. The LLM does not know what the image generator is capable of, and the generator has no way to ask the LLM for clarification or to search the web for visual references.

This disconnection leads to what researchers call a coordination failure. Reasoning, tool invocation (like searching the web for a reference image), and the final acts of drawing are not handled by the same decision-making entity. ToolArtist changes this by making the generator itself the agent.

The Architecture: Unified Multimodal Models (UMMs)

ToolArtist is built upon a Unified Multimodal Model (UMM). Unlike dual-tower architectures that use separate encoders for text and images (like CLIP-based systems), a UMM treats both modalities as tokens in a shared sequence. This allows the model to "speak" in text and "draw" in visual tokens using the same underlying weights.

The core contribution of ToolArtist is the way it is post-trained to become "fully agentic." The model doesn't just output an image; it generates a trajectory of thought. If a prompt is complex, the model first generates reasoning tokens (Chain of Thought), then decides whether to call an external tool (such as a search engine), processes the results, and finally generates the visual tokens to render the image.

RAD-GRPO: Reinforcement Learning for Drawing Agents

One of the most technical aspects of the ToolArtist paper is the introduction of a new reinforcement learning algorithm: Reason-Act-Draw Group Relative Policy Optimization (RAD-GRPO).

RAD-GRPO is an extension of the Group Relative Policy Optimization (GRPO) method, which gained prominence for its efficiency in training reasoning models like DeepSeek-R1. GRPO removes the need for a separate critic model by computing the baseline reward from a group of outputs for the same prompt. This significantly reduces the compute and memory overhead during the RL phase.

In ToolArtist, RAD-GRPO is adapted to handle the unique requirements of image generation. The researchers introduced two complementary reward systems:

Intent Rewards: This measures how well the agent follows the logical steps required by the prompt. Did it recognize it needed to search? Did it use the search results correctly?
Quality Rewards: This measures the visual fidelity and semantic alignment of the final generated image.

By jointly optimizing for both reasoning and drawing, the model avoids the pitfall of "hallucinating" logical steps that don't actually inform the final image. The policy learns that better reasoning leads to higher drawing rewards.

The Training Pipeline

The post-training of ToolArtist follows a two-stage process.

First, the researchers used Supervised Fine-Tuning (SFT). They created a "teacher agent" that had access to high-quality search tools and image-generation tools. They collected thousands of successful trajectories where the teacher solved complex visual tasks. Importantly, they converted these trajectories into a format compatible with UMMs, where the technical tool calls are hidden, but the "thoughts" and resulting images are preserved.

Second, the Reinforcement Learning (RL) stage applies the aforementioned RAD-GRPO. During this phase, the model is encouraged to explore different reasoning paths. It is penalized if it skips necessary research steps or if the final image doesn't match the researched facts. This creates a model that is "stubborn" about accuracy—it won't start drawing until it is sure it has the right references.

Why Agentic Generation Matters

The implications of this research are practical for developers and researchers. Most open-world tasks are not simple descriptors. They are goals. If a user asks for "a poster for a tech conference in Tokyo that looks like it was designed in the 1960s," a standard model might get the "Tokyo" and "tech" parts right, but fail on the specific graphic design aesthetics of 1960s Japan unless it was heavily represented in the training set.

ToolArtist can search for 1960s Japanese poster design, analyze the common color palettes (like ochre, teal, and bold reds), identify the typical typography styles, and then apply those learned constraints to its internal drawing engine. This moves us away from the "lottery" of prompting and toward a reliable, iterative engineering process for visual media.

Conclusion

ToolArtist represents a shift toward more capable, autonomous multimodal systems. By integrating reasoning and act-ready capabilities directly into the generation policy, it overcomes the limitations of passive, single-step models. The use of RAD-GRPO demonstrates that reinforcement learning is not just for chat and coding—it is a powerful tool for visual creativity as well. As UMMs continue to scale, the distinction between "thinking" and "creating" will continue to blur, leading to tools that understand the world as well as they can depict it.

Sources

Primary Research Paper: ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
Context on GRPO: DeepSeek-V3 Technical Report
Supporting Research on UMMs: Show-o: One Single Transformer to Unify Diffusion and Autoregression

ABSeeker: Solving the Credit Assignment Problem in Long-Horizon Search Agents

Prabhakar Chaudhary — Thu, 06 Aug 2026 15:37:59 +0000

ABSeeker: Solving the Credit Assignment Problem in Long-Horizon Search Agents

Most current Large Language Model (LLM) agents operate through a sequence of discrete actions—searching for information, navigating web pages, and synthesizing evidence. While these "agentic" workflows are powerful, they suffer from a fundamental training inefficiency known as the sparse reward problem. When an agent fails at the end of a long-horizon task, standard reinforcement learning (RL) or supervised fine-tuning (SFT) often penalizes the entire trajectory. Conversely, if an agent succeeds, it rewards every step taken, even those that were redundant or accidental.

In a recent paper titled "ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment" (arXiv:2608.05102), researchers introduce a framework designed to bridge this gap. By converting trajectory-level outcomes into dense, step-level supervision, the authors demonstrate that even a relatively small 4-billion-parameter model can match the performance of models nearly ten times its size.

The Challenge of Multi-Step Reasoning

Training agents to perform long-horizon tasks—those requiring dozens of sequential actions—is difficult. In a typical scenario, an agent might be asked to find the intersection of two complex datasets or verify a specific fact that requires multiple hops across different web domains.

Standard Supervised Fine-Tuning (SFT) usually relies on "gold" trajectories. If the training data contains a trajectory where the agent eventually reached the correct answer, the model is taught to replicate every single token in that sequence. Similarly, Reinforcement Learning (RL) typically applies a reward based on the final binary outcome: success or failure.

The mechanical failure here is one of "credit assignment." A successful trajectory might contain several hallucinations or wasted search queries that eventually led to a lucky break. Rewarding these steps encourages the model to be inefficient. On the flip side, a failed trajectory might contain valuable intermediate reasoning steps that were ultimately let down by a single typo or a dead link in the final stage. Penalizing these useful intermediate actions slows down the learning process significantly.

Answer-Backtracked Credit Assignment (ABC)

The core contribution of the ABSeeker research is the Answer-Backtracked Credit Assignment (ABC) framework. This method reverses the reasoning process to create a dense supervision signal. The framework consists of two primary stages: Clue Recovery and Step Scoring.

1. Answer-Backtracked Clue Recovery

Instead of looking at the agent's actions chronologically, the ABC framework starts with the ground-truth answer. It uses a "teacher" model to identify the specific "clues" or information kernels that were necessary to reach that answer. For a query like "Who designed the building where the 1992 treaty was signed?", the clues recovered might include:

The treaty name.
The building location.
The architect's identity.

By identifying these clues independently of the agent's actual path, the system creates a benchmark of "required information."

2. Clue-Anchored Step Scoring

Once the clues are identified, the framework evaluates every search step in an agent's trajectory. A step is rewarded if it successfully uncovers one or more of these required clues. This allows the system to assign a high "credit score" to a search query that found the correct building, even if the agent later failed to identify the architect.

This dense scoring mechanism transforms a single binary success/failure bit into a detailed map of utility. It prevents the model from being punished for success (when it makes mistakes during a successful run) or rewarded for failure (when it gets lucky).

Training ABSeeker: ABC-SFT and ABC-GRPO

The researchers applied this dense scoring signal to two distinct training phases.

First, they implemented ABC-SFT. Unlike standard SFT, which gives equal weight to all tokens, ABC-SFT reweights the loss of each turn based on its utility score. Steps that successfully retrieved crucial clues are given higher weights, forcing the model to pay more attention to the patterns that lead to quality information retrieval.

Second, they integrated the scores into ABC-GRPO (Group Relative Policy Optimization). GRPO is a form of reinforcement learning that compares a group of trajectories produced by the same model. By using the step-level ABC scores as the reward function, the model learns to prioritize actions that maximize information density per step. This approach is effective because it doesn't just reward the final answer; it rewards the efficiency of the search process itself.

Quantifiable Gains in Efficiency

The results of the ABSeeker experiment are notable. The researchers used a Qwen3.5-4B base model and trained it on a dataset of 8,500 examples. Despite the small model size and modest training data, ABSeeker achieved a score of 37.3% on the BrowseComp benchmark.

When a context management system was added—allowing the model to better organize the clues it had already found—the performance reached 55.3%. This result puts the 4B-parameter ABSeeker on par with models in the 30B-parameter range that were trained without dense credit assignment. This suggests that the capability of an agent is not solely a function of its parameter count, but rather how effectively it can evaluate its own progress.

Why Dense Rewards Matter

The success of ABSeeker highlights a shift in scaling AI. While adding more parameters is a reliable way to increase performance, optimizing the training loop to handle the specific requirements of agentic reasoning can produce similar results at lower cost. By rethinking credit assignment, the researchers have provided a blueprint for creating more efficient and more reliable long-horizon agents.

The core benefit of ABC is the reduction of gradient noise. When a model is trained on long sequences with sparse rewards, the signals telling the model how to change its weights become unstable. By providing a reward at every step, the ABC framework ensures that the model receives immediate feedback on every action. It address the on-policy vs. off-policy dilemma by providing a stable signal that remains valid even as the agent explores new ways to search.

Sources

Inside GPT-Live: How OpenAI Rebuilt ChatGPT's Voice Stack for Full-Duplex Conversation

Prabhakar Chaudhary — Thu, 06 Aug 2026 07:43:59 +0000

Inside GPT-Live: How OpenAI Rebuilt ChatGPT's Voice Stack for Full-Duplex Conversation

Most voice assistants are built like walkie-talkies. You press to talk, release to listen, and the system tries to guess the exact moment you finished speaking so it can start its reply. Get that guess wrong and you either get cut off mid-sentence or sit through an awkward pause. That turn-detection problem has haunted voice AI since the first interactive voice response systems.

On August 3, 2026, OpenAI published a detailed engineering post on how it spent six months rebuilding ChatGPT's voice infrastructure to get rid of that tradeoff. The result is GPT-Live, a "turnless" speech model that listens and speaks simultaneously, now powering ChatGPT Voice. This post walks through the architecture described in OpenAI's engineering write-up, with detail from RuntimeWire's technical analysis of the announcement.

The Problem: Turn Detection Is a Bottleneck

Earlier ChatGPT voice systems treated a conversation as a sequence of turns. A separate voice-activity detector had to decide when the user had stopped talking before the model could respond. That design creates an unavoidable tradeoff: decide early and you risk cutting the user off; decide late and you add an audible pause to every exchange.

This is a latency problem that no amount of faster inference fully solves, because the decision itself is the bottleneck. Every turn boundary is a judgment call about silence, breathing, filler words, and unfinished sentences. As OpenAI's earlier write-up on low-latency voice at scale noted, conversational turn-taking is one of the hardest problems in real-time voice AI.

The Fix: One Continuous Full-Duplex Model

GPT-Live removes the separate turn detector from the audio path entirely. Instead of two phases (listen, then speak), the model runs full-duplex: it continuously processes incoming speech while generating outgoing audio, and decides several times per second whether to listen, speak, pause, interrupt, or hand off to another system.

The implications go beyond feeling more natural. Because the model never waits for a clean turn boundary, interruptions become first-class interactions. You can talk over the assistant mid-answer and it can adjust on the fly — something that felt broken in every previous generation of voice assistants.

Separating Speech from Everything Else

The most interesting architectural decision is what OpenAI did around the model. Speech travels between the client and GPT-Live over a dedicated fast path — a low-latency lane dedicated to the audio loop. Everything else (search, tool calls, persistence, and deeper reasoning) sits behind an asynchronous boundary.

This means a slow tool call no longer stalls the conversation. When you ask a question that requires a web search, GPT-Live keeps talking while the search runs in the background, then weaves the result into the flow when it's ready. The system is effectively two models presented as one assistant: GPT-Live handles timing and speech, while a frontier model — GPT-5.5 Instant for the lighter settings, GPT-5.5 Thinking for Medium and High — handles the work that needs more computation. OpenAI can also swap out that delegated reasoning model as newer systems arrive, without rebuilding the voice model around each release.

From Python to Go: Engineering for Consistent Latency

OpenAI rewrote the media frontend and inference logic in Go, replacing an earlier Python asyncio implementation. The stated goal was consistency rather than peak speed: the new system's 95th-percentile frame-delivery performance matched the previous system's median. In real-time audio, worst-case latency is what users actually feel, so moving the whole distribution left matters more than winning a benchmark.

Long conversations posed another problem. Context grows over time, and model instances may need to be replaced as capacity changes. OpenAI built a handoff process that warms a replacement instance, loads it with the current context, and runs both instances in parallel before switching traffic. The same mechanism lets it compact an oversized context without pausing the conversation.

WARP: Cutting Session Startup from Six Round Trips to One

Below the model layer, standard WebRTC setup requires several protocol handshakes before media can flow. OpenAI developed the WebRTC Abridged Roundtrip Protocol (WARP), which reduced media and data startup from six network round trips to one. A related system called Instant Connect negotiates session parameters ahead of time, allowing the client to begin a session with a single UDP packet when those parameters are still valid.

This matters at scale. ChatGPT Voice and Dictation serve more than 150 million people each week, and every session startup that gets cut from six round trips to one is a measurable reduction in perceived lag. OpenAI is advancing WARP through an Internet Engineering Task Force working group, and says support has already landed in libwebrtc and Pion — so the work generalizes beyond OpenAI's own stack.

What Shadow Deployment Revealed About Capacity

Before the July 8 launch, OpenAI routed a gradually increasing share of production voice sessions through the new system while Advanced Voice Mode kept serving everyone else. This shadow deployment exposed the replacement stack to real networks, session lengths, and geographic traffic.

The testing surfaced a counterintuitive finding: GPU throughput alone is a poor measure of voice capacity. Voice calls keep connections open and send frames continuously, placing sustained pressure on CPU stream handlers, queues, and network services. OpenAI reports that one supporting component saturated earlier than load tests predicted, causing inference requests and latency to accumulate. At hundreds of millions of weekly users, natural conversation depends as much on regional routing, connection setup, and recovery behavior as it does on model inference.

What It Means for Developers

GPT-Live is currently powering ChatGPT Voice (GPT-Live-1 for Go, Plus, and Pro users; GPT-Live-1 mini for Free users), and OpenAI says an upcoming GPT-Live API will expose the same architecture to developers. That API is the part worth watching: a voice system built around continuous interaction and background delegation raises the bar for customer support, tutoring, and desktop agents.

For anyone building voice products, the takeaways are practical. Separate the audio loop from application logic so slow services can't stall speech. Optimize for consistent latency, not best-case latency. And treat connection setup as a first-class latency problem — six round trips of handshaking is six round trips too many when users can hear the difference.

SparseSpec-L: How a Sparse KV Cache Makes Long-Context LLM Inference 2.79 Faster — Without Any Training

Prabhakar Chaudhary — Wed, 05 Aug 2026 16:11:16 +0000

SparseSpec-L: How a Sparse KV Cache Makes Long-Context LLM Inference 2.79× Faster — Without Any Training

Speculative decoding has become one of the more practical tools for cutting LLM inference latency. The idea is simple: use a fast draft mechanism to propose several tokens at once, then let the full model verify them in a single parallel pass. When the draft is good, you get multiple tokens for roughly the cost of one.

The catch is that most setups require a separate draft model — a smaller sibling trained to mimic the target model's distribution. That works in controlled settings, but it adds memory overhead, requires careful alignment, and tends to degrade at long context lengths where the draft model's compressed representation diverges from the target's.

A new paper from Liu et al. — SparseSpec-L: A Sparse Glimpse of the Whole — takes a different approach. It uses the target model itself as the drafter, but with a dynamically sparsified KV cache during the draft phase. The result is a training-free, single-model framework that achieves up to 2.79× speedup over standard autoregressive decoding, with particular gains at long context lengths where other methods struggle.

Why Long-Context Speculative Decoding Is Hard

Standard speculative decoding has a well-known problem: the acceptance rate — the fraction of draft tokens the target model agrees with — drops as context grows. A draft model trained on shorter sequences tends to lose coherence at 32K or 64K tokens, producing drafts the target model rejects more often. When rejection rates climb, the overhead of running the draft mechanism can make inference slower than standard autoregressive decoding.

There's also a subtler issue called efficiency inversion. Even with a good draft model, there's an optimal speculation length — the number of tokens you propose per step. Propose too few and you leave speedup on the table. Propose too many and the marginal cost of drafting exceeds the marginal gain from parallel verification. Fixed speculation lengths can't adapt to this.

Self-speculative decoding methods (like LayerSkip) address the separate-model problem by using a truncated version of the target model — skipping intermediate layers — to generate drafts. This eliminates the structural mismatch, but layer-skipping still requires careful calibration and can hurt draft quality at long contexts.

The SparseSpec-L Approach

SparseSpec-L's core insight is that you don't need to skip layers to make drafting cheap. Keep the full model architecture but compress the KV cache it attends to during the draft phase. The full KV cache is retained for verification, so the output distribution is mathematically identical to standard autoregressive decoding — the method is lossless.

The framework operates in a sparse-to-full pipeline:

Draft with a sparse KV cache. The model attends to a compressed subset of past tokens rather than the full context. This reduces memory bandwidth and compute, making each draft step faster.
Verify with the full KV cache. The verification pass uses the complete context, ensuring the accepted tokens match what the full model would have produced.
Recycle attention statistics. SparseSpec-L reuses the per-head attention scores from the previous verification step. Tokens that received high attention are flagged as important and retained in the next draft's sparse cache — no extra forward pass needed.

The sparse cache preserves three categories of tokens: sink tokens (the first few tokens, which tend to receive disproportionate attention in transformer models), recent tokens (the last window of tokens), and important historical tokens (the top-K positions by aggregated attention score from the verification pass). This mirrors the structure used in KV cache eviction methods like SnapKV and H2O, but here the eviction is temporary — the full cache is always available for verification.

The Entropy-Based Speculation Controller

The second major contribution is an adaptive controller that dynamically adjusts the speculation length $k$ at each step. This directly addresses the efficiency inversion problem.

The controller tracks the output entropy of drafted tokens. High entropy means the drafter is uncertain — those tokens are more likely to be rejected. Low entropy means the drafter is confident — those tokens are more likely to be accepted. By maintaining running estimates of the mean entropy for accepted and rejected tokens, the controller estimates a soft acceptance probability $\pi_i$ for each position in the draft.

Given these estimates, the controller selects the speculation length $k^*$ that maximizes expected step-wise efficiency:

$$k^* = \arg\max_{k \in K} \left[ \frac{1 + \sum_{i=1}^{k} \pi_i}{k \cdot C_d + C_v} \right]$$

Here $C_d$ is the cost of one draft step and $C_v$ is the cost of one verification step. The numerator is the expected number of tokens generated; the denominator is the total compute cost. The controller picks the $k$ that maximizes this ratio — no training required, just online statistics.

In practice, the controller extends speculation aggressively when the drafter is confident and contracts it when uncertainty rises. This avoids the performance craters that fixed-length speculation methods hit when context becomes difficult to predict.

Benchmark Results

The paper evaluates SparseSpec-L on Llama-3-8B-Instruct and Mistral-7B-v0.3, with context lengths from 10K to 60K tokens. Key results:

Up to 2.79× speedup over standard autoregressive decoding at 60K context length
Consistent gains across summarization, question answering, and code completion tasks
Outperforms LayerSkip and auxiliary-model speculative decoding at context lengths above 20K
No degradation in output quality — the method is mathematically lossless

The speedup grows with context length — the opposite of most speculative decoding methods. The sparse KV cache becomes relatively cheaper as context grows (the full cache gets larger, but the sparse cache stays bounded), while the verification pass benefits from parallel token processing regardless of context length.

What This Means for Practitioners

SparseSpec-L is notable for what it doesn't require: no draft model to train or maintain, no architectural changes, no fine-tuning. Load your existing model, configure the sparse cache budget and the entropy controller's candidate set, and run. The authors provide code on GitHub alongside the paper.

The practical implications are clearest for applications that already use long contexts — retrieval-augmented generation over large document sets, multi-turn agents with long conversation histories, code assistants working on large repositories. These are exactly the settings where existing speculative decoding methods tend to underperform.

There are real limitations. The speedup depends on the acceptance rate, which varies by task and model. The entropy controller's estimates are noisy early in a generation before it has accumulated enough statistics. The sparse cache budget is a hyperparameter that needs tuning — too aggressive and draft quality drops; too conservative and the speedup shrinks.

For teams running inference on long-context workloads and looking for latency improvements without the overhead of a separate draft model, SparseSpec-L is a practical option worth evaluating. The training-free property is particularly valuable in production settings where maintaining a separate draft model aligned to the target model's distribution is an ongoing engineering burden.

The broader trend is worth watching: as context windows grow toward millions of tokens, the memory bandwidth bottleneck in autoregressive decoding becomes increasingly severe. Methods that reduce that bottleneck without requiring additional training infrastructure are likely to see wider adoption.

Primary source: SparseSpec-L: A Sparse Glimpse of the Whole (arXiv:2607.27735)

WaiT for the Signal: Why Image Generators Should Build the Big Picture Before the Texture

Prabhakar Chaudhary — Wed, 05 Aug 2026 06:51:06 +0000

If you have built with diffusion or flow-matching image models, you have seen the central tension: a model has to decide both the composition of an image and the tiny details that make it look convincing. At high resolution, those are not equally difficult problems—and treating them as though they are can be expensive.

A new Meta-led paper, WaiT for the Signal: Simple Frequency-Aware Flow-Matching, posted on August 4, proposes a surprisingly direct fix. It uses a wavelet transform to split an image into coarse and fine frequency bands, then lets the fine bands literally wait until the coarse image has begun to take shape.

The authors report up to 50% lower sampling compute in their setup, alongside strong high-resolution image and video results. Those are research results rather than a production guarantee, but the design is worth understanding.

The mismatch in ordinary generative schedules

Diffusion models and flow-matching models usually begin with noise and iteratively move toward an image. Flow matching learns a vector field that describes that trip from noise to data; see the original Flow Matching paper for the general formulation.

In a conventional pixel-space setup, every spatial frequency participates throughout the same time interval. A low-frequency component—large shapes, lighting, pose, horizon line—gets no special treatment relative to a high-frequency component such as hair strands, brick texture, or sensor-like grain.

That is convenient, but it is not how the signal behaves. The paper’s observation is that fine-frequency components become indistinguishable from noise earlier in the trajectory than coarse components. Asking a network to model those fine components too soon is partly asking it to reason about noise that has not acquired useful structure yet.

A practical analogy is progressive image loading. There is little value in downloading a crisp weave pattern before you know whether the image contains a jacket, a tree, or a building. WaiT makes a similar prioritization part of generation itself.

What WaiT changes

WaiT stands for Wavelet-aware image Transformer. Its key building block is a lossless discrete wavelet transform (DWT). The DWT decomposes an image into one low-frequency band and several high-frequency bands. The low-frequency band carries the broad structure; the high-frequency bands encode detail in different directions.

The method then assigns different noise schedules to those bands:

Coarse band: runs across the full generation timeline.
Fine bands: are kept as noise over an initial portion of that timeline.
Crossover: at a chosen time, the fine bands are introduced, the bands are recombined with an inverse DWT, and the model refines them jointly.

The paper’s title captures the intuition: high-frequency information should “wait for the signal.” Once there is a believable global scene, details have context. Texture can align with object boundaries, lighting, and motion instead of competing with an unresolved composition.

Crucially, this is presented as a scheduling change rather than a sprawling multi-stage generator. The authors apply it to a pixel-space Just image Transformer (JiT) with band-specific scheduling and a resolution embedding, rather than requiring a separate cascade for every resolution. That relative simplicity is important for practitioners: the idea may be easier to test in an existing transformer-based flow-matching pipeline than a wholesale architecture replacement.

A simplified mental model

# Pseudocode: the transform is lossless; scheduling is the new idea.
low, high = dwt(image)

# During training, sample time t.
low_t = add_noise(low, t)

if t < crossover:
    high_t = pure_noise_like(high)       # detail has not joined yet
else:
    high_t = add_noise(high, remap(t))   # detail is now modeled

prediction = model(low_t, high_t, t)
loss = flow_matching_loss(prediction, target_velocity)

The exact parameterization, time mapping, loss, and network inputs matter; this is not a reproduction recipe. But it shows the useful distinction: frequency decomposition is standard signal processing, while the contribution is coordinating when each part of that signal becomes a learning target.

Why this could matter at 512px and above

Higher resolution multiplies the number of pixels and increases the volume of high-frequency content. It also makes failures more visible. A generated face can have the right overall geometry but break down in eyelashes, skin texture, or repeated patterns; a video can preserve a subject while introducing flicker in fine motion.

The authors report a pixel-space FID of 1.43 on ImageNet at 512×512 and a 2B-parameter result of 1.3. They also report scaling the approach to 1024×1024 text-to-image generation and an FVD of 0.84 on Kinetics-600 for video. More interesting than any single leaderboard value is the reported compute-quality trade-off: delaying detail can reduce sampling compute by up to 50% in their experiments.

That claim needs the usual engineering caution. Sampling cost depends on resolution, batch size, hardware, scheduler, model size, quality target, and the baseline being compared. A production team should measure end-to-end latency and throughput on its own prompts and safety stack—not infer a universal 2× speedup from a paper.

Still, the direction is attractive for teams serving generated media. If early steps are devoted to global structure, a system may be able to choose a cheaper preview path, reserve detail refinement for accepted candidates, or vary the crossover based on requested output size.

A better way to inspect generated images

WaiT also questions how image generators are evaluated. Fréchet Inception Distance (FID) is widely used, but its Inception features rely on an input resized to 299×299. For 512px or 1024px output, that can weaken its sensitivity to local textures—the very behavior the method targets.

The paper therefore uses three measurements:

FID for overall distribution and global coherence.
5-crop FID (5cFID) to look at local crops at native resolution.
High-frequency Fréchet Wavelet Distance (hFWD) to isolate texture fidelity.

The broader lesson is useful even if you never implement hFWD. Do not evaluate a high-resolution generator with one aggregate score alone. Pair global metrics with targeted checks for the defects users notice: text rendering, faces, repeated patterns, small objects, temporal flicker, and prompt-specific failure cases.

Where developers can experiment

WaiT is most relevant to researchers and platform teams training pixel-space image or video generators. It is less immediately actionable for an application that only calls a hosted image API. But its decomposition suggests several experiments:

1. Profile work by frequency, not only by step

When profiling a denoiser, ask whether late-stage steps add useful detail or merely consume budget. Saving intermediate outputs and inspecting wavelet-band energy can reveal whether the model is resolving structure before texture.

2. Treat previews and final renders differently

For a creative tool, a coarse-only or coarse-first preview could feel responsive, with fine-band refinement triggered when the user selects an image. That requires product work and perceptual testing, but it matches the method’s core hierarchy.

3. Add local-quality regression tests

Keep your global benchmark, then add cropped-detail and frequency-aware checks. A model that improves a global metric while smearing material texture is not necessarily an improvement for a design, media, or simulation workflow.

The important constraint: detail cannot be an afterthought

WaiT is compelling precisely because it does not discard high-frequency information. It postpones it, then refines it jointly with the coarse image. That distinction matters. Many images fail when details are generated independently of the scene; a late refinement stage still needs access to the established global signal.

The paper is also a reminder that model progress does not always mean adding parameters. Sometimes the useful question is whether the training trajectory respects the structure already present in the data. Images have frequency hierarchies. Giving a generator permission to resolve those hierarchies in order is a clean idea—and one developers building high-resolution generation systems should keep on their radar.

Beyond Size: The Three Pillars of Test-Time Scaling in Large Language Models

Prabhakar Chaudhary — Wed, 05 Aug 2026 06:47:18 +0000

Beyond Size: The Three Pillars of Test-Time Scaling in Large Language Models

The narrative of artificial intelligence for the last decade has been dominated by a single, powerful trend: scaling. From the early days of AlexNet to the massive clusters powering GPT-4, the formula seemed simple—more data and more parameters lead to better performance. This paradigm, famously codified as the "Scaling Laws," suggested that we could predict model improvements simply by looking at the amount of compute poured into the pre-training phase. However, as the industry pushes against the boundaries of available high-quality data and the physical limits of hardware, a new dimension of scaling is emerging. It isn't about how large the model is, but how long it "thinks" before it speaks.

This shift toward "test-time scaling" marks a transition from static intelligence to dynamic reasoning. Instead of relying solely on the patterns learned during training, models are now being equipped with the computational budget to explore, verify, and refine their answers at the point of inference. While the concept was popularized by the release of models like OpenAI’s o1 series, the underlying mechanics remained somewhat opaque. A recent comprehensive study by Hariri et al. (2026), titled "Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility", provides a much-needed formal framework for understanding this new frontier.

The Three Regimes of Inference Compute

The core contribution of the Hariri et al. paper is the formalization of test-time scaling into three distinct structural regimes. Rather than treating all "extra compute" as a single scalar budget, the authors map how compute is allocated across the implicit prefix tree of an autoregressive model.

1. Single-Trajectory Sequential Scaling

This is the most familiar regime, often associated with Chain-of-Thought (CoT) prompting. In this mode, the model generates a single sequence of tokens. Compute is scaled by allowing the model to generate more tokens—essentially "talking to itself" to work through a problem. The researchers describe this as extending deliberation along a single path. While simple, this approach is limited by the model's ability to maintain coherence over long sequences and the risk of "hallucinating" a wrong turn that derails the entire reasoning process.

2. Leaf-Level Scaling with Terminal Reduction

When a model encounters a difficult problem, curiosity often beats persistence. Leaf-level scaling involves generating multiple independent completions (the "leaves" of the tree) and then using a reduction mechanism to pick the winner. Common techniques include "Best-of-N" sampling (using a verifier to pick the best response) or "Majority Voting" (self-consistency). As noted in earlier work like Snell et al. (2024), scaling compute in this way can often be more effective than simply using a larger model, provided the model has a non-trivial chance of reaching the correct answer in at least some of its attempts. It distributes the risk of failure across many parallel trials.

3. Prefix-Level Scaling (Search)

This is the most complex and potentially powerful regime. Instead of finishing a response before evaluating it, prefix-level scaling involves searching over partial states. This might include beam search, Monte Carlo Tree Search (MCTS), or other lookahead algorithms. The model looks at several possible next steps, evaluates their potential, and decides which path to pursue further. This allows for early pruning of "dead ends" and a more efficient allocation of tokens toward promising directions. It essentially turns the language model into a search agent, a technique that has its roots in classical AI but is now being revitalized with deep learning verifiers at every step.

The Evaluation Crisis

One of the most pressing issues identified by the researchers is that our current evaluation metrics are ill-equipped for this new era. In the pre-training era, we measured "accuracy" on a test set. In the test-time scaling era, "accuracy" is no longer a fixed property of the model; it is a function of the inference protocol.

If Study A reports 80% accuracy using 100 samples and a majority vote, and Study B reports 75% accuracy using a single MCTS search with the same total token budget, how do we compare them? Hariri et al. argue that reporting accuracy without the specific inference protocol and its associated compute cost (in FLOPs or tokens) makes results meaningless. They propose a new "evaluation profile" that maps performance against compute budget across different regimes. This allows researchers to see not just which model is better, but which system (model + inference algorithm) is more efficient at different resource levels.

Moving Toward Compute-Optimal Reasoning

The implications of this research go beyond just benchmarks. It suggests a future where the boundary between "training" and "inference" begins to blur. If we can achieve GPT-5 level performance on a specific task by running a smaller, GPT-4 class model for a few minutes of "thinking," should we still invest hundreds of millions in the next training run?

The answer likely lies in the "compute-optimal" strategies discussed by Snell et al. (2024). Their research showed that the effectiveness of different test-time strategies depends heavily on the difficulty of the prompt. For easy questions, extra compute is a waste. For "impossible" questions (where the model’s base probability of success is near zero), no amount of sampling will help. The "sweet spot" is the intermediate difficulty, where the model can be guided toward success through search and verification.

Conclusion

Test-time scaling represents the next major evolution in LLM capability. By treating inference as a search and reasoning process rather than a simple sequence generation task, we are unlocking higher-order intelligence from our existing models. However, as Hariri et al. (2026) make clear, this requires a rigorous shift in how we build, evaluate, and describe these systems. We need to move away from reporting static accuracy scores and start describing "reasoning systems" that optimally balance the speed of intuition with the depth of deliberation.

The era of merely "bigger" models is giving way to an era of "smarter" use of compute. Whether through extended chains of thought, massive parallel sampling, or complex tree searches, the goal remains the same: ensuring that when a model answers, it has done the hard work of thinking first.

Primary Source:

Hariri, M., et al. (2026). Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility. arXiv:2608.04001

Supporting Sources:

Snell, C., et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. arXiv:2408.03314
OpenAI (2024). Learning to Reason with LLMs. openai.com

Explorative Modeling: Why the Training Loop May Matter More Than the Generator

Prabhakar Chaudhary — Tue, 04 Aug 2026 18:45:18 +0000

In deep learning, scaling up parameter counts and training data has long served as the primary strategy for boosting model capability. However, generative modeling still faces a fundamental hurdle when capturing complex, multimodal data distributions: how to generate clean, un-blurred outputs without relying on hundreds of slow, sequential sampling steps during inference.

A July 2026 paper titled Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation by Alexi Gladstone, Heng Ji, and Yilun Du presents a new approach to this problem. Rather than breaking down the generation process into multi-step chains during inference, Explorative Modeling (XM) shifts the decomposition into the training loop itself. By exploring $K$ candidate outputs during each training step and optimizing solely for the best match against target data, XM allows models to commit to single, coherent modes without blurring. This technique introduces "generative expressivity"—the capacity to capture distinct modes—as a third pretraining axis alongside parameter size and dataset volume.

The Core Mismatch: Factorizing Generation vs. Factorizing Training

The Mode Blurring Problem

To understand why Explorative Modeling is necessary, consider standard reconstructive training. When a single input or prompt corresponds to multiple valid targets—such as predicting the next visual frame of an unpredictable trajectory or generating an image from a broad text description—a single direct regression model trained with conventional loss functions tends to output the mathematical average of all plausible targets. In image generation or continuous control, this averaging manifests as "mode blurring," producing unrealistic, fuzzy predictions.

Exposure Bias and Multi-step Generation

To avoid mode blurring, conventional generative frameworks like diffusion models and visual autoregressive transformers factorize the generation procedure. They decompose a complex generation task into dozens or hundreds of small, incremental, nearly unimodal steps.

While this factorization prevents mode averaging, it introduces a substantial train-inference mismatch known as exposure bias. During training under teacher forcing, models observe true, ground-truth context. During inference, however, models must rely on their own past predictions. Small errors early in the generation horizon compound over time, leading to distributional drift and epistemic underidentification, where the model encounters self-generated states that lack clear historical context. Furthermore, running multi-step inference chains creates significant computational overhead.

How Explorative Modeling Works

The Best-of-K Training Mechanism

Explorative Modeling resolves this bottleneck by keeping the generation process intact—enabling end-to-end generation—and instead factorizing the training loop.

Instead of generating a single output per input and computing a loss against the ground truth, the training procedure works through four key steps:

Candidate Generation: For a given training input, the model generates $K$ candidate outputs.
Matching Evaluation: Each candidate is evaluated against the ground-truth target data using a loss metric.
Selective Backpropagation: The system identifies the single candidate output that best matches the ground-truth target.
Gradient Update: Gradients are backpropagated exclusively through that best-matching candidate, updating the network weights only on the winning branch.

By optimizing only the closest candidate match, the model is never forced to fit the average of inconsistent targets. Instead, predictions commit to specific, distinct data modes.

Conceptual Workflow

The core mechanism can be conceptualized in Python-like pseudo-code:

def explorative_training_step(model, x_input, y_target, K):
    # 1. Generate K candidate outputs from the model
    candidates = [model(x_input) for _ in range(K)]

    # 2. Compute loss for each candidate against the ground truth
    losses = [compute_loss(cand, y_target) for cand in candidates]

    # 3. Select the index of the best candidate
    best_idx = torch.argmin(torch.tensor(losses))

    # 4. Backpropagate loss ONLY through the best-matching candidate
    best_loss = losses[best_idx]
    best_loss.backward()
    optimizer.step()

During inference, because the model learned to map inputs directly to coherent modes during training, generation can occur in a single step or over significantly fewer steps, eliminating multi-step sampling chains.

Generative Expressivity as a Third Pretraining Axis

Synergistic Scaling Across Modalities

The research presented by Gladstone et al. demonstrates that exploration ($K$) acts as a fundamental third pretraining axis alongside model parameters and data volume. The authors report that scaling the amount of exploration monotonically improves model performance across discrete and continuous domains, including language, video, and image generation.

Importantly, the benefits of exploration scale synergistically with standard pretraining axes:

Data Scale: As dataset sizes increase, the performance gains from exploration grow from 7% to 36%.
Model Scale: As parameter counts scale up, the performance gains from exploration increase from 13% to 23%.

Empirical Benchmarks and Efficiency

The research team reported substantial efficiency improvements when comparing Explorative Models to conventional generative baselines:

Sample Efficiency: 6.2x improvement over traditional training.
FLOP Efficiency: 4.1x improvement in total floating-point operations.
Parameter Efficiency: 47% improvement.

In visual generation tasks documented in the arXiv paper repository, integrating exploration into near-state-of-the-art recipes like Representation Autoencoders (RAE) achieved a 1.43 Fréchet Inception Distance (FID) on 256x256 ImageNet without classifier-free guidance, while converging approximately 300x faster than standard Scalable Interpolant Transformers (SiT) baselines.

In robotics and control tasks (including behavior cloning and goal-conditioned world modeling), end-to-end Explorative Models matched the task performance of diffusion-based frameworks like Diffusion Policy and Diffuser while requiring 16x to 256x fewer inference steps.

Limitations and Open Questions

While the results demonstrate strong gains, several key assumptions and limitations remain important for practitioners:

Candidate Coverage Assumption: The success of best-of-$K$ training depends on the likelihood that at least one candidate among the $K$ generated samples lies reasonably close to the target mode. If $K$ is too small for an extremely complex distribution, candidates may fail to cover the ground-truth target.
Training-Time Compute Cost: Generating $K$ candidate outputs per step increases compute requirements during training. Although FLOP efficiency improves overall due to faster convergence, the per-step overhead during training is higher than single-candidate forward passes.
Ranking Dependency: The quality of mode commitment relies heavily on the accuracy of the matching loss metric used to evaluate candidates during step selection.

Practical Takeaways for ML Practitioners

For engineering teams working on generative architectures, Explorative Modeling offers a compelling perspective shift:

Rethink Training vs. Inference Complexity: If your deployment is constrained by high inference latency—such as real-time robotics or interactive image generation—shifting complexity into training-time exploration can drastically reduce required inference steps.
Address Mode Blurring at the Objective Level: When direct regression yields blurry or averaged outputs, modifying the training loop to search over candidate matches can allow single-step models to commit to sharp, distinct modes.
Leverage Exploration as a Scaling Lever: Beyond simply adding model parameters or collecting more data, increasing generative expressivity via candidate exploration ($K$) represents a practical third knob for scaling model quality.

For further technical details, check out the primary paper on arXiv:2607.27372 or follow broader trends in computer science research.

Decoupling Physical Control and Reasoning: DeepMind's Gemini Robotics 2 Architecture

Prabhakar Chaudhary — Tue, 04 Aug 2026 18:42:39 +0000

Why Decouple Reasoning from Motor Control

General-purpose robots have to pull off two very different jobs at once. They need to read a cluttered, full-room visual scene, hold a multi-minute plan in memory, and converse with a person — and, in the same instant, close a high-frequency control loop that keeps a balancing humanoid upright and moves a delicate hand without dropping whatever it holds. Cramming both jobs into a single end-to-end network forces uncomfortable trade-offs: the large context window you want for reasoning fights the low latency you need for torque control.

On July 28, 2026, Google DeepMind pushed directly against that trade-off with Gemini Robotics 2, followed on July 30 by Gemini Robotics ER 2. Rather than one monolithic network, the suite splits the problem across three specialized models — whole-body vision-language-action (VLA) control, high-level embodied reasoning, and on-device adaptation — each tuned to a different cadence and context size. The same modular thinking is visible across recent robotics and VLA research collected on the arXiv robotics listings and on Hugging Face Papers, where decomposed perception-planning-control stacks have become a recurring pattern. Understanding DeepMind's specific split clarifies why this architecture is gaining traction.

The Three-Model Split

ER 2: High-Level Task Reasoning

Gemini Robotics ER 2 is the cognitive planner of the stack. It is a vision-language model built for embodied reasoning: it ingests the live camera feed and a natural-language instruction, then decomposes a task that may run several minutes into structured sub-goals. Beyond planning, ER 2 manages dialogue with a human supervisor, interprets spatial context, and coordinates multiple robots operating in a shared workspace — deciding which sub-task gets handed to which platform.

Operating more slowly than the control layer (roughly a few times per second), ER 2 trades frequency for breadth of context. That separation matters: a reasoning model can afford to run a large context window and a careful forward pass because it is not on the critical 100 Hz balance-control path.

Gemini Robotics 2: Whole-Body VLA Controller

If ER 2 decides what to do, Gemini Robotics 2 decides how the hardware moves. As a vision-language-action (VLA) model, it drives motor control for bi-arm manipulators and full humanoid bodies, from feet to fingertips.

The model unifies dynamic balance with fine manipulation. It generates trajectories for whole-body actions like crouching, walking, and navigating cluttered spaces. For physical interaction, it controls a range of end-effectors: five-fingered, 22-degree-of-freedom (DoF) hands for delicate tasks such as tying knots, alongside two-fingered grippers for precise packing and placement.

On-Device 2: Rapid Embodiment Adaptation

Operating at the edge, Gemini Robotics On-Device 2 is an efficient VLA variant optimized for local execution directly on robot hardware. Its job is low-latency closed-loop control plus adjustment to hardware variation — the part of the problem most sensitive to per-robot kinematic quirks.

DeepMind reports that On-Device 2 can adapt to a new robot embodiment with only a few hours of operational data and fewer than 200 demonstration examples. That low-shot capability targets a real bottleneck in physical AI: retargeting fine control policies to new kinematics without collecting enormous demonstration datasets for every new morphology.

The Execution Loop

A physical goal requires continuous handoff across all three layers. Consider a user instructing a humanoid to clear a cluttered workspace and pack items:

Goal Planning — ER 2 reads the camera stream, analyzes the scene, breaks the multi-minute task into discrete sub-goals, and assigns them.
Action Dispatch — ER 2 passes structured intent targets to Gemini Robotics 2.
Trajectory Generation — Gemini Robotics 2 converts sub-goals into coordinated joint control, balancing walking stability with manipulator positioning.
Local Execution — On-Device 2 runs closed-loop control on edge hardware, correcting for surface friction and micro-delays in real time.

Conceptual pseudocode for that loop:

# Conceptual: Gemini Robotics 2 orchestration loop

class GeminiRoboticsOrchestrator:
    def __init__(self, er_model, vla_model, ondevice_model, hardware):
        self.er_planner = er_model          # Gemini Robotics ER 2
        self.vla_controller = vla_model     # Gemini Robotics 2
        self.ondevice_agent = ondevice_model # On-Device 2
        self.robot = hardware

    def execute_user_task(self, prompt: str):
        # 1. High-level planning via ER 2
        visual_state = self.robot.get_camera_feed()
        subtasks = self.er_planner.plan_multistep_task(prompt, visual_state)

        for subtask in subtasks:
            # 2. Runtime safety check
            if self.er_planner.detect_human_proximity(visual_state):
                if not self.er_planner.verify_collaborative_safety(subtask):
                    self.robot.trigger_emergency_stop()
                    return "Stopped: human safety boundary exceeded."

            # 3. Whole-body trajectory via the VLA controller
            motion_plan = self.vla_controller.generate_whole_body_trajectory(
                subtask=subtask,
                kinematics=self.robot.get_kinematic_spec(),  # e.g. 22-DoF hand + legs
                current_pose=self.robot.get_joint_states(),
            )

            # 4. Low-latency edge execution via On-Device 2
            while not motion_plan.is_complete():
                local_obs = self.robot.get_sensor_readings()
                adjusted_cmd = self.ondevice_agent.adapt_control_step(
                    planned_cmd=motion_plan.next_step(),
                    sensor_delta=local_obs,
                )
                self.robot.send_motor_commands(adjusted_cmd)

This is illustrative, not a reference implementation — but it captures the core idea: each model owns a layer, and the orchestrator just passes intent down and sensor data up.

Safety: The ASIMOV-Agentic Benchmark

Physical environments demand explicit safety verification, so DeepMind introduced the ASIMOV-Agentic benchmark to evaluate agent behavior under physical ambiguity. It measures three capabilities:

Uncertainty handling — recognizing when sensor data or environment cues are too weak for safe execution.
Unsafe tool-call refusal — refusing action requests that exceed mechanical limits or risk collision.
Human escalation — pausing to request human verification when task safety is ambiguous.

At runtime, ER 2 enforces active safety constraints: when a human is present, it detects them in real time and can trigger an automated emergency stop. This is safety enforced deterministically at the control layer, rather than only at the prompt level — a distinction that matters when a model can actuate physical hardware.

What's Actually Available

Developers evaluating the platform should separate launch claims from public access:

Gemini Robotics ER 2 — available via Google AI Studio and in private preview on the Gemini Enterprise Agent Platform.
Gemini Robotics 2 and On-Device 2 — accessible to early-access partners, not the general public.
Safety methodology — documented in DeepMind's Gemini Robotics 2 Safety Technical Report.

The low-shot adaptation claim (under 200 demonstrations in a few hours) is notable, but independent verification across varied third-party hardware will require broader deployment. For now, the architectural pattern is more immediately actionable than the specific weights.

Takeaways for Robotics Builders

Decouple semantic planning from control rate. High-level VLMs work best as asynchronous planners (around 1–5 Hz), leaving fast motor balance to dedicated low-latency controllers running at 100 Hz or more.
Put refusal logic in the control layer. As ASIMOV-Agentic suggests, safety needs deterministic refusal mechanisms where the actuators are driven — not just prompt-level instructions.
Abstract kinematics behind a hardware seam. Splitting general whole-body control from local on-device fine-tuning makes it far easier to port a policy to a new hand or leg configuration.

By separating long-horizon reasoning, whole-body motor control, and local hardware tuning into distinct models, Gemini Robotics 2 offers an actionable blueprint for building physical AI that can adapt across robot morphologies without forcing one network to do everything at once.

Why LLMs Still Struggle With Tabular Prediction

Prabhakar Chaudhary — Tue, 04 Aug 2026 18:41:59 +0000

Most business prediction problems do not arrive as prose. They arrive as rows: account attributes, transactions, sensor readings, test results, and a target column. For this kind of data, gradient-boosted trees and other conventional methods remain hard to displace.

A new paper, Why Large Language Models Fail at Tabular Prediction, asks a much more useful question than “can an LLM classify a table?”: what, specifically, breaks as the task becomes more like ordinary tabular machine learning?

The answer from the authors’ controlled experiments is input dimensionality. Their result matters because it separates a real limitation from several explanations that sound plausible but did not hold up in their tests.

The experiment was about prediction, not table chat

The paper evaluates frontier LLMs in a pure inference setup: a model receives labeled examples and must predict labels for new rows in a single generation pass. There is no fine-tuning, retrieval pipeline, tool calling, or agent loop to compensate for the base model. This is deliberately narrow. It asks whether a general-purpose language model can act as a direct tabular learner.

Across 31 benchmark datasets, the authors compare nine methods and 252 configured classical models. That scope is important: a weak result on one CSV is easy to explain away as prompt design or a quirky dataset. A consistent trend across many tasks is harder to dismiss.

The headline is not simply that LLMs lose to established tabular baselines. It is that their accuracy declines as the number of input dimensions grows, while the classical baselines in the study stay stable or improve. The paper therefore treats dimensionality as the central failure mode rather than an incidental property of difficult datasets.

Four popular explanations did not survive testing

There are several standard reasons developers give for poor LLM performance on tables. The researchers turn these into falsifiable hypotheses.

“The classes overlap too much.” If the data are noisy or not cleanly separable, perhaps the model cannot infer a decision boundary. The experiments indicate this is not enough to explain the gap: performance problems remain even when the signal is made easier to separate.

“CSV destroys the table structure.” Turning columns and rows into a token sequence feels like an obvious mismatch. But changing the serialization format did not fix the issue. The model can access column information, yet it still fails on signals that should be simple.

“Numbers are tokenized badly.” Numeric strings split into tokens in awkward ways, so perhaps magnitude comparisons are the culprit. Lowering numeric precision and related interventions did not produce the expected recovery.

“There are too many test rows in one prompt.” A prompt with many predictions might spread the model’s compute too thin. Again, reducing that load was not the explanation.

None of these findings says formatting, numeric representation, or context length never matter in production. They do. The narrower point is that they do not account for the observed collapse in this direct-prediction setting. That distinction should change how we debug these systems: switching from CSV to JSON is unlikely to solve a high-dimensional learning problem.

Dimensionality is a different kind of obstacle

Tabular features are often heterogeneous and only weakly related: a customer’s region, account age, device type, transaction count, income band, and dozens of derived signals may each contribute a little. As features accumulate, useful local similarity becomes difficult to identify. This is one form of the familiar curse of dimensionality.

In two dimensions, the paper finds that LLM behavior resembles local methods such as low-k nearest neighbors or short-length-scale Gaussian processes. In other words, the model’s predictions can look like a neighborhood-based decision rule. The reported grid agreement reaches 91.6% for some such comparisons.

At higher dimensions, that resemblance disappears. The authors could not reproduce the LLM’s behavior with any of their classical models, even after adding tuned dimension-dependent noise. That is a valuable negative result. The model is not merely becoming an ordinary nearest-neighbor classifier with worse settings; its prediction behavior becomes hard to characterize and less accurate.

This also clarifies why fluent explanations can be misleading. An LLM can describe a dataset, identify likely feature meanings, and write a plausible analysis plan. Those abilities do not imply that the model has acquired the inductive bias needed to estimate a reliable decision function over many structured variables.

What should developers use instead?

For supervised prediction on a table, start with methods designed for tables. Gradient-boosted decision trees are still a strong default for mixed numeric and categorical features, moderate dataset sizes, missing values, and non-linear interactions. XGBoost’s original paper remains useful background for why boosted trees became such a practical baseline.

There is also active work on models that are explicitly built for tabular prediction rather than adapted from next-token prediction. TabPFN-2.5, for example, is a tabular foundation model evaluated against conventional approaches. Its existence reinforces the design lesson: “foundation model” is not synonymous with “general-purpose LLM.” Architecture, training distribution, and task framing all matter.

General-purpose LLMs still have useful roles around a tabular workflow:

generate SQL, feature-engineering code, and experiment scaffolding;
translate business questions into measurable targets;
summarize data dictionaries and data-quality checks;
assist with model documentation and error analysis;
provide a natural-language interface to a model trained elsewhere.

Those are orchestration and interface tasks. They should not be confused with replacing the predictive model itself.

The practical takeaway

Do not treat a chat model’s ability to read a table as evidence that it can learn from the table. If the goal is accurate tabular prediction, establish a tree-based baseline first, measure it with a held-out evaluation protocol, and only then test an LLM-based approach against it.

The new paper is a preprint, so its conclusions deserve replication and extension. It also studies pure inference rather than fine-tuned or tool-augmented systems. But its experimental approach is worth adopting: isolate one explanation at a time, reject the convenient ones when the data rejects them, and focus on the structural constraint that remains.

For a broader map of this research area, see the survey on large language models for tabular data. The main message is restrained but useful: language models can make tabular workflows easier to operate, while the actual predictive core still needs methods with the right bias for the data.

DEV Community: Prabhakar Chaudhary

Meta Muse Glimmer-30B: How a Dense Local Model Is Rethinking On-Device Agentic AI

Meta Muse Glimmer-30B: How a Dense Local Model Is Rethinking On-Device Agentic AI

Why Dense Instead of MoE?

Architecture: Hybrid Attention and a Dedicated Perception Encoder

Speculative Decoding via DFlash

Agentic Capabilities: What "Built for Agents" Actually Means

Benchmark Performance

Deployment Options

What This Signals for Local Agentic AI

LFM2.5-2.6B: How Liquid AI Built an Agentic Model That Runs on Your Phone

LFM2.5-2.6B: How Liquid AI Built an Agentic Model That Runs on Your Phone

Why On-Device Agents Are Architecturally Different

The Hybrid Architecture: Fewer Attention Layers, Fixed-Size State

A Four-Stage Post-Training Pipeline

Benchmark Results: Where It Leads and Where It Doesn't

Inference Speed Across Hardware

What This Means for Agent Deployment

The Broader Context

What OpenAI’s Astra Math Results Teach Us About Verifiable AI Workflows

What OpenAI’s Astra Math Results Teach Us About Verifiable AI Workflows

The Astra Case Study: Machine-Checked Mathematics

The Trust Boundary: Model Exploration vs. Deterministic Verification

Generator vs. Verifier: Why Formal Verification Matters

What Developers Can Borrow

Limitations and System Caveats

From Prompting to Reasoning: How ToolArtist Unifies Multi-Step Logic and Image Generation

From Prompting to Reasoning: How ToolArtist Unifies Multi-Step Logic and Image Generation

The Problem of Passive Generation

The Architecture: Unified Multimodal Models (UMMs)

RAD-GRPO: Reinforcement Learning for Drawing Agents

The Training Pipeline

Why Agentic Generation Matters

Conclusion

Sources

Further Reading

ABSeeker: Solving the Credit Assignment Problem in Long-Horizon Search Agents

ABSeeker: Solving the Credit Assignment Problem in Long-Horizon Search Agents

The Challenge of Multi-Step Reasoning

Answer-Backtracked Credit Assignment (ABC)

1. Answer-Backtracked Clue Recovery

2. Clue-Anchored Step Scoring

Training ABSeeker: ABC-SFT and ABC-GRPO

Quantifiable Gains in Efficiency

Why Dense Rewards Matter

Sources

Further Reading

Inside GPT-Live: How OpenAI Rebuilt ChatGPT's Voice Stack for Full-Duplex Conversation

Inside GPT-Live: How OpenAI Rebuilt ChatGPT's Voice Stack for Full-Duplex Conversation

The Problem: Turn Detection Is a Bottleneck

The Fix: One Continuous Full-Duplex Model

Separating Speech from Everything Else

From Python to Go: Engineering for Consistent Latency

WARP: Cutting Session Startup from Six Round Trips to One

What Shadow Deployment Revealed About Capacity

What It Means for Developers

Further Reading

SparseSpec-L: How a Sparse KV Cache Makes Long-Context LLM Inference 2.79 Faster — Without Any Training

SparseSpec-L: How a Sparse KV Cache Makes Long-Context LLM Inference 2.79× Faster — Without Any Training

Why Long-Context Speculative Decoding Is Hard

The SparseSpec-L Approach

The Entropy-Based Speculation Controller

Benchmark Results

What This Means for Practitioners

WaiT for the Signal: Why Image Generators Should Build the Big Picture Before the Texture

The mismatch in ordinary generative schedules

What WaiT changes

A simplified mental model

Why this could matter at 512px and above

A better way to inspect generated images

Where developers can experiment

1. Profile work by frequency, not only by step

2. Treat previews and final renders differently

3. Add local-quality regression tests

The important constraint: detail cannot be an afterthought

Further Reading / Sources

Beyond Size: The Three Pillars of Test-Time Scaling in Large Language Models

Beyond Size: The Three Pillars of Test-Time Scaling in Large Language Models

The Three Regimes of Inference Compute

1. Single-Trajectory Sequential Scaling