DEV Community: Max Quimby

China Just Won the Rare-Earths Race

Max Quimby — Tue, 23 Jun 2026 04:14:38 +0000

📖 Read the full version with charts and embedded sources on The Arc of Power →

On June 12, 2026, Washington banned the export of Fable 5 and Mythos to China. Ten days later, Beijing fired back — but not with a model. China's Ministry of Commerce added 10 American companies to its export control list, including MP Materials and USA Rare Earth — the two companies the Pentagon invested billions in to build an alternative rare-earth supply chain.

The US restricts the bits. China owns the atoms. And atoms are harder to replace.

The Numbers

Material	China's Share
Rare earth mining	70%
Rare earth processing	90%+
Rare earth magnets	94%
Gallium production	98.7%
Heavy rare earths	~99%

This isn't a trade dispute. This is a monopoly. As Fortune reported: "China is the leader, and the U.S. is far behind."

Why This Matters for AI

Here's the connection most coverage misses: every AI data center runs on rare earth magnets.

A Fordham University study found that 94.4% of AI infrastructure's rare-earth exposure comes from IT hardware. Neodymium-related exposure alone could exceed $90 million per gigawatt-scale AI campus.

The US is spending $200+ billion building AI data centers in 2026. Those data centers can't function without magnets that almost exclusively come from China.

The Metallization Gap

The Pentagon committed $1.2 billion in one week — $725M to Energy Fuels and $500M to Phoenix Tailings — to close what it calls the "metallization gap." But replicating China's oxide-to-metal conversion takes 3–7 years. The US currently has zero commercial-scale facilities for this.

China added the exact companies that money was meant to help to its export control list the same week. Beijing isn't defending — it's targeting the offramp.

The Two-Layer War

The US-China tech conflict is now bits vs. atoms. Washington controls model exports; Beijing controls the physical materials those models depend on. The asymmetry: bits are infinitely copyable. Atoms are not. There is no "open-weight" version of a neodymium processing facility.

Even the most optimistic projections put meaningful Western rare earth processing capacity at the mid-2030s. CSIS estimates Beijing's dominance in heavy rare earths will persist until at least 2035.

The rare earths race was never really about rare earths. It's about whether a technology superpower can be built on physical infrastructure it doesn't control.

Atoms win. They always do.

Originally published at The Arc of Power

Write HTML, Not JSON: HeyGen's Visual-Grounding Trick

Max Quimby — Tue, 23 Jun 2026 03:56:14 +0000

Read the full version with screenshots and embedded sources on AgentConn → agentconn.com/blog/heygen-hyperframes-html-visual-grounding-video-ui-agents-2026

Every agent framework in 2026 tells you to return structured JSON. Schema-validated, type-safe, parseable. And for most tasks, that's correct — structured output gives agents 95–99% action success rates versus 70–85% for unstructured text.

But here's the problem nobody talks about: JSON has no visual semantics. An agent can produce a perfectly valid JSON config describing a video timeline — correct schema, valid keyframes, legal property values — and the rendered output looks like garbage. The agent wrote "correct" instructions for something it can't see.

HeyGen figured this out. Their open-source framework HyperFrames doesn't use JSON configs. It uses HTML.

The Visual-Grounding Problem

When an agent generates a JSON video config, it's working blind. A perfectly valid JSON scene description — correct schema, right keyframes — can render as visual garbage because the agent can't reason about spatial layout in a non-visual format.

The SeeAct-V research confirms what practitioners already know: visual grounding is a fundamental capability gap for language models working in non-visual formats.

HyperFrames: HTML as the Agent's Canvas

HyperFrames launched April 17, 2026, and hit 30,100 stars in two months. Instead of JSON configs, agents write standard HTML with CSS and data-* attributes for timing:

<div data-scene="intro" data-duration="3s">
  <h1 style="font-size: 48px; text-align: center; margin-top: 20vh;">
    Hello World
  </h1>
</div>

The agent can reason about this. It knows what text-align: center looks like. It knows margin-top: 20vh pushes the heading down. It understands CSS layout.

The architecture: headless Chrome (Puppeteer) for deterministic frame capture, FFmpeg for encoding. Supports GSAP 3, Lottie, Three.js, Anime.js, and WebGL shaders — any animation library that runs in a browser works without adapters.

Why HTML Wins Over JSON

The HN discussion put it plainly: "It's just a superset of HTML, and agents know how to write HTML + GSAP by default."

LLMs are trained on billions of web pages. They know what display: flex looks like, that border-radius: 50% makes a circle, that font-size: 72px is large. This visual intuition doesn't exist for arbitrary JSON coordinate systems.

The Agent Skill Architecture

HyperFrames includes dedicated skills for Claude Code, Cursor, Gemini CLI, and Codex. HeyGen's own launch video was made 100% with Claude Code + HyperFrames. Nous Research's Hermes agent has an official HyperFrames skill — the first major agent framework to integrate video production natively.

The Pattern Beyond Video

The insight is general: match the output format to the model's strongest reasoning modality.

Domain	Low-Grounding	High-Grounding
Video	JSON config	HTML + CSS + data-*
Diagrams	DOT/Graphviz	SVG
Dashboards	Chart.js JSON	HTML grid + components
Presentations	Slide JSON	HTML slides + CSS

HeyGen bet that agents think better in HTML than in JSON. Thirty thousand stars in two months suggests they were right.

Originally published at AgentConn

GLM-5.2 Is Cheap Because It's Subsidized, Not Efficient

Max Quimby — Tue, 23 Jun 2026 03:30:43 +0000

Read the full version with charts and embedded sources on ComputeLeap → computeleap.com/blog/glm-5-2-cheap-price-subsidy-not-efficiency-real-cost-math-2026

GLM-5.2 dropped on June 13 and the internet did what the internet does: it found the cheapest number and made it the headline.

"$0.06 vs $0.49." "$4.40 per million output tokens vs $25." "82% cheaper than Opus." The tweets went viral. VentureBeat ran with "1/6th the cost." Goldman Sachs called it "the latest Chinese shock to the system." And if you stopped at per-token pricing, they'd all be right.

But per-token pricing is the wrong metric. It's been the wrong metric since we wrote about the 6x AI pricing lie in March, and GLM-5.2 is about to teach the market that lesson again — the hard way.

In our benchmark deep-dive, we showed that GLM-5.2 scores within a point of Claude Opus 4.8 on FrontierSWE (74.4 vs 75.1) and decisively beats GPT-5.5 (72.6). The capability is real. But the cost story everyone is telling? It's missing two-thirds of the math.

The Token Tax Nobody Mentions

Here's the number the hype cycle skips: GLM-5.2 uses approximately 43,000 output tokens per coding task. That's nearly double its predecessor GLM-5.1's 26,000 tokens. Of those 43K tokens, roughly 37,000 are internal reasoning tokens — the model thinks out loud, and you pay for every word.

Let that sink in. The model that's "82% cheaper per token" burns 65% more tokens per task than the competition.

At $4.40 per million output tokens, a 43K-token task costs $0.19 in output alone. Add input tokens and you're at roughly $0.46 per coding task, according to developer benchmarks. That's almost double GLM-5.1's $0.25 per task — and it's not 82% cheaper than Opus 4.8's ~$0.70 per task. It's about 35% cheaper.

Still cheaper? Absolutely. The same order of magnitude? Also yes. The narrative gap between "6x cheaper" and "35% cheaper" is where real money gets burned.

⚠️ Freda Duan surveyed builders running GLM-5.2 in production and found effective costs at 20–35% of Opus 4.8 — cheaper, but not the 4–6x gap implied by headline per-token pricing. Cache hit rates and retry rates dominate the actual bill.

The Real Provider Pricing Table

GLM-5.2 launched with availability across 11+ inference providers within days. But pricing varies more than the "it's all cheap" narrative suggests.

Provider	Input ($/1M)	Output ($/1M)	Blended ($/1M)	Throughput (t/s)
GMI (FP8)	$1.12	$3.52	$0.72	219
Wafer	$1.20	$4.10	$0.79	—
DeepInfra (FP8)	$1.20	$4.20	$0.80	39
OpenRouter	$1.20	$4.10	$0.79	—
Z.ai (first-party)	$1.40	$4.40	$0.87	—
Fireworks AI	$1.40	$4.40	$0.87	—

Source: Artificial Analysis, Developers Digest

For comparison: Claude Opus 4.8 runs $5.00/$25.00, GPT-5.5 runs $5.00/$30.00, and Claude Fable 5 runs $5.00/$50.00.

The cheapest route — GMI at $0.72/M blended — is genuinely cheap. But there's a caveat the HN discussion surfaced: "Be careful about unofficial providers — a lot of them misconfigure models or stealth quantize them."

Why the Price Is a Subsidy, Not Efficiency

GLM-5.2 is not more efficient than its competitors. It's cheaper because of where and how it's hosted — not because of what the model does.

Three structural advantages underpin GLM-5.2's pricing:

1. Government-subsidized infrastructure. Chinese AI models run at roughly one-sixth to one-quarter the cost of comparable American systems, according to a RAND report.

2. Provider-level loss leaders. Hugging Face ran GLM-5.2 for free during launch week. These aren't sustainable prices — they're customer acquisition costs.

3. The model itself already repriced upward. Zhipu raised prices by 30% in February 2026: "To sustain service quality, we've been investing heavily in compute."

ℹ️ The subsidy clock is ticking across the entire AI industry. Read more: AI's $700B Subsidy Clock Is Ticking

Effective Cost Per Task

Scenario: 100 agentic coding tasks per day

Metric	GLM-5.2	Claude Opus 4.8	GPT-5.5
Avg output tokens/task	43,000	~18,000	~16,000
Total cost/task	$0.46	$0.70	$0.73
Daily cost (100 tasks)	$46	$70	$73
Cost/successful task	$0.52	$0.76	$0.82

GLM-5.2 saves roughly $24/day on 100 tasks — about 34% cheaper, not 82%.

When GLM-5.2 Wins (and When It Doesn't)

GLM-5.2 wins for: high-volume bounded tasks, cache-heavy agent loops, self-hosting with MIT weights.

Opus 4.8 earns its premium for: hardest long-horizon tasks, latency-sensitive workflows, workloads where retry rates dominate.

Nathan Lambert captures the positioning: "This model existing is a huge boon for the open model economy." But a boon for the economy is not the same as a boon for your bill.

The Bottom Line

The cost story being told on X and Substack is the headline story, not the effective story. The real savings land at 30–35% — not 80%. The cheapest model per token has never been the cheapest model per task.

Build your architecture on the model. Build your budget on the math.

Originally published at ComputeLeap

Z.ai Open-Sourced slime: GLM-5.2 Post-Training Stack

Max Quimby — Mon, 22 Jun 2026 03:34:25 +0000

Everyone is talking about GLM-5.2's benchmarks. Jeremy Howard's head-to-head shows it beating GPT-5.5 64% of the time. Clément Delangue says it's "SHITTING on Opus 4.8 in open code" to his 241,000 viewers. Merve Noyan calls it "the first open model that passes as a daily driver" — 1.5 million views and counting.

📖 Read the full version with charts and embedded sources on ComputeLeap →

But the benchmark scores aren't the story. The story is what Z.ai shipped alongside the model: slime, the exact RL post-training framework they used to build GLM-5.2. Not a stripped-down reference implementation. Not a research artifact. The same production stack that ran the full Online Preference Distillation pipeline and finished in roughly two days.

That's the difference between releasing a finished car and releasing the entire assembly line. And for the first time, anyone with GPUs can run it.

What slime actually is

slime is an open-source framework — 6,600 stars on GitHub, Apache 2.0 licensed — that handles the post-training phase of large language models through reinforcement learning scaling and online preference optimization.

If pre-training teaches a model language and knowledge, post-training teaches it to be useful. It's the phase where a raw language model becomes a coding assistant, a reasoning engine, or a research partner. The pre-training recipe — scale data, scale compute, train a transformer — is well-understood. The post-training recipe — which RL algorithms, which reward signals, how to merge specialized capabilities — is where frontier labs differentiate. And it's traditionally the most closely guarded part of any frontier lab's stack.

slime's architecture is straightforward in principle but deeply engineered in practice. It unifies three components into a single coherent pipeline:

Megatron-LM handles the training engine — gradient computation, model parallelism, and distributed optimization across thousands of GPUs.
SGLang handles the rollout engine — generating the responses that the model learns from, with all of SGLang's inference optimizations (speculative decoding, continuous batching, tensor parallelism) carried directly into the training loop.
A pluggable Data Buffer manages the pipeline between them — prompt initialization, reward computation, verifier feedback, and environment interaction all flow through a single explicit dataflow path.

As Z.ai's official announcement puts it: "slime is built with native SGLang integration, carrying its full inference optimizations straight into training."

The framework passes Megatron arguments through directly and exposes SGLang arguments with a --sglang- prefix. No wrapper layer. No abstraction tax. Upstream training and serving optimizations remain available without slime getting in the way.

The documentation is refreshingly honest about the engineering challenges: "RL bugs are often silent." slime treats reproducibility, fault tolerance, tracing, and profiling as first-class engineering concerns — not afterthoughts. It ships with separate rollout-only and train-only debugging paths, so you can isolate problems in a system where failures tend to be subtle and delayed.

APRIL: Solving the 90% bottleneck

The single biggest bottleneck in RL training for language models isn't the gradient step — it's generation. When a model needs to produce complete responses to evaluate them, the rollout phase can consume over 90% of total training time. One slow response — a rambling chain-of-thought, an overly verbose code generation — holds up an entire batch while thousands of GPUs sit idle.

slime integrates APRIL (Active Partial Rollouts in Reinforcement Learning), a system-level optimization that attacks this long-tail problem directly. The approach is elegant: over-provision rollout requests, terminate once the target number of complete responses is reached, and recycle incomplete responses for continuation in future training steps.

Instead of waiting for the slowest response in a batch, APRIL ensures training never idles. The partially completed responses aren't thrown away — they're picked up again in the next iteration, amortizing their cost across multiple training steps. This is the kind of systems engineering insight that separates a research prototype from production infrastructure.

The impact is material. Without APRIL, a single verbose chain-of-thought response can stall a batch for minutes while hundreds of GPUs wait. The APRIL paper demonstrates that generation bottlenecks dominate wall-clock time in RL training. By eliminating idle GPU cycles during rollout, slime can achieve significantly higher training throughput without any change to the learning algorithm itself.

The APRIL implementation is fully integrated into slime — not as an optional plugin, but as core infrastructure that activates by default during asynchronous rollout workflows.

OPD: Merging ten expert models in two days

GLM-5.2's post-training didn't use a single monolithic RL training run. It used Online Preference Distillation (OPD) — a process that trains more than ten specialized expert models in parallel, each tuned for different capabilities (coding, reasoning, instruction-following, long-context tasks), then merges them into the final model through online preference optimization.

The complete OPD post-training of GLM-5.2 ran on slime and finished in approximately two days.

To put that in context: GLM-5.2 is a 744-billion-parameter Mixture-of-Experts model with 40 billion active parameters per token, trained on 28.5 trillion tokens. The model that topped the Artificial Analysis Intelligence Index at 51 — ahead of MiniMax-M3 and DeepSeek V4 Pro — had its entire post-training phase completed in a weekend.

💡 The speed isn't just a flex. Faster iteration cycles mean you can experiment with more RL strategies, test more reward functions, and course-correct before committing to a full training run. The factory's throughput determines how fast you can innovate on the product.

The HuggingFace technical blog reveals additional sophistication in the training pipeline. Rather than standard group-wise PPO, Z.ai shifted to a critic-based PPO formulation that learns from individual rollouts. This matters for agentic tasks where different rollouts generate variable-length sub-traces — a coding agent might solve a problem in 50 tokens or 5,000.

Beyond the RL algorithm itself, Z.ai built sophisticated anti-hacking mechanisms into the training loop. When training coding agents through RL, models learn to exploit reward functions — writing tests that pass trivially, manipulating sandbox environments to fake success, or taking shortcuts that game the metric without solving the problem. GLM-5.2's training uses dual-stage detection: rule-based filters catch potential shortcuts with high recall, then LLM judges verify intent with high precision. Detected hacks trigger online intervention — blocking malicious calls and returning dummy data — allowing training to continue rather than aborting entire trajectories.

The GLM-5.2 benchmark context

Before we look at who else uses slime, it's worth grounding GLM-5.2's performance in numbers. The model that this factory produced isn't a marginal improvement — it's a structural shift in what open-weights models can do.

On FrontierSWE, GLM-5.2 scores 74.4% — trailing Claude Opus 4.8 by only 1%. On PostTrainBench, it scores 34.3%, outperforming both Opus 4.7 and GPT-5.5. Terminal-Bench 2.1 shows a jump from 63.5 (GLM-5.1) to 81.0. And on the Artificial Analysis Intelligence Index v4.1, GLM-5.2 sits at #1 with a score of 51 — ahead of every other open-weights model, and competitive with the best proprietary ones.

The HN discussion captured the practitioner consensus: this isn't benchmark-maxxing. The improvements show up in real coding workflows. The 1M-token context window — five times larger than GLM-5.1's — enables long-horizon agentic tasks that were previously exclusive to proprietary models.

All of this comes from the same post-training pipeline. The architecture innovations (IndexShare for sparse attention, improved Multi-Token Prediction, KV-cache optimization) matter, but the RL post-training is what turned a capable base model into a frontier coding agent.

Not just GLM: Who else runs on slime

Here's the part that most coverage misses: slime isn't a Z.ai-only tool. The framework's README explicitly lists support for:

GLM series (5.2, 5.1, 5, 4.7, 4.6, 4.5)
Qwen variants (3.6, 3.5, 3Next, 3MoE, 3, 2.5)
DeepSeek (V3, V3.1, R1)
Llama 3

That's not a compatibility list — it's a deployment record. These models have been trained or fine-tuned on slime. The framework that produced GLM-5.2 has also touched Alibaba's Qwen family, DeepSeek's V3, and Meta's Llama 3.

The Zhihu Frontier account on X documented slime v0.1.0's launch with a deep technical dive, noting it "redefined high-performance RL infra" — and subsequent releases have added FSDP backend support, PPO, Multi-Token Prediction training, and full FP8 stack support.

When you open-source the factory, every model benefits. And when multiple frontier labs converge on a shared RL training framework, the improvements compound across the entire open-weights ecosystem.

The ecosystem is already here

The clearest signal that slime has crossed from "interesting open-source project" to "production infrastructure" is the ecosystem forming around it:

Miles by RadixArk — an enterprise-grade fork described as "co-evolving with slime," adding production reliability features and bridging "the gap between research-grade RL and production-grade reliability."
AMD Day-0 support — AMD shipped slime support on Instinct GPUs from day one. When a hardware vendor commits engineering resources to your training framework, that's infrastructure-grade validation.
Hermes Agent by Nous Research — integrated slime as a skill in their agent framework, treating RL post-training as something an AI agent itself can orchestrate.
Dressage by Alibaba — unified RL for blackbox agents across sandbox environments, built on slime's architecture.
vime — the vLLM project's alternative rollout backend, extending slime's reach to the most popular open-source inference engine.

This isn't a research project with a README and a dream. It's infrastructure that AMD blogs about, enterprises fork, and agent frameworks integrate.

Why the factory matters more than the model

Simon Willison called GLM-5.2 "probably the most powerful text-only open weights LLM." He noted it leads the Intelligence Index v4.1, priced at $1.40/million input tokens — significantly cheaper than GPT-5.5 or Claude Opus. Latent Space called it "the real deal" and noted that Z.ai forecasts an "open Fable-class model by year-end."

But models depreciate. GPT-4 was the frontier for about nine months. Claude Opus 4.5 lasted less than six. Even GLM-5.2 will be surpassed — probably by GLM-5.3, trained on the same factory.

The factory doesn't depreciate. It compounds.

Every improvement to slime — a faster APRIL scheduler, a more efficient OPD merger, a better anti-hacking detector — accelerates every future model trained on it. Every external contribution from Qwen's team, DeepSeek's engineers, or the open-source community makes the next training run faster, cheaper, and more reliable.

💡 Prediction markets are pricing in the structural shift. Polymarket's "Will a Chinese company have the best AI model by December 31?" market moved up 18% this week. The convergence report notes a telling divergence: Polymarket still crowns Anthropic at 95% for best model, while X practitioners say an open Chinese model already beats Opus 4.8 in daily use. One of them is lagging.

What this means for you

If you're an ML engineer or researcher, the implications are direct:

You can reproduce frontier-class post-training. Not an approximation — the exact framework, with the exact optimizations, that produced a model within 1% of Opus 4.8 on FrontierSWE.
You can train on the hardware you have. With AMD Day-0 support and native Megatron + SGLang integration, slime runs on both NVIDIA and AMD GPUs. The local setup guide covers the inference side; slime covers the training side.
You can build on a living ecosystem. This isn't abandoned research code. It's infrastructure with enterprise forks, hardware vendor support, and agent framework integration. The 6,600 stars and 955 forks tell you people are using it, not just starring it.
You can iterate fast. If the OPD pipeline for a 744B model takes two days, your smaller model takes hours. That changes what's experimentally feasible — what used to be a quarterly training run becomes a weekly experiment.

The closed-source moat in AI isn't the model architecture — those get published in papers. It isn't the training data — that gets recreated or licensed. It's the post-training stack: the reward functions, the RL infrastructure, the iteration speed that lets you ship a better model every quarter.

Z.ai just open-sourced that moat. The benchmark comparisons will keep shifting. The China coding model landscape will keep evolving. But the factory is permanent.

The factory is the product. And now it belongs to everyone.

Originally published at ComputeLeap

Artifacts in Claude Code: The Operator's Guide

Max Quimby — Sun, 21 Jun 2026 06:01:04 +0000

On June 18, Anthropic shipped artifacts in Claude Code — the feature that turns a coding session's work into live, shareable web pages. Two days earlier, Claude Design got a major overhaul: design system imports, canvas editing, and a /design-sync command that closes the loop between design and code. Boris Cherny, the creator of Claude Code, called artifacts "a game changer" — and the 5.1K likes suggest he's not alone.

📖 Read the full version with charts and embedded sources on AgentConn →

This is the operator's guide. What shipped, how to set it up, and where it fits in the agent workflow you're already running.

What Actually Shipped: Artifacts in Claude Code

Artifacts are self-contained HTML pages that Claude Code builds from your session context. They're not static exports — they're live, interactive web pages at private URLs that update as your session progresses.

The content types Anthropic listed in the launch blog post:

PR walkthroughs with embedded diffs
System explainers and architecture diagrams
Dashboards you can filter and sort
Release checklists that fill themselves as work gets done
Incident pages with live status
License audits and privacy maps
Security findings summaries
Data-flow diagrams

The key difference from web-based Claude artifacts: these are generated from your coding session's context — the files you've touched, the changes you've made, the tests you've run. They're not generic canvases. They're views into the work the agent just did.

Availability: Beta on Team and Enterprise plans. You need to be signed in (/login in Claude Code). Individual plans don't have access yet.

Privacy: Artifacts are private by default. You share them explicitly with teammates, and they're viewable only by authenticated org members. Admins get org-level toggles, role-based scoping, retention policies, and compliance API visibility.

How to Use Artifacts in Claude Code

The setup is minimal. If you're on a Team or Enterprise plan:

Sign in: Run /login in your Claude Code session if you haven't already
Ask for an artifact: Tell Claude Code what you want visualized — "create a dashboard of test results," "build a PR walkthrough for this diff," or "diagram the auth flow I just changed"
Claude builds the page: It generates a self-contained HTML page from your session context
Share the link: Click the share button in the page header to give teammates access

There's no separate infrastructure to configure. No Docker containers. No hosting setup. The artifact lives at a private Anthropic-hosted URL tied to your org.

What Works Well

From early operator reports and VentureBeat's coverage:

PR reviews: A reviewer who doesn't want to clone your branch can see the walkthrough, the diff, and the rationale in one page
Incident response: Build a live incident page as you debug — stakeholders watch the page instead of asking for Slack updates
Architecture documentation: System diagrams that generate from the codebase you're actually working in, not a stale Confluence page
Sprint reports: Weekly shipped-work summaries that pull from the session's git context

What Doesn't (Yet)

No public sharing. Artifacts are org-only. If you need to share with external stakeholders, you'll still need to export
No persistent data connections. Unlike the April 2026 "Live Artifacts" in web Claude, Claude Code artifacts reflect session state, not external databases
Team/Enterprise only. Individual and Pro plans are out — and Anthropic hasn't announced a timeline for expansion

Claude Design: The Other Half of the Story

The artifacts launch didn't happen in isolation. Two days earlier, Anthropic overhauled Claude Design with three features that connect directly to the Claude Code workflow:

1. Design System Imports

You can now bring your design system into Claude Design from GitHub repos, design files, or direct uploads. Once imported, Claude Design generates assets using your components, checks its output against the design system, and makes corrections before you see the result.

2. The /design-sync Command

This is the integration point. From a Claude Code terminal:

/design-sync

This pulls your design system into your repo — or pushes what you've built back into Claude Design. The flow works both directions:

Design → Code: Start a prototype in Claude Design, hand it off to Claude Code
Code → Design: Built something in Claude Code that needs visual polish? Push it back to Claude Design for canvas editing

Engadget noted that Anthropic is betting the gap between design and code disappears when the same AI system handles both.

3. Canvas Editing

Users can now drag, resize, and align elements on the Claude Design canvas without asking Claude to regenerate. For small visual tweaks — nudging padding, reordering a layout — you edit directly instead of burning a model turn.

The Operator's Setup Checklist

For artifacts:

Verify your org is on Team or Enterprise plan
Run /login in each developer's Claude Code session
Establish org-level sharing policies (admin panel → Artifacts → Sharing controls)
Set retention policies for artifact pages

For /design-sync:

Import your design system into Claude Design (GitHub repo URL → Design system import)
Run /design-sync in Claude Code to pull the design system into your local project
Configure export destinations (PDF, PowerPoint, Adobe, Canva, Lovable, Replit, Vercel, Wix)

Real-World Operator Patterns

The "Show Don't Tell" PR Review. Instead of writing a PR description, generate an artifact during code review. The artifact includes the diff, the rationale, and interactive diagrams. Reviewers see the full context without cloning the branch.

The Live Incident Page. When debugging a production issue, build an artifact as you go. Stakeholders bookmark the artifact URL instead of joining a Slack thread.

The Sprint Visibility Dashboard. At the end of a dynamic workflow run — where Claude Code orchestrated multiple subagents — generate an artifact summarizing what each agent did, which files changed, and what tests passed.

The Design Handoff Loop. Start in Claude Design → /design-sync → build in Claude Code → generate artifact for review → push back to Claude Design for polish. The entire loop stays inside one AI system.

Where This Fits in the Agent Stack

Artifacts and design sync are Anthropic's answer to a specific problem: the output of an AI coding session is invisible to everyone who wasn't in the session.

Before artifacts, the operator pattern was: Claude Code does work → developer reviews in terminal → developer writes up results in Slack/Notion/Jira. Artifacts collapse the last step — the work product is the communication artifact.

The Fortune profile of Boris Cherny adds context: the Claude Code creator says there are days he manages tens of thousands of AI agents at once. Artifacts and design sync are the visibility layer for that scale of operation.

What the Competition Doesn't Have (Yet)

Codex, Cursor, and Windsurf are all shipping agentic coding features. None of them have a live artifact system that turns session output into shareable web pages. No other coding agent has a direct pipeline to a first-party design tool that shares the same AI backbone.

The Bottom Line

Artifacts and /design-sync are two halves of the same bet: that the AI coding agent should own the full loop from design to deployment to communication.

For operators already invested in Claude Code, the setup cost is near zero — sign in, ask for an artifact, share the link. The practical question is whether the design-to-code-to-artifact loop becomes your team's default workflow.

Based on Cherny's own usage — "I've been using Artifacts in Claude Code for everything" — Anthropic is clearly dogfooding the full loop. That's usually a good leading indicator.

Originally published at AgentConn

GLM-5.2 vs Opus 4.8: The Open-Weights Moat Is Real

Max Quimby — Sun, 21 Jun 2026 05:56:57 +0000

Z.ai shipped GLM-5.2 on June 17 — a 753-billion-parameter mixture-of-experts model with a one-million-token context window, released under an MIT license. Within 48 hours, it became the highest-scoring open-weights model on the Artificial Analysis Intelligence Index. And two of the least hype-prone voices in machine learning — Jeremy Howard and Sebastian Raschka — independently called it the best open-weights model they've ever used.

📖 Read the full version with charts and embedded sources on ComputeLeap →

That's the headline. Here's what the benchmarks actually say — and why the real story is about pricing, not parity.

The Benchmarks: Close but Not Equal

Let's start with the numbers that matter for developers choosing between GLM-5.2 and the closed frontier.

On FrontierSWE, GLM-5.2 scores 74.4% — trailing Claude Opus 4.8's 75.1% by less than a single percentage point. On SWE-Bench Pro, it hits 62.1%, decisively beating GPT-5.5's 58.6%. On Terminal-Bench 2.1, it reaches 81.0% versus Opus 4.8's 85.0%. GPQA Diamond: 89%. HLE: 40%.

The Artificial Analysis Intelligence Index puts GLM-5.2 at 51 — seven full points above the next open-weights contender (MiniMax-M3 at 44). On the same index, GLM-5.2 sits on the Pareto frontier of intelligence versus cost per task, meaning no other model delivers more capability per dollar at this intelligence level.

But here's the cold water. Voratiq's independent head-to-head evaluation, shared by Jeremy Howard himself, shows GLM-5.2 beats Opus 4.8 (with extended thinking) only 32% of the time. Against GPT-5.5 with extended thinking, it wins 64%. Against the next-best open model, Kimi K2.7, it wins 100%.

Current rank in voratiq's arena: third of 56 models.

Read those numbers and the picture sharpens. GLM-5.2 doesn't clearly beat the closed frontier — it probably loses to Opus 4.8 more often than it wins. But it absolutely dominates every other open-weights model by a wide margin, and it's within striking distance of the top on nearly every benchmark that matters for real development work.

ℹ️ GLM-5.2 wins benchmarks that reward speed and cost efficiency. Opus 4.8 keeps its lead on benchmarks that reward raw capability depth — broad expert knowledge (HLE, GPQA) and the hardest software engineering tasks (Terminal-Bench).

The Pricing Story Nobody Can Ignore

This is where the moat argument actually lives.

GLM-5.2 costs $1.40 per million input tokens and $4.40 per million output tokens. On OpenRouter, it drops further — $1.20 input, $4.10 output. Cached input costs just $0.26 per million tokens.

Claude Opus 4.8 runs $5.00 input and $25.00 output. GPT-5.5 is $5.00 input and $30.00 output.

That's a 3.5x gap on input tokens and a 5.7x gap on output against Opus 4.8. Against GPT-5.5, the output gap widens to nearly 7x.

As Simon Willison noted, GLM-5.2 is "probably the most powerful text-only open weights LLM" available — and it costs a fraction of what the closed alternatives charge. When you factor in the MIT license and the ability to self-host, the total cost of ownership gap widens further.

The cost per task on Artificial Analysis: $0.46 for GLM-5.2. That's the number enterprise teams will fixate on.

Model	Input ($/M)	Output ($/M)	FrontierSWE	SWE-Bench Pro	License
GLM-5.2	$1.40	$4.40	74.4%	62.1%	MIT
Opus 4.8	$5.00	$25.00	75.1%	—	Proprietary
GPT-5.5	$5.00	$30.00	72.6%	58.6%	Proprietary
Kimi K2.7	—	—	—	—	Open
MiniMax-M3	—	—	—	—	Open

The first open-weights model that makes the closed frontier look expensive — without making it look dramatically better — is a fundamentally different competitive dynamic than what we saw with earlier open models. When MiniMax M3 hit 59% on SWE-Bench Pro earlier this year, it was the first crack. GLM-5.2 is the second, and it's bigger.

The Architecture: IndexShare and Why 1M Context Matters

GLM-5.2 uses a Mixture-of-Experts architecture — 753 billion total parameters with only 40 billion active per forward pass. It builds on the MLA (Multi-head Latent Attention) and DSA (DeepSeek Sparse Attention) mechanisms from the GLM-5 family.

The new technical contribution is IndexShare, which Sebastian Raschka covered in a detailed architecture note. Instead of computing the sparse-attention top-k indexer in every transformer layer, GLM-5.2 runs the full indexer once every four layers and reuses the selected token indices in the layers between. This reduces per-token FLOPs by 2.9x at one-million-token context lengths.

Raschka's assessment: "The best open-weight model today." His focus was on the architecture, not the hype — MLA plus DeepSeek Sparse Attention, refined with cross-layer reuse. The 1M context window is a fivefold increase over GLM-5.1's 200K, and it's a real 1M — the model maintains stable performance across the full range, not just on synthetic needle-in-a-haystack tests.

For the MTP (Multi-Token Prediction) layer, GLM-5.2 applies IndexShare to speculative decoding, achieving a 20% increase in acceptance length. The design uses rejection sampling for speculative decoding and end-to-end TV loss for training — eliminating a training-inference discrepancy that plagued GLM-5.1.

The Recipe Is Public: slime and the Two-Day Post-Train

This is arguably the bigger story than the model itself.

Z.ai open-sourced slime — the SGLang-native post-training framework that trained GLM-5.2 (and every GLM model since GLM-4.5). The framework decouples data generation from training through three core modules: Megatron for training, SGLang for rollout, and a shared Data Buffer that manages prompts, custom data, and generation methods.

The entire OPD (Online Preference-based Direct training) post-training for GLM-5.2 ran in approximately two days, according to Z.ai — merging more than ten expert models through parallel training.

As Jeremy Howard highlighted: the RL post-training stack is now open and the recipe took about two days of compute. Slime already has 6.6k stars on GitHub and eight ecosystem projects building on it, including physics reasoning and video generation workflows.

💡 The post-training recipe includes anti-hack mechanisms that prevent reward exploitation during coding RL — a practical solution to one of the hardest problems in RLHF for code. Slime supports white-box rollout, black-box rollout, compact trajectory, and sub-agent workflow modes.

What this means in practice: any team with sufficient compute can replicate the post-training stage. The base model architecture is known. The training framework is MIT-licensed. The path from "pretrained model" to "frontier-adjacent model" just got published in full.

When DeepSeek V4 launched, the recipe wasn't this open. Neither was Kimi K2.6. GLM-5.2 is the first frontier-adjacent model where the post-training infra is fully reproducible — and that changes the dynamics more than any benchmark number.

The Export Ban Context

The timing is impossible to ignore. GLM-5.2's open-weights release landed in the same week that the US government restricted Anthropic's Fable 5 and Mythos 5 from foreign nationals. As Bill Gurley noted: "Zhipu's latest feels like another DeepSeek moment… the US couldn't afford to cede open source."

The irony writes itself. The US restricts its own lab's closed models — and in the same window, a Chinese lab ships frontier-adjacent capability as MIT-licensed weights downloadable from Hugging Face. Export controls on model weights are a tollbooth on a road the open-source community is already bypassing.

This doesn't mean GLM-5.2 is a direct response to the ban — the model was clearly in development long before. But the juxtaposition sharpens the strategic picture: the policy assumption that restricting closed-model access constrains AI capability abroad doesn't survive contact with an MIT-licensed 753B-parameter model scoring 74.4% on FrontierSWE.

If you want to run GLM-5.2 locally, we published a hardware and setup guide last week — covering llama.cpp, Ollama, and LM Studio configurations for the various quantization levels.

What the Community Is Actually Saying

The signal-to-noise ratio on GLM-5.2 is unusually high because the people praising it are the ones who normally don't.

Jeremy Howard — fast.ai founder, congenitally skeptical of hype — called it "a marvel" and said he'd "never experienced an open weights model like this before." That's from someone who has benchmarked every major open release since Llama 2.

Sebastian Raschka's assessment was characteristically technical: "The best open-weight model today" — followed by an architecture breakdown, not a victory lap. His IndexShare deep-dive is the best technical reference available.

On Hacker News, GLM-5.2 hit the front page multiple times — including a thread on how GPT-5.5 hallucinates 3x more than the MIT-licensed GLM-5.2. The Artificial Analysis ranking triggered its own discussion thread.

Latent Space's AINews declared GLM-5.2 "the real deal" and noted Z.ai is forecasting an "Open Fable" by end of year. VentureBeat's coverage led with the 1/6th cost angle. GLM-5.2 was also confirmed SOTA on PostTrainBench, beating both GPT-5.5 and Opus 4.8 on that specific evaluation.

The outlier note: GLM-5.2 is text-only. No vision support. In a world where VLMs (vision-language models) are becoming the default interface, that's a real gap — and it may explain why the Artificial Analysis score (51) still trails the closed frontier's multimodal offerings. For pure text and code, though, the consensus is clear.

What This Actually Means for Developers

The developer calculus has shifted. Not because GLM-5.2 beats the closed frontier — it doesn't, reliably. But because the gap is now small enough, and the cost delta large enough, that the decision matrix changes.

Use GLM-5.2 when:

Cost sensitivity matters more than squeezing the last 1-3% of capability
You need self-hosting for data sovereignty, compliance, or latency control
Your workload is code-heavy (SWE-Bench Pro, FrontierSWE scores are strong)
You want the insurance of MIT-licensed weights that can't be export-banned
You're running high-volume agentic workloads where $0.46/task vs $2+/task compounds

Stick with Opus 4.8 when:

You need the absolute ceiling on software engineering tasks
Broad expert knowledge (HLE, GPQA) matters for your use case
You rely on the Anthropic ecosystem (Claude Code, Artifacts, tool use)
Terminal-Bench performance (85% vs 81%) is the relevant benchmark

For teams already running open models through OpenRouter, GLM-5.2 slots in as the highest-capability option at a price point that makes batch processing and high-volume agentic loops economically viable. At $0.46 per task versus $2+ for the closed alternatives, a team running 10,000 agentic tasks per day saves roughly $15,000 daily — $450,000 per month. That's not a rounding error.

The Gemini 3.5 Flash "cheaper than frontier" claim we analyzed last month takes on a different complexion when the open-weights alternative offers frontier-adjacent quality at an even lower price point — with the option to self-host and eliminate API costs entirely.

The meta point: the question has shifted from "is there a credible open-weights alternative?" to "when does the closed-model premium stop being worth it?" That's the pricing story. And pricing stories are the ones that actually change enterprise buying decisions.

The Contrarian Read

Kevin Murphy's quiet observation deserves the last word: "Current LLMs are outrageously data inefficient (and hence compute inefficient) — this will be the next frontier."

The entire GLM-5.2 narrative — open weights at a fraction of the cost, post-training in two days, MIT license for anyone with the hardware — assumes the current paradigm continues. If data efficiency becomes the real differentiator, the advantage may not stay with whoever has the most GPU-hours. It may shift to whoever figures out how to do more with less data.

But that's a future bet. Today, the numbers are clear: GLM-5.2 scores within 1% of Opus 4.8 on FrontierSWE, costs a fifth as much, and ships with its entire post-training recipe published. The closed frontier still leads. The gap that justifies the premium is shrinking every quarter. Mistral couldn't close it from Europe. China is closing it from the open-weights side — and handing the recipe to anyone who wants to try.

That's not a capability story. It's a moat story. And for enterprise teams doing the math on their AI spend, it's the one that matters.

Originally published at ComputeLeap

Anthropic Walked Back the Agent SDK Credit Change

Max Quimby — Wed, 17 Jun 2026 03:54:17 +0000

On June 15, 2026 — the exact day the change was supposed to take effect — Anthropic paused the most controversial billing restructure in its history. The Agent SDK, claude -p, and third-party apps still draw from your regular subscription limits. Nothing changed.

📖 Read the full version with charts and embedded sources on AgentConn →

The reversal came after a month of sustained community pushback that included Jeremy Howard calling the policy "misleading", developers publicly announcing switches to OpenAI and DeepSeek, and Anthropic's own communications team getting Community-Noted on X within hours of the announcement.

If you build agents on Claude subscriptions — whether through the Agent SDK, claude -p in CI/CD, or third-party harnesses like OpenClaw — this is the single most important pricing development of 2026. Here is what happened, why Anthropic blinked, and what operators need to do next.

What Anthropic Tried to Do

On May 14, 2026, Anthropic emailed subscribers that effective June 15, all programmatic Claude usage would move off subscription pools into a separate monthly credit system metered at standard API rates:

Plan	Monthly Agent SDK Credit	Est. Tasks
Pro ($20/mo)	$20	~50
Max 5x ($100/mo)	$100	~250
Max 20x ($200/mo)	$200	~500
Enterprise Standard	$0	0
Enterprise Premium	$200	~500

Credits would be per-user, non-pooled, and non-renewable. When they ran out, requests would fail unless you enabled overflow billing at full API rates ($3/$15 per million input/output tokens for Sonnet 4.6).

The scope was sweeping: claude -p, GitHub Actions, Agent SDK calls, and any third-party app authenticating through your subscription — including OpenClaw, Conductor, and every harness that had built on Claude's subscription model.

The math: A community analysis documented the effective price increase at 12x–175x depending on workload. Heavy Agent SDK users were accessing $300–600 worth of API-equivalent compute on a $200 Max plan — a 15–30x subsidy Anthropic said subscriptions were never designed to sustain.

Why the Community Revolted

The announcement landed badly. Anthropic framed it as a "simplification" that would let developers "build on Claude without needing an API key." The community immediately recognized what it actually was.

Three criticisms dominated:

1. The "interactive" redefinition. Jeremy Howard flagged the core deception: Anthropic's announcement said "interactive use" was "unchanged," but the policy redefined "interactive" to mean "using an Anthropic front-end." If you used claude -p or the Agent SDK to do something interactively — like running a code review from your terminal — it now used credits, not your subscription. The unchanged heading was technically true and practically misleading.

2. The switching signals. Developers started publicly announcing exits. Aniket Panjwani calculated a 25x usage cut for his headless code review workflow and announced he would evaluate Gemini and DeepSeek. Kun Chen declared he was "increasingly bullish about OpenAI" after the change.

3. The Community Note. Anthropic's Lydia Hallie posted the announcement on X, framing the credit system as enabling developers to "build on Claude without needing an API key." The post was Community-Noted within hours — one of the fastest corrections on a major AI company's official communications in 2026.

Why Anthropic Blinked

The reversal was not just about angry tweets. The Decoder's analysis identified three structural pressures converging:

Competitive timing. On the same day as Anthropic's announcement, OpenAI offered new business customers two months of free Codex Pro — a $400 value — explicitly targeting developers considering switching. OpenAI is reportedly considering significant API price reductions. Implementing a billing change that pushes users toward competitors during a price war is strategically catastrophic.

IPO exposure. Anthropic has filed SEC paperwork seeking a public listing. Losing power users over billing changes immediately before going public damages valuation prospects. The optics of a community revolt during the IPO roadshow would be toxic.

The Fable 5 context. The U.S. government had just ordered Anthropic to disable Fable 5 and Mythos 5 access for non-U.S. citizens — a separate crisis that was already straining Anthropic's relationship with its user base. Stacking a billing change on top of an export-control crisis doubled the attrition risk.

What the Reversal Actually Says

Anthropic's statement was carefully worded: "We're working to update the plan to better support how users build with Claude subscriptions. Nothing changes for now."

Note: "for now." This is a pause, not a cancellation. The structural problem Anthropic tried to solve — agents consuming compute at rates subscriptions cannot sustain — has not gone away. A heavy Max 20x user running claude -p in production is still accessing 15–30x more API-equivalent compute than their $200 payment covers.

The New Stack's analysis frames the underlying economics clearly: every autonomous agent generates thousands of requests versus dozens for a human user. At Sonnet 4.6's API rates, a sustained claude -p workload can burn through $200 of API-equivalent compute in days, not months.

Anthropic will come back with a revised billing model. The question is whether the next version will be designed with operator input rather than announced as a fait accompli.

The Hacker News discussion of the reversal surfaced a telling detail: multiple commenters noted that Anthropic's original announcement had no public feedback period, no beta program, and no graduated rollout. The first time most operators learned about the credit split was from a billing email. For a company that positions itself as the safety-conscious, alignment-focused AI lab, the contrast between its technical transparency and its billing transparency was stark. Several developers pointed out that even AWS — not known for developer-friendly billing — provides 90-day deprecation notices for pricing changes this significant.

What Operators Should Do Now

The pause buys time, not permanence. If you build on Claude's subscription model, here is the playbook:

1. Audit your Agent SDK usage. If you do not know how much API-equivalent compute your agents consume, you cannot evaluate the next billing proposal. Run your June logs through Anthropic's token calculator. The agent observability patterns we covered earlier are exactly the tooling you need here.

2. Build with portability in mind. The harness abstraction layer is more important than ever. If your agent stack is Claude-specific, the next billing change will be a crisis. If it runs through an abstraction like Claude Code's harness pattern or one of the multi-harness orchestrators, it is a line-item.

3. Separate interactive and automated workloads. Even though the split was paused, the economic logic behind it was sound. Mixing human interactive use and autonomous agent compute on the same subscription creates unpredictable costs. Consider whether your production claude -p workloads should be on API keys regardless of subscription pricing.

4. Watch the next announcement. Anthropic said it will provide "advance notice" before implementing any revised version. Given the community's response to the first attempt, expect the next proposal to include a longer transition period, higher credit floors, or a graduated pricing model.

The Bigger Pattern

This is not just an Anthropic story. It is the first skirmish in a fight that every AI platform will face: subscription models were designed for human-speed usage, and agents break the economics.

OpenAI solved this by never including Codex CLI in ChatGPT subscriptions — it has always been a separate $200/month product. Google's Agent Development Kit runs on Vertex AI billing, not Gemini subscriptions. Anthropic was the outlier: the only major provider that let autonomous agents run on consumer subscription pools.

That outlier status is precisely what made the Claude ecosystem attractive to agent builders. The original Hacker News announcement thread — which hit the front page when the credit split was first announced — captured the sentiment: developers had built production workflows around the assumption that claude -p was part of the subscription, not a subsidized bonus that could be repriced at any time. The gap between "included in your plan" and "tolerated on your plan" turned out to be the entire business case for dozens of agent startups.

The walkback preserves that advantage for now. But the math has not changed. At some point, every AI provider will have to price for the reality that one human with an agent harness consumes more compute than ten humans typing prompts. The question is whether that pricing arrives as a surprise email or as a transparently communicated transition.

For operators building the next generation of agent-powered workflows, the lesson is clear: own your cost model. Do not let your production infrastructure depend on pricing that the provider itself considers unsustainable.

If you are evaluating how to structure your agent costs across providers, our DeepClaude vs Claude Code vs Codex Pro cost comparison breaks down the economics tier by tier. And for the broader context on why harness portability matters more than any single provider's pricing, the harness moat analysis lays out the structural argument.

The credit change is paused. The compute economics are not. Plan accordingly.

Originally published at AgentConn

Apple Paying Google $1B/Year to Run Siri on Gemini

Max Quimby — Wed, 17 Jun 2026 03:45:42 +0000

On January 12, 2026, Apple made the most consequential admission in consumer AI history: it chose Google's Gemini to power the next generation of Siri in a multi-year deal estimated at roughly $1 billion per year. The company that designed its own silicon to escape Intel's roadmap just handed its most personal product — the voice assistant that lives on 2 billion active devices — to its biggest rival.

📖 Read the full version with charts and embedded sources on ComputeLeap →

This is not a partnership in the conventional sense. It is a structural confession. Apple spent four years trying to build a frontier-class foundation model in-house, and it failed. The 150-billion-parameter cloud model Apple had running on its Private Cloud Compute infrastructure was not competitive. Google's custom 1.2-trillion-parameter Gemini model — eight times larger — was the only option that could deliver the assistant experience Apple had been promising since WWDC 2024.

At WWDC 2026 on June 8, Apple made it official. Siri was rebranded as "Siri AI", a ground-up rebuild running on Gemini technology and Nvidia's latest Blackwell GPUs. Tim Cook's farewell keynote framed it as the dawn of Apple's next era. The subtext was harder to spin: the most valuable company on Earth outsourced the intelligence layer of its flagship product.

The Deal: What $1 Billion Buys

The partnership grants Apple access to a custom 1.2-trillion-parameter Gemini model built specifically for Siri and Apple Intelligence. According to TechCrunch's reporting, Apple selected Google after evaluating competing proposals from OpenAI and Anthropic, concluding "that Google's technology provides the most capable foundation for Apple Foundation Models."

The financial terms tell their own story. Bloomberg's Mark Gurman estimated the deal at $1 billion annually, but Gene Munster at Deepwater Asset Management pegged the total value at $5 billion, arguing that maintaining two large models "wouldn't make a ton of sense for Apple." The deal is structured as a non-exclusive licensing agreement — Apple technically retains the right to integrate other providers — but as anyone following the Google Search default litigation knows, "non-exclusive" and "meaningfully contested" are very different things.

For context, Apple's existing Google Search default deal is worth approximately $20 billion annually to Apple. The Gemini arrangement may follow the same trajectory: a modest opening bid that balloons as the integration becomes load-bearing.

ℹ️ The deal gives Apple a custom 1.2T-parameter Gemini model — 8x larger than Apple's own 150B cloud model. Gene Munster estimates total value at $5B. Bloomberg puts the annual fee at ~$1B.

The Architecture: Three Layers of Siri

The rebuilt Siri operates on a three-layer architecture that reflects Apple's attempt to preserve its privacy guarantees while outsourcing the heaviest computation:

Layer 1 — On-Device. Simple tasks stay local, running on Apple's own compact models optimized for the Neural Engine in A-series and M-series chips. "Set a timer," "open Messages," and basic queries never leave the phone.

Layer 2 — Private Cloud Compute (PCC). Moderately complex requests route to Apple's own servers, where Apple controls the hardware, the software, and the encryption. This layer handles multi-step reasoning that exceeds on-device capacity but doesn't require Google's model.

Layer 3 — Google Cloud with Nvidia Blackwell B200. The hardest queries — the ones that require 1.2 trillion parameters of reasoning — route to Nvidia Blackwell B200 GPUs on Google Cloud. This is where the deal lives, and it is also where the privacy engineering gets creative.

Apple's Private Cloud Compute hardware could not run the 1.2T Gemini model at practical latency for the query volumes Siri requires. The solution: route those queries through Google's data centers while wrapping them in Nvidia's hardware-based confidential computing. Queries are anonymized, stripped of Apple ID linkage, and tokenized before reaching Google's infrastructure. Even Google's cloud operator cannot read the data in plaintext during processing.

An ACM conference paper presented in June 2026 independently validated Apple's three core PCC privacy claims. Apple's contract with Google also prevents Google from using Siri queries to train future Gemini models.

💡 Apple's three-layer design means your "Hey Siri, set a timer" never leaves your phone. Only the hardest queries — the ones that need 1.2 trillion parameters — touch Google infrastructure, wrapped in hardware-level encryption.

Winners and Losers

Google: The Biggest Win Since Android

For Google, this deal is a strategic masterwork. After years of watching ChatGPT dominate the AI narrative, Gemini just became the default intelligence layer for the world's most valuable device ecosystem. Bank of America analysts noted the deal reinforces "Gemini's position as a leading LLM for mobile devices."

The market agreed. News of the deal helped push Alphabet's market valuation above $4 trillion. Every Siri query that routes through Gemini is a query that does not go through ChatGPT — and every user who learns to rely on Gemini-powered Siri is a user who may choose Google's services elsewhere.

During Google's Q4 2025 earnings call, executives confirmed the partnership and projected that the context-aware Siri would debut later in 2026. For a company that spent 2024 playing defense against OpenAI's consumer momentum, this was the validation it needed.

OpenAI: The Distribution Deal That Got Away

The implications for OpenAI are severe. As Fortune bluntly stated, "OpenAI lost the most important distribution deal in AI." Apple's 2+ billion active devices represent the ultimate platform for scaling AI to mainstream users, and OpenAI no longer owns that pipeline.

ChatGPT is not gone from Apple's ecosystem — it remains available for "complicated, opt-in queries" — but it has been demoted from the default intelligence layer to an optional second opinion. This is the difference between being the engine and being the spare tire.

The timing compounds the damage. OpenAI's consumer growth rate had reportedly slowed, and the company's upcoming AI device (designed by Jony Ive) now faces a market where the dominant mobile platform's assistant is powered by its biggest competitor.

Apple: The Pragmatist's Play

The conventional reading is that Apple lost. Analyst Daniel Newman called 2026 a "make-or-break year" for Apple's AI strategy, and outsourcing to Google certainly looks like a concession.

But there is a contrarian case worth taking seriously: Apple does not need to own the model. It needs to own the context layer — the intimate, permissioned dataset generated by 2 billion active devices. No AI lab possesses this. Apple treats Gemini's reasoning as a licensed commodity. The integration, the on-device data, the privacy architecture — that is the moat.

The parallel to the Google Search deal is instructive. Apple has been "outsourcing" its browser's default search engine to Google for two decades, earning $20 billion a year for it. That deal did not make Apple weaker. It made Apple the tollbooth operator on the most valuable default in tech.

⚠️ The Google Search deal started small and grew to $20B/year. If the Gemini-Siri partnership follows the same trajectory, Apple may have just negotiated the most valuable AI distribution deal in history — as the buyer.

The Antitrust Shadow

The legal community noticed immediately. Vanderbilt antitrust professor Rebecca Haw Allensworth argued that the deal "essentially creates a second exclusive pipeline" raising the same structural concerns as the Google Search default arrangement.

The comparison is not academic. A federal judge ruled in 2024 that Google's search distribution agreements with Apple were anticompetitive — that "defaults matter more than formal exclusivity" and that "once entrenched, defaults are remarkably sticky."

The Gemini-Siri deal raises an identical structural question. Apple claims the arrangement is "expressly not exclusive," pointing to iOS 27's Extensions framework that theoretically allows alternative AI providers. But the antitrust analysis is damning: "Apple does not compete with Gemini. Apple neutralizes it by absorbing it." Gemini gets system-level integration while rivals face sandboxed, higher-friction access.

The affected parties extend beyond Google and Apple. AI startups, vertical AI companies, app developers, and content platforms all face growing foreclosure risks as Siri becomes the dominant intermediary between users and digital services.

No enforcement action has been filed yet. But the legal scholars writing about this deal are not using tentative language. They are drawing direct lines to Microsoft's browser monopolization case — and to the Google Search ruling that already found this exact structure anticompetitive.

What Siri AI Actually Does

The rebuilt assistant, confirmed at WWDC 2026, ships with three headline capabilities:

Cross-App Operation. Siri AI can chain actions across multiple apps in a single request. "Book a restaurant for Friday night, add it to my calendar, and text the group chat the details" is one prompt, not three.

On-Screen Awareness. Point Siri at what is on your screen and ask about it. It understands the context of the current app state — a departure from old Siri, which treated each query as context-free.

Personal Context Understanding. Siri AI draws on your on-device data — messages, emails, browsing history, app usage patterns — to personalize responses without sending that data to the cloud.

Apple also announced that SiriKit is being deprecated in favor of App Intents, signaling that every third-party developer needs to rebuild their Siri integration for the Gemini-powered architecture. The migration clock is ticking.

With iOS 26.4 expected to deliver these features to 1.5 billion daily users, this is one of the largest-scale AI deployments in history — running on a model Apple does not own, hosted on hardware Apple does not control.

The Bigger Picture

Apple's Gemini deal is not an isolated decision. It is the latest data point in a pattern that is reshaping the AI industry:

The model layer is commoditizing. When a $3 trillion company concludes it is cheaper to license a frontier model than build one, the economic signal is clear. Foundation models are becoming infrastructure — like cloud compute, like databases, like CDNs. The value is migrating to the integration layer above them.

Distribution is the new moat. OpenAI has the best consumer product in AI. It did not matter. Apple chose the model that came with the best infrastructure deal. In AI, as in every prior technology wave, whoever owns the distribution channel owns the margin.

Privacy is an architecture problem, not a marketing one. Apple's three-layer approach — with hardware-encrypted confidential computing on the hardest queries — is genuinely novel. It proves you can outsource intelligence without outsourcing trust, but only if you are willing to invest in the plumbing.

For developers, operators, and anyone building on top of AI models: this deal is a blueprint. The company that controls the interface, owns the user relationship, and manages the data layer will capture the value — regardless of whose model generates the tokens.

If you are comparing the three major AI assistants powering consumer devices today, our Claude vs ChatGPT vs Gemini comparison breaks down the capabilities head to head. And if you are curious about Apple's on-device AI ambitions — the Layer 1 that stays on your phone — our deep dive on the iPhone 17 Pro's 400B on-device LLM covers what Apple is building for the queries that never need Google at all.

The frontier lab that outsourced its brain may have made the smartest move in the AI race — not by building the best model, but by building the best tollbooth.

▶️ Apple Picks Gemini to Run AI-Powered Siri | Bloomberg Tech

▶️ BREAKING: WWDC 2026 - Can Gemini Save Apple?

Originally published at ComputeLeap

The Narration War: Who Defines the Fable 5 Freeze

Max Quimby — Tue, 16 Jun 2026 04:26:27 +0000

At 5:21 PM Eastern on June 12, Anthropic received a letter from the US government that ended the shortest product lifecycle in frontier AI history. Three days after launching Claude Fable 5 — and one day after CEO Dario Amodei published a policy essay explicitly asking the government to hold legal authority to block unsafe AI releases — the government used exactly that authority against him.

📖 Read the full version with charts and embedded sources on The Arc of Power →

The freeze itself is a fact. What it means is already being fought over by three competing narrators, each offering a version designed to set the precedent for the next time a frontier model gets pulled. This is the narration war — and the winner gets to define how the US government, AI labs, and the market interact for the next five years.

Three Narrators, One Freeze

Every major inflection in technology governance produces a fight over the canonical narrative. The Fable 5 freeze is no different — except it's happening in real time, with prediction markets scoring each narrator's credibility as they speak.

Narrator 1 — The Safety Hawks: The government acted on a legitimate national security concern. Anthropic was warned, refused to cooperate, and forced the government's hand. David Sacks, the AI czar, is the primary voice.

Narrator 2 — The Self-Fulfilling Prophecy: Anthropic's own doomsday rhetoric about AI risk invited the crackdown. When you spend years telling the government that your product is as dangerous as nuclear weapons, eventually someone takes you at your word. Nathan Lambert is the primary voice.

Narrator 3 — The Palace Coup: This wasn't about safety at all. It was about personality clashes between Sacks and Amodei, corporate sabotage by Amazon, and a White House that doesn't understand the technology. Axios is the primary outlet.

Narrator 1: The Safety Hawks

David Sacks' X thread on the freeze collected 32,000+ likes — the single most-engaged post on the event. His narrative: a "highly credible trusted partner" identified a jailbreak, the government asked Amodei to fix it, Amodei refused, the government acted reluctantly.

Fortune reported that Amazon CEO Andy Jassy had independently flagged vulnerabilities. Semafor added the intelligence angle: Chinese access concerns drove the urgency.

But the safety-hawk narrative requires believing the timeline is coincidental — that the government just happened to discover a jailbreak 72 hours after launch, and that none of this was influenced by the bitter personal relationship between Sacks and Amodei.

Narrator 2: The Self-Fulfilling Prophecy

Nathan Lambert's Interconnects post called this "the starting gun of a new era in AI governance" — but argued Anthropic fired the starting gun on itself. His key claim: Anthropic's fear-mongering "accelerated this moment by 6-12 months."

The timeline supports him. On June 10 — one day after launch — Amodei published "Policy on the AI Exponential," calling on the government to hold legal authority to block unsafe frontier AI. Two days later, the government did exactly that. Latent Space captured the irony: "Fable and Mythos officially too dangerous to release."

Narrator 3: The Palace Coup

Axios broke the reframing: "'They screwed us': Personality clashes sent Anthropic's models offline." The palace-coup narrative strips the safety frame and replaces it with interpersonal dynamics — the Sacks-Amodei conflict, Amazon as both investor and competitor reporting vulnerabilities to the White House, and a communication breakdown where the two sides "speak in different languages."

Simon Willison amplified the reframing, lending it credibility with the technical audience.

What the Markets Say

Polymarket's "best AI model end of June" sits at 91.5% for Anthropic. Despite the ban. The market says this is temporary.

"Trump orders federal review of AI model releases" trades at just 28%. The market doesn't expect a new regulatory regime.

The Polymarket data contradicts all three narrators:

Against safety hawks: a real emergency wouldn't price Anthropic at 91.5% in two weeks
Against self-fulfilling prophecy: a permanent governance shift would push federal review above 28%
Against palace coup: a purely personal conflict wouldn't produce Commerce Department crisis talks

The market's implicit narrative: this was a policy accident. The system self-corrects.

The 6-Day Timeline

June 9 — Launch. June 10 — Amodei publishes policy essay asking for government authority. June 11 — Amazon tests Mythos, Jassy alerts White House. June 12, 5:21 PM — Directive received, models go dark. June 14 — Sacks X thread (32K likes). June 15 — Axios reframes as personality clash. Commerce meeting.

Six days from launch to personality-clash reframe. That's the speed at which narratives compete in 2026.

Who Wins

The narration war isn't about who's right — it's about which frame sticks. The safety-hawk frame serves the government and Amazon. The self-fulfilling prophecy frame serves competing labs. The palace-coup frame serves the media and Anthropic's defenders.

The prediction markets cut through by pricing the outcome: Anthropic at 91.5%, federal review at 28%. The money says: aberration, not precedent. The narration war is loud, but the money is quiet.

The Fable 5 freeze will resolve. But the frame that wins determines whether the next frontier model launch happens with a safety review, a political blessing, or a corporate knife-fight behind the scenes.

In power analysis, the answer is usually: all three at once.

Originally published at The Arc of Power

Run Your Coding Agent on Local Weights: Operator Playbook

Max Quimby — Tue, 16 Jun 2026 04:07:17 +0000

Your frontier model just got pulled. On June 12, the US government issued an export control directive that forced Anthropic to disable Claude Fable 5 worldwide — with almost no notice. If your coding agent was wired to Fable 5, it went dark. Hugging Face CEO Clément Delangue's response summed up the mood: "Fable is banned. Long live local AI."

📖 Read the full version with charts and embedded sources on AgentConn →

The Hacker News thread that followed — "Has anyone replaced Claude/GPT with a local model for daily coding?" — collected 93 points in hours. The answers were surprisingly practical: developers reporting that Qwen 3.6 and Gemma 4 on RTX 3090s handle 80% of their daily coding. Not 100%. But enough to survive a frontier model going dark overnight.

This isn't a model review. It's an operator playbook: what hardware you need, which agent harnesses work with local weights, where tool calling breaks down, and how to build an 80/20 hybrid that makes your agent stack ban-proof.

The Hardware You Actually Need

The single most common question in every local-model thread: "How much VRAM?" The answer in 2026 is more nuanced than "buy a 4090," because Mixture of Experts (MoE) architectures have dramatically changed the math.

VRAM	Best Model Fit	What You Can Do	What Breaks
8 GB	Qwen 2.5 7B	Basic completions, simple edits	No agentic workflows, poor tool calling
16 GB	Qwen 3.6 35B-A3B (MoE)	Comfortable agentic coding, most daily tasks	Struggles with 5+ file refactors
24 GB	Qwen3-Coder-Next / Gemma 4 26B	Serious agentic work, SWE-bench 58.7%	Still 80/20 vs frontier on hard problems
48 GB+	Full-precision large models	Near-frontier for most tasks	Diminishing returns vs cloud cost

The sweet spot is 24 GB — matching the RTX 3090 ($489 used), RTX 4090, and RTX 5090 entry tier. At that budget, a developer spending $60–100/month on Claude API tokens recoups the GPU cost in 5–8 months.

💡 MoE (Mixture of Experts) models like Qwen 3.6 35B-A3B activate only a fraction of their parameters per token. This is why a "35B" model runs on 16 GB VRAM.

Which Models Run Agentic Coding on Consumer GPUs

Qwen 3.6 35B-A3B — The current community favorite. Released April 2026 with explicit agentic coding focus. Runs on 16 GB. The HN consensus: "it's the first local model that doesn't feel like a science experiment."

Qwen3-Coder-Next — The specialized coding variant. Scores 58.7% on SWE-bench Verified with 256K context, running on a single 24 GB GPU.

Gemma 4 26B-A4B — Google's MoE entry. Fast, low-VRAM, excellent for completions. Struggles in agentic scenarios — better as a copilot than an autonomous agent.

Pick Your Harness: PI, Aider, Cline

PI Agent (61K+ stars, MIT) — Terminal-native, ships with four core tools, works with any local Ollama model.

Aider — Most mature open-source terminal coding agent with architect/editor mode for the 80/20 pattern.

Cline — VS Code-native with Plan/Act mode, supports multiple LLM backends.

The Reliability Problem (and How Forge Solved It)

If each tool-calling step succeeds 90% of the time, a 5-step workflow has a 59% success rate. Forge, published as an ACM CAIS 2026 paper, wraps any self-hosted LLM with guardrails: an 8B model goes from 53% to 99% task completion.

⚠️ The compounding reliability problem: 90% per-step accuracy = 59% on 5 steps, 35% on 10 steps. Forge addresses this at the harness layer, not the model layer.

The 80/20 Hybrid

The operators getting the most value aren't going all-in on local. They route 80% of routine coding locally (completions, single-file edits, tests) and 20% to cloud (multi-file refactors, complex debugging). Monthly cloud spend drops from $80 to ~$16. The GPU pays for itself in 8 months.

The Operator's Checklist

Hardware: 24 GB VRAM (RTX 3090 used: ~$489)
Model runtime: Ollama or vLLM with Qwen 3.6 35B-A3B
Agent harness: PI Agent, Aider, or Cline
Reliability layer: Forge-style guardrails for models under 24B
Cloud fallback: One provider for the hard 20%
Routing logic: Single-file → local, multi-file → cloud

The harness is the moat, not the model. Your coding agent should run on whatever weights are available — local, cloud, or both.

Originally published at AgentConn

OpenRouter Fusion vs Claude Fable 5: 7x Slower, 4x the Cost

Max Quimby — Tue, 16 Jun 2026 03:33:31 +0000

OpenRouter just launched Fusion, a multi-model routing API that fans your prompt out to multiple LLMs simultaneously, synthesizes their responses through a judge model, and returns a single answer. The pitch: frontier-level intelligence at half the price of Claude Fable 5. The Hacker News reality check: 7× slower and 4× the cost of just calling a single top model directly.

📖 Read the full version with charts and embedded sources on ComputeLeap →

So which is it?

The timing is not a coincidence. With Anthropic's Fable 5 freeze still reverberating — the model pulled barely a week ago over export control concerns — operators are scrambling for a single-vendor-risk hedge. OpenRouter is selling exactly that: don't depend on one frontier model when you can blend several. But the economics of multi-model routing are more nuanced than the marketing suggests.

How Fusion Actually Works

Fusion operates in three sequential phases, documented in OpenRouter's plugin guide:

Panel Phase. Your prompt goes out to up to 8 models in parallel. The default Quality preset sends to Fable 5 + GPT-5.5; the Budget preset uses Gemini 3 Flash + Kimi K2.6 + DeepSeek V4 Pro.

Judge Phase. A designated judge model (Claude Opus by default) receives all panel responses and performs comparative analysis — consensus, contradictions, partial coverage, unique insights, and blind spots.

Synthesis Phase. Your primary model crafts the final response from the judge's analysis.

The critical detail: you pay for every underlying completion plus the judge call. A 3-model panel = roughly 4–5× the cost of a single completion.

⚠️ Fusion pricing is cumulative — Quality costs 3.2× what a single Opus 4.8 call costs. Budget is the cost-efficient option at 0.40× of solo Fable 5.

The DRACO Numbers

Configuration	DRACO Score	Cost per Prompt
Fusion Quality (Fable 5 + GPT-5.5)	69.0%	$0.29
Claude Fable 5 (solo)	65.3%	~$0.10
Fusion Budget (Flash + Kimi + DeepSeek)	64.7%	$0.04
GPT-5.5 (solo)	60.0%	~$0.06

Quality beats solo Fable 5 by 3.7 points. Budget comes within 0.6 points at 40% of the cost.

Caveats: Fable 5 completed only 93/100 tasks (content filters), DRACO is text-only English-only, and Fusion showed "no advantage for long-horizon tasks."

The HN Reality Check: 7× Slower, 4× the Cost

The HN thread (200 pts, 78 comments) tells a sobering story. Top comment from a dev who built a similar system: "Fusion was 7× slower and 4× the cost compared to calling Opus 4.7 directly."

Deeper concern: having one model judge another essentially asks "how closely does this resemble the answer you would have given me." Additional rounds = "cranking up the temperature" without better answers.

💡 HN consensus: multi-model judging works for verifiable answers but poorly for ambiguous domains.

Most interesting finding from a related thread: fusing identical models also boosts performance — suggesting gains come from test-time compute, not model diversity.

Budget vs Quality: Two Very Different Products

TokenMix's review breaks down the annual math at 10K prompts/month:

Quality Fusion: $34,800/yr (2.9× more than Fable 5 for 3.7 DRACO points)
Solo Fable 5: $12,000/yr
Budget Fusion: $4,800/yr (60% less, within 0.6 DRACO points)

Budget Fusion is a genuine cost play. Quality Fusion only pencils out for high-stakes domains (legal, compliance, medical).

When to Use (and Skip) Fusion

Use Quality when: output value >$1/task, need cross-model consensus, verifiable answers, 1-3s latency OK.

Use Budget when: high-volume batch, frontier-adjacent on a budget, vendor diversification matters.

Skip entirely for: real-time (<500ms), code completion, chat, long-horizon tasks.

💡 Heuristic: if a skilled human reviewer would consult three experts before answering, Fusion fits. If that's overkill, single-model wins.

The Verdict

Budget Fusion delivers on the "half the price" promise for batch workloads. Quality Fusion costs 3× more and only makes sense for high-value-per-task domains. Use it surgically, not as your default router.

Originally published at ComputeLeap

MDASH: How 100 Agents Beat One Frontier Model

Max Quimby — Mon, 15 Jun 2026 04:26:49 +0000

On May 12, Microsoft's Autonomous Code Security team published a benchmark result that reframed the AI security conversation: their system, MDASH, scored 88.45% on the CyberGym vulnerability benchmark — five points ahead of Anthropic's Mythos Preview (83.1%) and nearly seven ahead of OpenAI's GPT-5.5 (81.8%).

Read the full version with charts and embedded sources on AgentConn

The result was not surprising because MDASH used a better model. It was surprising because MDASH did not use one model at all. Microsoft stitched together publicly available models into a structured pipeline of more than 100 specialized agents — and that pipeline outperformed every single-model entry on the leaderboard.

For anyone building multi-agent systems, the implication is direct: when the task is complex enough, composition beats scale.

What MDASH Actually Is

MDASH stands for Multi-model Agentic Scanning Harness. Built by Microsoft's Autonomous Code Security (ACS) team, it orchestrates 100+ specialized AI agents across an ensemble of frontier and distilled models to autonomously discover, debate, validate, and prove exploitable vulnerabilities in codebases.

The key design decision: MDASH is model-agnostic. Frontier models serve as heavy reasoners. Distilled models handle high-volume debate and filtering. A separate SOTA model acts as an independent counterpoint. Models can be swapped and A/B tested without rewriting pipeline stages — when a better model arrives, you change a config file, not your architecture.

Key insight: As Microsoft's VP of Agentic Security Taesoo Kim put it: "The harness does the work, and the model is one input." If swapping a model requires rewriting your pipeline, you have built a model integration, not a system.

That framing is worth sitting with. In a world where the best model you depend on can vanish in 72 hours, the durable value sits in the orchestration layer — the system around the model — not in any single model itself.

Industry analyst Patrick Moorhead captured the significance of this shift:

The 5-Stage Pipeline

MDASH runs a five-stage pipeline where each stage is handled by a specialized cohort of agents. This is the architecture that beat Mythos:

Stage 1: Prepare

The system ingests the source target, builds language-aware indices, and maps the attack surface by analyzing past commits. This stage draws threat models and identifies high-value code paths before any scanning begins.

Stage 2: Scan

Specialized auditor agents run over candidate code paths. Each auditor emits candidate findings with hypotheses and evidence. They generate theories about what could be wrong — but they do not validate those theories. That is someone else's job.

Stage 3: Validate

A second cohort of agents — debaters — argue for and against each finding's reachability and exploitability. This is where MDASH diverges most sharply from single-model approaches. When an auditor flags something as suspect and the debater cannot refute it, that finding's posterior credibility goes up. The ensemble disagreement is the signal.

Stage 4: Dedupe

After validation, MDASH collapses semantically equivalent findings via patch-based grouping. Multiple auditors often flag the same underlying issue through different code paths — deduplication ensures the pipeline does not waste prover resources on redundant work.

Stage 5: Prove

Prover agents construct triggering inputs that confirm vulnerabilities exist dynamically. For C/C++ targets, they use AddressSanitizer (ASan) to confirm memory violations. This stage transforms theoretical findings into working proof-of-concept exploits.

The division of labor is the architectural insight. An auditor does not reason like a debater, which does not reason like a prover. Each pipeline stage has its own role, prompt regime, tools, and stop criteria. This specialization lets MDASH catch cross-file ownership bugs — where a memory-lifecycle violation only becomes visible by comparing patterns across several source files — that collapse into noise when a single model processes each function in isolation.

Why Composition Beat Scale

The CyberGym benchmark — developed by UC Berkeley researchers — measures how well AI systems can reproduce real-world vulnerabilities across 1,507 tasks drawn from 188 open-source software projects. Each task gives the system a vulnerability description and a codebase frozen at the pre-patch commit, and success requires producing a working proof-of-concept that triggers the flaw.

This is exactly the kind of task where single models hit a ceiling. The benchmark requires:

Cross-file reasoning — finding bugs that span multiple source files and ownership boundaries
Multi-step validation — confirming that a theoretical vulnerability is actually reachable and exploitable
Proof construction — building a working exploit, not just flagging a suspicious pattern

No single model excels at all three. Mythos is a powerful model wrapped in an agent framework — but it is still one model doing everything. MDASH distributes the cognitive load across specialists, each optimized for their specific stage.

The composition thesis is simple: when the task decomposes into distinct subtasks that require different reasoning patterns, a pipeline of specialists outperforms a single generalist — even if the generalist is individually more capable on any one subtask.

Microsoft's blog makes this explicit: "Discovery requires composition that no single prompt can achieve. The bugs found are not visible to a model handed a single function, but are visible to a system that can sequence cross-file pattern comparison, multi-step reachability analysis, debate between specialized agents, and end-to-end proof construction."

The emphasis on end-to-end is key — the system does not hand off between stages; it owns the full loop.

The Benchmark Caveat

Let's be honest about what we know and what we don't.

Benchmark caveat: CyberGym scores are self-reported. Microsoft, Anthropic, and OpenAI each ran their own systems against the benchmark and reported their own numbers. No independent third party has verified the submitted scores. Treat the exact percentages as directional, not definitive.

GeekWire's Todd Bishop flagged this critical detail. The benchmark code is public, but the verification process is not. That does not make the results meaningless — but it does mean the precise margin should be held loosely. The architectural argument (composition > scale for complex multi-step tasks) is supported by the results, but not conclusively proven by them.

There is also the question of improvement trajectory. By Build 2026 on June 2, MDASH's score had risen to 96.55% — a gain of approximately 10 percentage points in under three weeks. That jump likely reflects ongoing model-panel refinements (swapping in better models, tuning prompt regimes) rather than fundamental architecture changes. It also underscores the advantage of a model-agnostic system: the harness absorbs model improvements via config changes, not rewrites.

Real-World Results

The benchmark numbers matter less than what MDASH found in production. On its way to the May 2026 Patch Tuesday, MDASH discovered 16 previously unknown Windows vulnerabilities:

10 kernel-mode flaws including bugs in the TCP/IP stack and IKEv2 service
6 user-mode flaws across networking and authentication components
4 critical remote code execution vulnerabilities, including CVE-2026-33824 — a double-free in Windows' IKEEXT service reachable remotely over UDP port 500 by an unauthenticated attacker

Microsoft is telling customers to expect bigger Patch Tuesdays going forward as AI-powered vulnerability discovery accelerates the rate at which flaws are found and fixed.

Historical validation was also strong: MDASH achieved 96% recall on 28 MSRC cases spanning five years for clfs.sys, and 100% recall on 7 cases for tcpip.sys. In a StorageDrive private test, it found 21 out of 21 planted vulnerabilities with zero false positives.

The real-world CVE output, not just the benchmark score, is what makes this system credible.

What This Means for Builders

MDASH is a cybersecurity system. But the architectural pattern — multi-model orchestration beating single-model scale — generalizes beyond security.

The core lesson: When your task has multiple distinct phases that require different reasoning strategies, build a pipeline of specialized agents rather than asking one model to do everything. This is the same insight behind Anthropic's defending-code-reference-harness and the broader harness-as-moat thesis — the durable competitive advantage is in the system you build around models, not in which model you rent.

Practical patterns from MDASH that transfer to any multi-agent stack:

Separate discovery from validation. Auditors and debaters have different objectives and different failure modes. Combining them in one prompt dilutes both.
Use ensemble disagreement as signal. When multiple agents with different prompts and models disagree, that disagreement carries information. Build your pipeline to surface and use it, not suppress it.
Make your system model-agnostic. If swapping a model requires rewriting your pipeline, you have built a model integration, not a system. MDASH absorbs model upgrades via config changes — so should yours.
Deduplicate before expensive stages. Validation and proof-of-concept generation are the most compute-intensive stages. Removing redundant findings before they reach those stages is simple but high-leverage.
Prove, don't flag. The jump from "this looks suspicious" to "here is a working exploit" is the difference between a finding and actionable intelligence. Build your pipeline to go all the way.

For a deeper look at agent orchestration patterns and how different loop designs affect reliability, see our orchestration tools roundup. And for the other side of the coin — what happens when the frontier model your harness depends on gets banned — see the Mythos security agent review for context on the model MDASH just beat.

The question Microsoft is answering with MDASH is not "which model is best?" It is: "What can you build when you stop trying to find one model that does everything?"

The answer, apparently, is the top of the leaderboard.

Originally published at AgentConn