DEV Community: AI Explore

GLM-5.2: The Open-Weight AI Model Undercutting Frontier Pricing by 10x — Day 28/30

AI Explore — Fri, 18 Sep 2026 13:17:58 +0000

TL;DR — GLM-5.2 ships with a 1,048,576-token context window, weights on Hugging Face under zai-org/GLM-5.2, and pricing of $0.561/M input and $1.764/M output tokens. Probes showed it nailed a multi-step reasoning problem at 218.2 tokens/sec but stumbled badly on a coding task, taking 106.8 seconds to produce only 700 tokens of truncated, unfinished code. The value case is real, but it's uneven — know which workload you're pointing it at before you commit.

Every open-weight release this year has claimed to close the gap with frontier labs at a fraction of the price. GLM-5.2 is the latest to make that pitch, and on paper the numbers back it up: a context window over a million tokens, weights published on Hugging Face, and completion pricing under two dollars per million tokens. The question that actually matters, though, isn't whether the spec sheet looks competitive — it's whether the model behaves consistently enough to build a real workload on top of it. The probes say: sometimes, and you need to know which times.

What's actually in the box

GLM-5.2 (zai-org/GLM-5.2 on Hugging Face) ships with a context length of 1,048,576 tokens — the same neighborhood as the handful of other 1M-context open-weight models that have shown up this month. Pricing sits at $0.561 per million input tokens and $1.764 per million output tokens. That completion price is the number worth sitting with: it's roughly a tenth of what a top-tier closed frontier model typically runs, which is exactly the value proposition the "after the summit" price war has been selling all year. The weights being open means you can also self-host it if the economics of your volume justify the hardware, rather than being locked into someone else's meter.

Three probes, two different models

Live probing is where marketing numbers meet reality, and GLM-5.2 produced a split result. On a reasoning task — a two-pump tank-filling word problem — it was fast and correct: 218.2 tokens/second, a full four-step derivation, and the right final answer (20 more minutes, after correctly computing the net fill rate of 30 L/min and the 1,800-liter remainder). That's a clean, well-organized answer with no wasted tokens.

The structured-extraction probe asked for strict JSON with three keys, "ONLY the JSON object," from an invoice snippet. GLM-5.2 got the values right — vendor, date, and total all correctly pulled — but wrapped the output in a markdown code fence instead of returning a bare JSON object as instructed. Minor, but if your pipeline parses raw model output without stripping fences, that's the kind of thing that breaks a production extraction job at 2 a.m. It ran at a respectable 95.7 tokens/second in 3.9 seconds.

The coding probe is the one that should give anyone pause. Asked for a straightforward interval-merging function plus a one-sentence complexity note, GLM-5.2 took 106.8 seconds and produced just 700 completion tokens at 6.6 tokens/second — and the output cuts off mid-function, before the merge loop even finishes, with no complexity statement at all. That's not a stylistic quibble; it's an incomplete deliverable. Whether that's a one-off probe hiccup or a real weak spot in this model's coding path isn't something a single transcript can settle, but it's a data point you shouldn't ignore if code generation is your primary use case.

Who should actually build on this

Start with the workloads where the reasoning probe's behavior is representative: multi-step numeric reasoning, structured business logic, policy-style Q&A, and long-document analysis that leans on the 1M-token window rather than tight code correctness. Teams doing contract review, financial reconciliation walkthroughs, or long-context summarization where the model needs to hold an entire document set in memory and reason over it are squarely in the zone where $0.561/$1.764 per million tokens beats renting frontier API access by a wide margin, especially at volume.

Structured-output pipelines — invoice parsing, log-to-JSON conversion, form extraction — are also a reasonable fit, but only if your downstream parser is defensive enough to strip markdown fences and validate schema rather than trusting a literal "return only JSON" instruction to be followed to the letter. That's cheap insurance and worth building regardless of which model you use.

Coding-agent workloads are the case I'd flag for caution based on what showed up here. A 106-second wait for an unfinished function is not something you want inside an autonomous agent loop that fires dozens of these calls per task; it either times out, produces broken diffs, or silently corrupts a multi-step plan. If your primary job is code generation or agentic tool use, this single transcript is a reason to run your own harness against your own repo before committing spend, not a reason to write the model off outright — one bad probe run doesn't prove a systemic issue, but it does mean the burden of proof is on you to verify before production.

The honest recommendation: treat GLM-5.2 as strong for reasoning-heavy, long-context, structured-data work at a price point that genuinely undercuts frontier alternatives, and treat its coding path as unproven until you've tested it against your own codebase and latency budget.

What the price war actually changes

The interesting story here isn't GLM-5.2 in isolation — it's what a $1.764/M completion price does to everyone else's roadmap. When an open-weight model with a 1M-token context and Hugging Face weights lands at roughly a tenth of frontier completion pricing, every other vendor's cost curve gets renegotiated whether they like it or not. That pressure is good for builders: it means the "near-frontier at commodity price" tier is no longer a promise, it's a shelf full of options, and the real differentiator shifts from "can it do the task" to "does it do the task reliably enough, fast enough, for what I'm paying." GLM-5.2 clears the first bar convincingly and the second bar unevenly — which, at this price, might still be a trade worth making for the right workload.

Tomorrow closes out the series with a model built on a different premise entirely — Olmo 3 32B Think, and the case for calling something "fully open" rather than just open-weight.

The numbers (measured, not quoted)

I ran three quick probes against GLM-5.2 via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	106.8s	700	6.6	completed
Reasoning	2s	447	218.2	completed
Structured output	3.9s	370	95.7	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 1,048,576 tokens · hosted pricing $0.561/M input · $1.764/M output · weights: zai-org/GLM-5.2 on Hugging Face

Credits — where it's due

Zhipu AI (Z.ai) — for training GLM-5.2 and releasing the weights openly: zai-org/GLM-5.2. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Qwen3.8-2.4T: The 2.4 Trillion Parameter Open-Weight MoE AI Model — Day 27/30

AI Explore — Thu, 17 Sep 2026 14:16:34 +0000

TL;DR — Qwen3.8-2.4T-A95B is a 2.4 trillion parameter mixture-of-experts model with 95B active parameters per token and a 1,048,576-token context window. Probes showed it nailing a coding task and a math word problem cleanly, but returning nothing at all on a simple JSON extraction after burning 700 completion tokens and 19.3 seconds. It's a model almost nobody self-hosts — this is a rent-it-by-the-token release, priced at $2/$6 per million tokens.

At some point in a 30-day tour of open-weight models, you stop being surprised by big numbers. Then you see 2.4T in a model name and you're surprised again. Qwen3.8-2.4T-A95B is, per its own Hugging Face ID, exactly what it says: 2.4 trillion total parameters, with 95 billion active per token. That's not a typo-friendly round number — it's the largest open-weight release this series has covered, and it forces a question that matters more than the parameter count itself: who is this actually for?

The shape of the thing

Mixture-of-experts math means the 2.4T figure and the 95B figure aren't in competition — they're both true at once. Every token routes through a 95-billion-parameter slice of the network, but which slice varies token to token, which means the full 2.4T of weights has to be reachable, not just the active fraction. That's the part people miss about MoE at this scale: you don't get to load a "95B model." You need the whole 2.4T resident somewhere, active experts change every forward pass.

The context window is the other headline number: 1,048,576 tokens, exactly 1Mi, per the metadata. Combined with the compute needed to reach that context economically, this reads like a model built for institutional-scale batch work — full repositories, full case files, full transcript archives — not chat turns. Pricing on the hosted API is $2 per million prompt tokens and $6 per million completion tokens, meaning a single maxed-out 1M-token context request costs a bit over $2 just for the input, before you've gotten a word back.

What the probes actually showed

Two of three probes looked genuinely solid. The coding task — write merge_intervals and state its complexity — produced correct, idiomatic code: sort by start, walk once, merge on overlap, and it correctly named the complexity as O(n log n) for the sort, not a lazy O(n) guess. That took 5.1 seconds and 222 completion tokens, about 43.4 tokens/second.

The reasoning probe — a two-pump tank-filling problem — was answered with a clean four-step derivation: net fill rate of 30 L/min for the first 20 minutes, 600 liters added, 1,800 liters remaining, then 20 more minutes at pump A's 90 L/min alone. The arithmetic checks out at every step, and the final answer (20 minutes) is correct. This one ran faster and cheaper in relative terms — 3.8 seconds, 235 tokens, 61.8 tokens/second.

The third probe is the interesting one. A simple instruction — extract vendor, date, and total from an invoice into strict JSON, return only the JSON object — took 19.3 seconds, generated 700 completion tokens, and returned an empty string. Not malformed JSON, not a garbled attempt — nothing visible in the output at all. Whatever those 700 tokens were doing, none of them surfaced as the requested object. For a model this large, on a task this trivial, that's a real miss worth flagging plainly rather than smoothing over. Scale bought correctness on the harder-looking reasoning task and lost it entirely on the "just format this" task — which is a useful reminder that parameter count and task reliability aren't the same axis.

Who can actually run this

This is the section that matters more than any benchmark claim: almost nobody self-hosts a 2.4T-parameter model. Even at aggressive 4-bit quantization, 2.4 trillion parameters lands somewhere north of a terabyte of weights that all need to be reachable for routing, before you add KV cache for a context window that stretches past a million tokens. That rules out a single workstation, a single 8-GPU node, and most private clusters. What's left is exactly what the API pricing implies: this is a rent-it model, served by whoever operates the inference cluster, consumed over an API the same way you'd consume a closed frontier model from a proprietary lab.

So the honest use cases skew toward organizations that have already accepted API-based inference as their operating model and want an open-weight option in that mix — for reasons like avoiding vendor lock-in on model choice, auditing weights for compliance reasons, or fine-tuning downstream on infrastructure they control even if base inference runs elsewhere. A legal team processing thousand-page discovery sets in one context window, a research group needing whole-codebase reasoning without chunking, or an enterprise wanting a second frontier-class model to benchmark against a closed provider's output — those are workloads where the 1M context and 95B active compute earn their price. Casual chatbots, latency-sensitive customer support, or anything running on a laptop are the wrong jobs for this model; a distilled or Nano-class open model wins on cost and latency for those every time.

The other honest use case is signaling: a lab willing to open-weight a 2.4T model is making a statement about where the ceiling on "open" now sits, separate from whether most teams should reach for it. It's the model you point to when someone claims open weights can't compete with closed frontier scale — not necessarily the model you put in production this quarter.

What it says about the open-vs-closed line

Vendor framing around trillion-plus-parameter open releases tends to lean on raw scale as the headline, and Qwen3.8-2.4T's spec sheet — 2.4T total, 95B active, over a million tokens of context — is genuinely at the frontier tier by the numbers alone. But the empty-output result on the JSON probe is a useful counterweight to that framing: scale is necessary for some classes of problem and irrelevant to others, and "biggest open model released this season" doesn't automatically mean "most reliable model for a given task." The coding and reasoning probes suggest real capability sits behind the parameter count. The structured-output miss suggests that capability doesn't automatically generalize to every task shape, even a trivial one. Judge the number on its own terms — it's a serious release — and judge the behavior separately, because they don't always agree.

Tomorrow closes out the season with GLM-5.2 — worth watching to see whether the newest closed-adjacent open release handles the boring tasks as well as the hard ones.

The numbers (measured, not quoted)

I ran three quick probes against Qwen3.8-2.4T (A95B MoE) via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	5.1s	222	43.4	completed
Reasoning	3.8s	235	61.8	completed
Structured output	19.3s	700	36.2	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 1,048,576 tokens · hosted pricing $2/M input · $6/M output · weights: Qwen/Qwen3.8-2.4T-A95B on Hugging Face

Credits — where it's due

the Qwen team at Alibaba Cloud — for training Qwen3.8-2.4T (A95B MoE) and releasing the weights openly: Qwen/Qwen3.8-2.4T-A95B. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Anthropic's AI Protein Design Run, Number by Number: 354 Binders, 27%, and What It Doesn't Prove

AI Explore — Wed, 16 Sep 2026 22:05:03 +0000

On August 18, 2026, Anthropic published a paper claiming something that is easy to misread in either direction. It did not cure anything. It did not design a drug. What it did was hand a language model the entire job of a protein binder design campaign — target research, epitope choice, tool installation, compute budgeting, ranking — and then synthesise every single design the model returned, without editing any of them, and measure what stuck.

The paper is Autonomous de novo protein binder design with Claude, authored by Amir Shanehsazzadeh at Anthropic. Two Claude models ran the campaigns: Claude Opus 4.8 and Mythos Preview, the invitation-only model that has since been succeeded by Claude Mythos 5. Two independent contract research organisations did the wet-lab work without knowing which model, campaign, or rank any sequence came from.

Here is every number that matters, including the ones that went badly.

What was actually automated

A binder design campaign normally eats weeks of a specialist's time per target. You pick which region of the target to model, decide which surface to bind, install and configure a fast-moving stack of diffusion and structure-prediction models, filter candidates, decide whether a redesign round is worth the GPU spend, and finally choose which handful to send for synthesis.

Anthropic wrote that working knowledge into a single protocol prompt of roughly 16,000 words and handed it to Claude as a system prompt. Critically, the prompt names no epitope, no scaffold, no target construct, and no sequence for any target. Humans did four things: chose the 16 targets and passed each one to Claude as nothing more than a name, a UniProt accession, an organism and an oligomeric state; supplied a cloud GPU account with a fixed budget and time limit; placed the synthesis orders; and interpreted the binding data at the end.

Everything in between — which region of the protein to attack, which of ten generators to run, how stringently to filter, which 30 designs to deliver and in what order — was Claude's. Campaigns ran 24 to 48 hours unattended. The paper notes that after infrastructure outages, "only short, non-technical instructions were needed to resume."

The headline: 354 of 1,320 designs bound

Sixteen targets went in. One of them — the mature GDF-8 dimer — aggregated under assay conditions and gave uninterpretable readings at both CROs, so its 120 designs are excluded from every count. That leaves 1,320 designs across 15 targets.

Claude produced confirmed binders against 14 of those 15 targets. In total 354 designs bound, a pooled hit rate of 26.8%. Anthropic puts the typical rate for a published campaign at 10–15%.

Hit rate: what bound, by campaign
Share of delivered designs that showed concentration-dependent binding at one or both CROs
Typical published binder campaign (Anthropic's stated baseline)

10–15%
Claude Opus 4.8 — all 13 targets at once, 48 h (88 of 390)

22.6%
Mythos Preview — all 13 targets at once, 48 h (104 of 390)

26.7%
All campaigns pooled (354 of 1,320)

26.8%
Mythos Preview — one target at a time, 24 h (158 of 450)

35.1%
Source: Shanehsazzadeh, Autonomous de novo protein binder design with Claude, 18 Aug 2026, Table 1 and text.

Two campaign formats ran. In the multi-target format, each model designed against 13 targets simultaneously inside one 48-hour session. In the single-target format, Mythos Preview got a dedicated 24-hour session per target — with roughly 2.8× the compute per target, which the paper is careful to say means focus and budget cannot be separated as explanations for the higher rate.

Claude's own ranking predicted what would bind

This is the result I find hardest to dismiss, and it gets less attention than the headline. Each campaign delivered 30 designs in rank order — Claude's own judgement of which were most likely to work, made before anything was synthesised.

If that ranking were noise, the hit rate would be flat across the list. It is not. The single design Claude ranked first in each campaign bound 49% of the time. Across the top five it was 44%, across the top ten 39%, and across all 30 it was 28%.

Claude's own ranking carried real signal
Hit rate by the rank Claude assigned each design before anything was synthesised
Rank 1 only — the single design Claude put first per campaign

49%
Top 5 ranks

44%
Top 10 ranks

39%
All 30 delivered designs

28%
A ranking that meant nothing would sit flat at 28% across all four rows.

In other words, if you had only ordered the one design Claude put at the top of each list, roughly half of them would have worked. That is a model that knows something about its own output.

RBX1: the only clean comparison against humans

Most of the paper has no human control group, and Anthropic says so plainly. There is one exception, and it is the most quotable result in the paper.

RBX1, a subunit of an E3 ubiquitin ligase, was recently the subject of an open design competition. Across 245 de novo designs submitted by human entrants, 9 bound — a 3.7% hit rate. Claude's three campaigns against the same target produced 28 binders from 90 designs.

RBX1: the one head-to-head against humans
Same target, same assay plate — an open design competition versus Claude's campaigns
Open competition entrants (9 of 245 designs bound)

3.7%
Claude, across three campaigns (28 of 90 designs bound)

31.1%
Claude's tightest RBX1 binder: KD 3.9 nM. The competition's winner, re-run on the same plate: 45 nM.

Affinity told the same story. Claude's tightest RBX1 binder came in at a dissociation constant of 3.9 nM. The competition's winning entry, re-synthesised and measured on the same plate under the same conditions, was 45 nM — roughly ten times weaker.

One caveat worth stating: the competition entrants were working under their own constraints and design budgets, not a matched protocol. It is a real comparison, not a controlled one.

How tight were the binders?

"Bound" is a low bar on its own. The useful question is how tightly. Of the 354 binders, 142 came in under 100 nM, 78 under 10 nM — the rough threshold at which a binder starts being useful as a reagent — and 38 under 1 nM. Six TREM2 binders hit the assay's detection floor and are reported only as below 100 pM, meaning the instrument could not tell how tight they actually were.

How tight were the binders?
Equilibrium dissociation constant (KD) for the 354 designs that bound — lower is tighter
Bound at all (354 designs)

354
KD below 100 nM

142
KD below 10 nM — the usual bar for a strong binder

78
KD below 1 nM

38
Six TREM2 binders sat at the assay's detection floor and are reported only as below 100 pM.

Anthropic reports high-affinity binders (KD under 10 nM) against at least six targets, and designs matching or beating the best previously reported affinity against at least four. There is also a result nobody optimised for: cross-species reactivity was only a secondary objective in the prompt, yet 130 of the 233 binders tested against the mouse version of their target bound that too — which is precisely the property you need for a molecule to be testable in animals.

Where it failed, and why that matters more

Three targets went badly, and the failures are more informative than the successes.

Against MBP (maltose-binding protein), all 90 designs failed. Against 15-PGDH, 1 of 30 bound. Against TNFα — a compact homotrimer that is already the target of five approved biologics — 12 of 150 designs bound, an 8% rate, and every one of them came from Opus 4.8. Both Mythos Preview campaigns against TNFα returned 0 for 60.

Where the campaigns failed
Designs that bound, out of designs delivered, on the targets that went badly
MBP — maltose-binding protein

0 of 90
15-PGDH

1 of 30
TNFα — every binder came from Opus 4.8; Mythos Preview got 0 of 60

12 of 150
Pooled average, all 15 targets

354 of 1,320
Confidence scores missed these — MBP and BBF-14 designs scored much like the ones that worked.

The uncomfortable part is the confidence scoring. Claude's in-silico predictions for the MBP and BBF-14 designs scored about as well as its predictions for targets that worked. The system had no idea it was failing. In a field where the whole promise is "filter in silico, synthesise only the winners," a confidence score that fails silently on hard targets is the thing that will cost people money.

The stack underneath, and what it cost

Everything Claude used is open source, which is the part of this paper with the longest tail. Ten structure generators contributed designs that were ordered: RFdiffusion (118 designs) and RFdiffusion3 (267), PXDesign (358 — the single largest contributor), Genie 3 (185), FreeBindCraft (135), BoltzGen (134), Proteina-Complexa (100), FoldCraft (14), BoltzDesign1 (2) and Protein Hunter (2).

Sequences came overwhelmingly from SolubleMPNN, the soluble variant of ProteinMPNN — 1,133 of the tested designs — with 111 from SolubleCaliby and 21 from stock ProteinMPNN. Ranking used an ensemble of ESMFold2, ESMFold2-Fast and Protenix v2. Notably, AlphaFold 3's weights, Rosetta/PyRosetta and ESM3 were all excluded on licensing grounds, and the campaigns worked anyway.

Compute was not trivial but not exotic: about 12,500 NVIDIA H100-hours for a 48-hour multi-target campaign, and about 2,500 H100-hours per target in the 24-hour single-target format, run on rented cloud GPUs.

How much of this is the model, and how much is the prompt?

Here is the honest tension in the paper. The 16,000-word protocol prompt is itself a substantial piece of expert work — and only about a third of it is science.

Two thirds of the 16,000-word prompt is not science
Anatomy of the protocol prompt every campaign received as its system prompt
Science and tooling — target dossiers, epitope choice, design strategy, filters, ranking

34.2%
Orchestration and verification — sub-agent delegation, clock discipline, ranking rules

34.7%
Operations — compute budget, pacing governor, reporting

31.1%
The expertise that made the campaigns work is partly in the model and partly written into this prompt by hand.

Science and tooling — target dossiers, epitope selection, design strategy, filters, the ranking score — is 34.2% of the prompt. Orchestration and verification is 34.7%, and operations (compute budget, a pacing governor, reporting) is another 31.1%. Two thirds of what makes these campaigns work is teaching the model to run a 48-hour job without falling over: delegate to sub-agents, watch the clock, verify its own outputs, don't burn the budget in hour six.

That cuts both ways. A skeptic can say the human expertise never left — it just moved into the prompt. That is fair, and the paper concedes it. But the same frozen prompt served all 16 targets without modification, which is the claim that actually matters: the expertise was written down once and then applied to targets it had never seen.

What this is not

The paper's own limitations section is unusually direct, and it deserves to be read before the headlines.

The evidence is binding, not structure and not function. Not a single design was structurally resolved. Every pose in the paper is a prediction. No design was tested for biological activity — a molecule that sticks to TNFα is not a molecule that does anything useful to TNFα.

The designs are not fully independent. Sequence variants of one backbone were each counted as a design. Counting only the best-ranked sequence from each of the 809 generated backbones, 200 bound — a 24.7% hit rate rather than 26.8%. The paper reports this itself and notes the main comparisons hold.

Each configuration ran exactly once. Model, campaign format and run-to-run chance are confounded. This is why Anthropic describes campaigns rather than ranking models — you cannot conclude from this data that Mythos Preview is better than Opus 4.8 at protein design, only that these particular runs went this way.

There was no parallel human campaign as a control, and most of the chosen targets are extensively characterised in the literature — meaning the model had plenty to work from.

Outside critics have pushed harder. Martin Shkreli, the convicted former pharmaceutical executive, called the work "not impressive" on X, arguing the affinities are unremarkable for this class of molecule and pointing out that none of the targets are intracellular — if you need something to bind a protein on the outside of a cell, a monoclonal antibody already does that job. It is a real objection: extracellular targets are the easier half of the problem, and the hard, undruggable-by-antibody targets sit inside the cell.

Anthropic's own framing is the one to keep: protein binders are not drugs. A high-affinity binder is the first step of a process that kills most candidates much later.

Why it still matters

Strip out the hype in both directions and one claim survives: a general-purpose language model, given a written protocol and a GPU budget, ran a multi-day computational biology campaign end to end and produced molecules that worked in someone else's lab, at a rate above the published norm, on targets it had never seen.

The tools it used are free. The prompts, all 1,440 design models, the per-design provenance and both CROs' binding data are released on Hugging Face under CC BY 4.0, with the scripts under MIT. Any lab with targets of interest and no computational protein design expertise can, in principle, rerun the protocol as it stands — or, more usefully, try to beat it.

That is the actual news. Not that AI designed a protein — that has happened before — but that the expensive, judgement-heavy orchestration around protein design turned out to be writable down, and that the write-up is public.

Credits and sources

The research is Anthropic's, and the paper is the work of Amir Shanehsazzadeh and the Claude Science team. The experimental validation — the part that makes this more than a simulation — was done independently by Adaptyv Bio (Lausanne) by surface plasmon resonance across five target concentrations, and by Twist Bioscience in a separate format; neither saw the other's data, the design models, or which model or rank produced a sequence. Adaptyv Bio returned usable results for 1,296 of the 1,320 designs.

The design stack belongs to the open-source structural biology community — the RFdiffusion, ProteinMPNN, BindCraft, Genie, Boltz, Protenix and ESMFold authors, among others. None of this campaign happens without a decade of their work being freely available.

Paper: Autonomous de novo protein binder design with Claude (Anthropic, August 18, 2026)
Anthropic write-up: How Claude is accelerating protein design and analytical chemistry
Data and prompts: Anthropic/claude-protein-binder-design on Hugging Face
Critical coverage: The Decoder on the methodology and its limits

All figures in this article were built from the numbers in the paper. Where the paper and the press release differ in rounding, the paper wins.

Kimi K3: The Trillion-Parameter Open-Weight AI Model You Rent — Day 26/30

AI Explore — Tue, 15 Sep 2026 13:17:08 +0000

TL;DR — Kimi K3 ships with a 1,048,576-token context window and pricing of $2.648/$13.283 per million tokens, and its weights are openly downloadable as moonshotai/Kimi-K3. Probe results show correct code, correct math, and valid JSON extraction, at throughput ranging from 13.7 to 36.9 tokens/sec depending on task. This is the class of open model where 'open-weight' and 'self-hostable' have quietly split apart.

Somewhere along the way, "open-weight" stopped meaning "something you can run." Kimi K3 is a good place to notice that split. The weights sit on Hugging Face under moonshotai/Kimi-K3, free for anyone to download. The context window is 1,048,576 tokens. And unless your organization already operates a GPU fleet built for models at this scale, the only sane way to touch it is through an API meter ticking at $2.648 per million input tokens and $13.283 per million output tokens.

What K3 is built for

The metadata is blunt about the intent: a full 1,048,576-token context and pricing structured for high-value completions rather than casual chat. That combination — huge context, premium output pricing — points at agentic workloads: long tool-use loops, multi-step research, code-repository-scale reasoning, where the model reads a lot and writes selectively. It's not priced or shaped like a model meant to answer one-line questions all day.

Moonshot AI positions this as part of a heavyweight, agent-oriented model family — that framing is a vendor claim, not a measured benchmark, but it's consistent with what the context window and pricing imply about the target workload.

What the probes actually showed

I ran three small, verifiable tasks against it rather than trusting a spec sheet.

The code probe asked for an interval-merging function. The output sorted intervals, merged overlaps with a single pass, and correctly stated O(n log n) complexity dominated by the sort — that's the right answer, and the example trace it included checks out. It took 30.7 seconds to produce 619 completion tokens, which works out to 20.1 tokens/sec — not fast, but the code was clean and idiomatic, not padded filler.

The reasoning probe was a two-phase tank-filling problem. It correctly computed the net fill rate during the overlap phase (30 L/min, 600 L after 20 minutes), then correctly solved for the remaining 1,800 L at 90 L/min for a final answer of 20 more minutes. The arithmetic is right at every step, and it showed its work rather than just asserting a number. This one ran at 13.7 tokens/sec — the slowest of the three, despite being the shortest output, which suggests reasoning-heavy generation costs more per token here than pattern-completion tasks do.

The structured-output probe asked for strict JSON extraction from an invoice string. It returned exactly {"vendor":"Meridian Office Supply","date":"03 March 2026","total":445.50} — valid JSON, correct field mapping, and it correctly picked the amount due as "total" rather than the subtotal, which is the trap most extraction prompts fail on. This ran fastest at 36.9 tokens/sec.

Across all three, the model got the substance right. The spread in throughput — 13.7 to 36.9 tokens/sec across tasks — is worth remembering: latency here isn't a single number, it's task-dependent, and reasoning-style generation on this model runs meaningfully slower than pattern extraction.

Where this actually earns its keep

This is the section that matters more than any spec: who should reach for a model like this, and for what.

Whole-repository code agents. A 1,048,576-token context can hold an entire mid-sized codebase plus its test suite plus a design doc, without chunking or retrieval tricks. If your agent needs to reason across files rather than snippets, this is the class of model that makes that tractable in one pass.
Long-document contract or compliance review. Feeding an entire contract set, a full regulatory filing, or a multi-hundred-page technical spec into one context window avoids the lossy summarization-of-summaries problem that chunked RAG pipelines run into.
Multi-step agentic tool use. The pricing and context shape suggest a model meant to sit in a loop — read state, call a tool, read the result, decide the next step, repeat — over long sessions without losing track of earlier context. The correct multi-step math in the reasoning probe is a decent proxy for this kind of chained logic holding up.
Structured extraction at the far end of a long pipeline. The clean JSON output, with the correct field picked out of ambiguous invoice text, is exactly the kind of task you'd chain after a long-context retrieval step — pull the needle out of the haystack, then hand back a typed object.

What it's not a good fit for: quick, cheap, high-volume chat traffic. At $13.283 per million completion tokens, a chatty back-and-forth app would burn budget fast, and the 13.7–20.1 tokens/sec range on substantive tasks means users waiting on a live response will feel it. This is a model for jobs where correctness and context depth matter more than latency and unit cost.

What self-hosting this would actually take

Here's the part the "open-weight" label glosses over. A model at this scale needs a serving stack most teams don't have lying around: multiple high-memory GPU nodes running tensor and pipeline parallelism just to hold the weights in memory, let alone serve concurrent requests with a million-token context — KV cache alone at that context length is enormous per active session. That's before you account for the engineering time to get a distributed inference stack tuned, monitored, and kept online.

Run the arithmetic on a single full-context call: 1,048,576 input tokens at $2.648/M is roughly $2.78 just to fill the window once, before a single output token is generated. That's the API's price for convenience. The self-hosted price is a capital outlay for GPU capacity that sits mostly idle unless your usage is sustained and heavy enough to amortize it — which, for most teams, it isn't. Renting via API isn't the fallback option here; for the vast majority of use cases, it's the only economically sane one.

Why "downloadable" still matters

So why does it matter that moonshotai/Kimi-K3 sits openly on Hugging Face if almost nobody will actually pull those weights and run them? Because open weights aren't only about who runs the model today — they're about who's allowed to inspect it, fine-tune it, or run it air-gapped tomorrow if they have the hardware and the reason. A government lab, a well-funded research group, or a company with existing GPU infrastructure can do things with an open trillion-parameter-class model that they simply cannot do with a closed one, regardless of API price. Most of us will never touch the raw weights. The fact that we're allowed to is the whole point.

Tomorrow closes out the heavyweight run with a model that takes the mixture-of-experts idea even further: Qwen3.8-2.4T (A95B MoE), and what it takes to activate just a sliver of a very large model at a time.

The numbers (measured, not quoted)

I ran three quick probes against Kimi K3 via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	30.7s	619	20.1	completed
Reasoning	13.3s	183	13.7	completed
Structured output	12.2s	449	36.9	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 1,048,576 tokens · hosted pricing $2.648/M input · $13.283/M output · weights: moonshotai/Kimi-K3 on Hugging Face

Credits — where it's due

Moonshot AI — for training Kimi K3 and releasing the weights openly: moonshotai/Kimi-K3. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

MiniMax M3: The Open-Weight LLM With 1M Context at $0.30 per M Tokens — Day 25/30

AI Explore — Mon, 14 Sep 2026 13:16:08 +0000

TL;DR — MiniMax M3 ships a 1,048,576-token context window priced at $0.30 per million input tokens, cheap enough to make 'just paste the whole codebase in' a real strategy instead of a joke. Probe results show fast, correct output on coding, arithmetic reasoning, and strict JSON extraction. The real question isn't whether it can hold a million tokens — it's whether stuffing that much text into a prompt beats a retrieval pipeline, and the answer depends entirely on what you're doing with the context once it's in there.

Every engineer who has built a RAG pipeline has had the same intrusive thought at 2am: what if I just... didn't. What if I skipped the chunking, the embedding model, the vector store, the reranker, and just pasted the entire document set into the prompt. MiniMax M3 is the model that makes that thought less insane, because it backs a 1,048,576-token context window with a $0.30 per million input token price tag, according to its listed metadata. That's the whole pitch: brute force, but affordable brute force.

What's actually in the box

MiniMax M3 (Hugging Face ID MiniMaxAI/Minimax-M3) is an open-weight model with a context length just over one million tokens and completion pricing at $1.20 per million output tokens. The input-to-output price ratio (4:1) tells you where this model wants to be used — reading, not writing. You're meant to pour in huge amounts of source material and get back something comparatively small: a summary, an extracted answer, a patch, a JSON object. That asymmetry is the entire economic argument for the "dump everything in" approach: input tokens are the cheap commodity here, and M3 is priced to reward hoarding them.

What the probes showed

I ran three quick probes rather than a context-stress test, but they're informative about whether the model is trustworthy once you've actually crammed that context window full. On a coding task — write merge_intervals for overlapping [start, end] pairs — it produced correct logic: sort by start, then merge whenever the current interval's start falls within the last merged interval's end. That's the standard, correct approach, generated at 223.4 tokens/second over 700 completion tokens in 3.1 seconds, which is snappy for a model this size. The visible output was cut off before it stated the complexity sentence the prompt asked for, so I can't grade that closing line, but the algorithm itself was right, which is the part that matters if you're using this for actual code review over a big repo.

The reasoning probe — a tank filling at 90 L/min while draining at 60 L/min for 20 minutes, then asking how much longer to fill the remaining volume alone — came back correct: 600 L filled in the first phase, 1,800 L remaining, 20 more minutes at 90 L/min. Clean, step-shown, no arithmetic slip. It ran fast too: 97.3 tokens/second, 1.5 seconds, 150 completion tokens. Nothing fancy, but nothing wrong, which is exactly the bar you need cleared before you trust a model with a million tokens of your data.

The structured output probe asked for strict JSON extracted from an invoice description — vendor, date, total. It returned valid JSON with all three fields correctly populated, including pulling the total ($445.50) rather than confusing it with the subtotal, at 76.2 tokens/second. For anyone planning to use M3 as the backend for document-extraction pipelines — which is a very natural use of a cheap, long-context model — that's the probe that matters most, and it passed cleanly.

Dump-everything vs. retrieval pipeline: when brute force wins

This is the actual editorial question, so let's be concrete about it. A retrieval pipeline — chunk, embed, index, retrieve top-k, stuff into a smaller context — exists to solve a cost and precision problem: models used to have small windows, and embedding search let you avoid paying for irrelevant tokens. M3's pricing changes the cost side of that equation. At $0.30/M input, feeding a full 400-page contract or a mid-sized codebase (say, 300,000 tokens) into a single prompt costs about nine cents. That's cheaper than the engineering time it takes to debug a broken retrieval index on a Tuesday afternoon.

Brute force wins clearly in a few situations. First, when the task requires holistic reasoning across the whole document — auditing a contract for inconsistent clauses, tracing a variable's usage across an entire codebase, finding all mentions of a specific claim across a long transcript. Retrieval-based chunking is bad at "look at everything and tell me if anything is wrong," because top-k similarity search assumes you know what you're looking for in advance. Second, when your corpus is small enough to fit whole but was previously excluded on cost grounds — a support team's full knowledge base, a legal team's single case file, a single customer's entire support history. Third, for one-off or exploratory tasks where building and maintaining a retrieval index isn't worth the investment: a one-time compliance sweep of a document set doesn't need a vector database that will be stale next week.

Where it falls apart

Brute force breaks down in three predictable ways. The first is scale: a million tokens is not infinite. A large monorepo, a year of Slack history, or a full legal discovery set will exceed it, and you're back to needing retrieval or summarization anyway — the context window just moved the ceiling, it didn't remove it. The second is latency and attention degradation: models generally get less reliable at pinpointing a needle buried deep in a huge haystack than at reasoning over material that's actually relevant, and stuffing the full context in doesn't guarantee the model weighs distant information correctly — that's a known soft spot for long-context models generally, not something these probes can confirm or deny for M3 specifically, so treat it as a risk to test on your own data before betting production traffic on it. The third is repetition cost: if you're running the same 300,000-token document through queries all day, you're paying for those input tokens every single call, whereas a retrieval index amortizes the embedding cost once and serves cheap lookups after that. Brute force is a great tool for infrequent, exploratory, or one-shot analysis over data too messy or too small to justify a pipeline. It's a bad tool for high-volume repeated queries against a stable corpus, where retrieval's upfront cost pays for itself fast.

Who should actually reach for this

Teams doing ad hoc document analysis — compliance review, contract auditing, codebase archaeology, "what changed in this six-month email thread" — get real value here because the $0.30/M input price makes exploratory, whole-document analysis cheap enough to run without justification. Teams building a production RAG-backed support bot or search product should not switch to "just paste it all in" as their architecture; the repeated-query economics don't favor it, and their corpus will likely outgrow a million tokens anyway. The honest rule of thumb: use M3's context window as a scalpel for one-off jobs on data too awkward to index, not as a replacement for retrieval infrastructure you're running at scale.

Tomorrow's episode looks at Kimi K3 — another open-weight model worth putting next to M3 on the same kind of long-document, cost-per-token math.

The numbers (measured, not quoted)

I ran three quick probes against MiniMax M3 via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	3.1s	700	223.4	completed
Reasoning	1.5s	150	97.3	completed
Structured output	1.3s	96	76.2	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 1,048,576 tokens · hosted pricing $0.3/M input · $1.2/M output · weights: MiniMaxAI/Minimax-M3 on Hugging Face

Credits — where it's due

MiniMax — for training MiniMax M3 and releasing the weights openly: MiniMaxAI/Minimax-M3. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Llama 4 Scout: The Open-Weight LLM With a 1.3M-Token Context Window — Day 24/30

AI Explore — Sun, 13 Sep 2026 13:16:19 +0000

TL;DR — Llama 4 Scout ships with a vendor-claimed 1.3 million token context window — big enough to hold a mid-size codebase or a full case file in one shot. Today's live probes found the model's structured-output extraction solid but wrapped in markdown it wasn't asked for, while two probes came back as outright fetch failures. The real story is what a window that size changes about system design, and where it still can't replace retrieval.

Most "long context" marketing is a rounding error dressed up as a feature. Llama 4 Scout is not that. Meta's stated context window for Scout lands around 1.3 million tokens — vendor-claimed, and worth saying plainly that our live metadata pull today came back null for context length, pricing, and Hugging Face ID, so none of that is independently confirmed in this run. But the number itself, wherever you first heard it, is the whole reason this model gets talked about: it's large enough to hold a real codebase, a legal case file, or a quarter's worth of support tickets in a single prompt, no chunking required.

What actually happened in the probes

Three tasks, three very different outcomes. The code generation probe (merge overlapping intervals, plus a one-sentence complexity claim) and the reasoning probe (a tank-filling word problem) both came back as fetch failed with a 0.5-second latency and zero completion tokens — not a bad answer, just no answer. That's an infrastructure miss on this particular pull, not a verdict on the model's reasoning, and I'm not going to pretend otherwise by inventing scores for tasks that never returned.

The structured-output probe did complete: extract vendor, date, and total from an invoice string, return only the JSON object. It ran in 0.7 seconds, produced 37 completion tokens at roughly 50 tokens/sec, and the JSON itself was correct — vendor, date, and total all pulled cleanly. The one flaw: the model wrapped the object in a markdown code fence (json...) despite being told to return the JSON object alone. That's a small thing until you're piping output straight into a parser expecting bare JSON — then it's a five-minute debugging session you didn't need. Worth knowing before you wire this into anything unattended.

What 1.3M tokens actually changes

Context windows in the 8K–32K range force an architecture decision on you whether you want it or not: you chunk, you embed, you retrieve, you re-rank, you hope the retriever picked the right five paragraphs. A window sized for a whole repository or a whole case file removes that decision for a meaningful slice of workloads. You can hand over an entire module — models, migrations, tests, config — and ask for a refactor that respects invariants scattered across files a retriever might never have surfaced together. You can hand over an entire deposition transcript and ask for every place a witness contradicted themselves, without pre-guessing which passages matter.

This is a genuine shift, not a marketing footnote. Retrieval-augmented pipelines exist largely because context was scarce and expensive. When the scarcity goes away for a given document size, an entire layer of infrastructure — the vector store, the chunking strategy, the re-ranker, the eval suite for retrieval quality — becomes optional for that use case. Optional doesn't mean gone; it means you get to choose it deliberately instead of by necessity.

Where this actually earns its keep

The honest use cases cluster around three shapes:

Whole-repo code review and refactoring. A mid-size service — say a few hundred files — can fit in one prompt alongside its tests and its architecture doc. Cross-file bugs (a constant defined once and silently reused wrong three modules away) are exactly the class of problem retrieval chunking tends to miss, because the retriever never had a reason to fetch both pieces together.
Case-file and contract analysis. Legal and compliance teams dealing with document sets that are large but bounded — a single litigation file, a single M&A due-diligence packet — can load the whole thing and ask cross-referencing questions ("does clause 14 in the amendment contradict section 3 of the original agreement?") without betting on a retriever's relevance ranking.
Long-running support and incident history. A quarter of tickets for one customer, or a year of postmortems for one system, fits in-window. Asking "has this exact failure mode shown up before, described differently" is a task retrieval handles poorly (different wording, same bug) and long context handles by just reading everything.

Where it doesn't earn its keep: anything that needs to search across a corpus larger than the window — a company-wide knowledge base, a multi-year document archive, a codebase in the tens of thousands of files. At that point you're back to retrieval whether you like it or not, and pretending a big window makes indexing unnecessary just moves the failure from "wrong chunk retrieved" to "prompt doesn't fit, silently truncated, nobody notices."

The honest limit: recall isn't retrieval

This is the part vendor pages gloss over. A model that can accept 1.3 million tokens is not the same as a model that reliably uses everything in those 1.3 million tokens. Long-context recall degrades unevenly — models are generally better at attending to the beginning and end of a huge prompt than the middle, and "needle in a haystack" performance is not flat across the window. None of that shows up in the marketing number, and none of it showed up in our probes today either, because two of the three tasks we ran didn't have anything to do with long-context recall at all — they were short-prompt sanity checks, and even those failed to return. That's a gap in today's data, not a claim about Scout's long-context behavior one way or the other, and it's worth being upfront about rather than filling it with a number I don't have.

The practical rule of thumb, independent of any specific benchmark: treat a huge context window as a capacity limit, not a quality guarantee. If a fact absolutely must be found and used correctly, don't just dump it into token 700,000 of a 1.3M-token prompt and hope. Put it near the start or end, or better, keep a lightweight retrieval step as a safety net even when you technically don't need it to fit — cheap insurance against attention that isn't as uniform as the spec sheet implies.

Where this leaves you

Llama 4 Scout is the right tool when your document, your repo, or your case file is big but bounded, and you'd rather hand over the whole thing than trust a retriever to guess the right five paragraphs. It's the wrong tool if you're treating "1.3 million tokens" as a substitute for an index over a corpus that's actually unbounded. And today's probe run is a small reminder that the plumbing between you and any model — the fetch, the parsing, the stray markdown fence — deserves as much attention as the model's raw capability. The context window is the headline; the JSON coming back wrapped in backticks you didn't ask for is the part that actually breaks your pipeline at 2 a.m.

Tomorrow's episode turns to MiniMax M3 — a very different bet on what an open-weight model should optimize for, and worth seeing right after a day spent on context size alone.

The numbers (measured, not quoted)

I ran three quick probes against Llama 4 Scout via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	—	—	—	failed: fetch failed
Reasoning	—	—	—	failed: fetch failed
Structured output	0.7s	37	50	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window — · hosted pricing see current provider pricing · weights: (link on the model's provider page)

Credits — where it's due

Meta AI — for training Llama 4 Scout and releasing the weights openly: the model's provider page. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Qwen3 Coder 30B A3B: The Open-Weight MoE Model for Local Coding Agents — Day 23/30

AI Explore — Sat, 12 Sep 2026 13:17:03 +0000

TL;DR — Qwen3 Coder 30B A3B is a mixture-of-experts model with 30B total parameters but only 3B active per token, aimed at laptop-speed coding help. Probe runs show clean interval-merging code, correct multi-step arithmetic, and valid JSON extraction, all at a 262,144-token context window and sub-$0.30/M completion pricing. It's a serious local option for autocomplete-style and agentic coding tasks, with honest limits versus hosted frontier assistants on gnarly, cross-file reasoning.

Act IV of this series is about models that show up in your actual workflow, not just your terminal experiments. Qwen3 Coder 30B A3B is the first one that belongs in your editor's sidebar. It's a mixture-of-experts coding model — 30 billion parameters total, but only about 3 billion active per forward pass — built specifically to answer the question every engineer running local models eventually asks: can this thing keep up with me while I'm typing?

What "A3B" Actually Buys You

The MoE design means the model routes each token through a small subset of expert layers instead of the full stack. That's why a "30B" model can behave, latency-wise, more like a much smaller one. In the probe run, the model produced a complete, correctly structured Python function in 12.1 seconds at 18.7 tokens/second — not blazing, but workable for an editor completion or a chat-style ask inside an IDE. A simpler JSON extraction task finished in 1.2 seconds at 28.5 tokens/second, which is the number that actually matters for autocomplete-adjacent use: short, structured asks come back fast.

Context window is the other headline figure: 262,144 tokens, per the model's metadata. That's large enough to hold a genuinely big chunk of a codebase — several mid-sized files, a test suite, and a task description — in a single pass, without the constant summarizing and re-chunking that smaller-context local models force on you.

Judging the Probe Output Honestly

I ran three tasks and read the transcripts line by line rather than taking the model's word for it.

The interval-merging function is correct. It sorts by start time, seeds the merged list with the first interval, then walks the rest: if the current start is less than or equal to the last merged end, it extends the end with a max; otherwise it appends a new interval. That's the standard, right algorithm, and the in-place sort plus single pass gives it the expected O(n log n) behavior overall (the sort dominates; the merge pass itself is linear). The code is clean enough to paste into a project without edits, which is the bar that matters for a coding assistant, not "does it compile in isolation."

The tank-filling reasoning problem is arithmetic, not code, but it's a fair proxy for whether the model can hold a multi-step plan in its head. Net fill rate for 20 minutes at 30 liters/minute gives 600 liters; remaining 1,800 liters divided by pump A's 90 liters/minute gives 20 more minutes. Both steps and the final answer check out — no dropped units, no sign error, which is a common failure mode in smaller local models on exactly this kind of layered word problem.

The JSON extraction task is the cleanest signal for anyone planning to wire this into a pipeline: the output is a single valid JSON object with the three requested keys, no markdown fences, no preamble, no trailing commentary. For a model meant to sit inside agent loops and tool calls, that discipline — say only what was asked, in the format asked — is worth more than raw benchmark bragging rights.

Where This Actually Fits: Editors and Agents

This is the use-case section, so let's be concrete about who should reach for this model and who shouldn't.

Local-first IDE completion. If you're running something like a Continue- or Cursor-style local backend and want inline suggestions without shipping every keystroke to a hosted API, the A3B's low active-parameter count is the whole point — it's the difference between a completion that lands before you finish the line and one that makes you wait.
Agentic coding loops on a budget. At $0.07 per million input tokens and $0.28 per million output tokens (per the listed pricing, whether you self-host and compare to that ceiling or actually route through a provider charging it), an agent that reads a large diff, proposes a patch, and re-reads test output in a loop stays cheap even with the 262K context window fully in play. That context size means a multi-file refactor task doesn't need aggressive chunking to fit.
Air-gapped or regulated dev environments. Teams that can't send proprietary source to a third-party API — defense contractors, some fintech shops, anyone under a strict NDA — get a coding assistant that never leaves the network. That's not a performance advantage, it's a policy one, but it's often the deciding one.
CI-adjacent structured tasks. The clean JSON behavior in the probe suggests this model is a reasonable fit for pipeline steps like "extract the failing test names from this log" or "summarize this PR diff into a structured changelog entry" — jobs where format compliance matters more than creative problem-solving.

Where it's the wrong tool: anything that needs the absolute frontier of reasoning depth — gnarly concurrency bugs, subtle cross-service architecture tradeoffs, or the kind of "why does this fail only in production" debugging that benefits from a much larger, more expensively trained model's broader pattern library. Hosted flagship assistants still have an edge there, and pretending a 3B-active-parameter model closes that gap entirely would be overselling it. The honest pitch for Qwen3 Coder 30B A3B is: it's very good at the 80% of coding work that's mechanical, well-specified, or format-driven, and it's local, fast, and cheap enough to run constantly instead of sparingly.

The Trade-Off, Stated Plainly

Vendors will tell you MoE coding models "match" dense models several times their active size — that's a vendor claim, not something this probe set can confirm or deny at scale. What the probe set does confirm is narrower and more useful: on a correct-algorithm task, a multi-step arithmetic task, and a strict-format extraction task, the outputs were all correct and well-formed, at latencies that make it plausible as a live coding companion rather than a batch-job model. That's the realistic bar for "local coding help" in 2026 — not beating the best hosted assistant on the hardest problem you can find, but being good enough, fast enough, and cheap enough that you actually reach for it first.

Tomorrow's episode in this 30-day series turns to Llama 4 Scout — a different bet on what an open-weight model should optimize for.

The numbers (measured, not quoted)

I ran three quick probes against Qwen3 Coder 30B (A3B MoE) via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	12.1s	227	18.7	completed
Reasoning	7.2s	164	22.9	completed
Structured output	1.2s	33	28.5	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 262,144 tokens · hosted pricing $0.07/M input · $0.28/M output · weights: Qwen/Qwen3-Coder-30B-A3B-Instruct on Hugging Face

Credits — where it's due

the Qwen team at Alibaba Cloud — for training Qwen3 Coder 30B (A3B MoE) and releasing the weights openly: Qwen/Qwen3-Coder-30B-A3B-Instruct. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Open-Weight AI in Robotics: The VLA Models Running Without Cloud — Day 22/30

AI Explore — Fri, 11 Sep 2026 13:16:20 +0000

TL;DR — Vision-language-action (VLA) models like OpenVLA give robots a slow, high-level 'brain' that plans grasps and subgoals, while a classical controller still closes the real motion loop at hundreds of Hz. Open-weight VLAs let labs and factories fine-tune manipulation policies on their own hardware without shipping video to the cloud, but the models are still compute-bound multi-billion-parameter systems that need real GPUs, not microcontrollers.

A robot arm doesn't think the way a chatbot does. It can't afford to. Ask a language model a question and a half-second of latency is invisible. Ask a robot to close its fingers around a coffee mug and a half-second of hesitation is a dropped mug. That mismatch — between how slow it's safe for a "brain" to think and how fast a body has to move — is the whole engineering story of open-weight AI in robotics, and it's why the field ended up splitting the job into two systems instead of one.

The split brain: intent at 1-10 Hz, motion at 1,000 Hz

No published on-robot vision-language-action (VLA) system runs the actual balance or motion control loop. Instead, a VLA or LLM component runs at roughly 1-10 Hz on onboard compute — Jetson-class boards or Hailo-10H NPUs — and produces task-level intent: a target pose, a grasp point, a subgoal, according to a detailed breakdown of on-robot inference constraints from promptquorum.com. A separate classical controller — PID or MPC, usually on a microcontroller or dedicated real-time core — closes the actual motion loop at 100 to 1,000 Hz. The VLA says "pick up the mug." The controller figures out, a thousand times a second, exactly how many degrees to rotate each joint to get there without knocking anything over.

This isn't a workaround, it's the architecture. A multi-billion-parameter model doing vision and language reasoning is compute-bound work — the kind that wants a real GPU, not a microcontroller — as semianalysis.com put it plainly in a piece on where robots actually do their thinking. Nobody is routing an emergency stop through a language model. Safety interlocks stay deterministic; the LLM only gets to propose, never to directly actuate.

OpenVLA: what open-weight looks like in the wild

The reference point for open VLA work is OpenVLA, a 7-billion-parameter open-source model pretrained on 970,000 robot episodes from the Open X-Embodiment dataset. It's built from a fused SigLIP + DinoV2 visual encoder feeding a Llama 2 7B backbone that predicts tokenized actions, decoded straight into joint commands. On out-of-the-box evaluation across the WidowX (Bridge V2) and Google Robot (RT-series) platforms, OpenVLA's authors report it outperforming the prior generalist policies RT-1-X and Octo — and notably outperforming RT-2-X, a 55-billion-parameter closed VLA nearly eight times its size.

The more interesting number for practitioners is adaptation speed. The team fine-tuned OpenVLA onto two new physical setups: a stationary Franka Emika Panda arm controlled at 5 Hz, and the Franka-DROID rig running at 15 Hz — using parameter-efficient fine-tuning rather than a from-scratch training run. That's the open-weight pitch in miniature: you're not building a manipulation policy from zero, you're adapting a pretrained one to your particular arm, your particular gripper, your particular lab bench.

Where this actually gets used

Three constraints keep showing up in real deployments, and they explain why open weights matter here specifically rather than just being ideologically nice to have:

Academic and startup manipulation research. Labs running Franka arms or WidowX setups don't have RT-2-X's training budget or Google's fleet of robots. OpenVLA gives them a checkpoint that already generalizes to unseen objects, positions, and instructions, and lets them spend their compute on fine-tuning to a specific task rather than on pretraining a vision-language-action stack from scratch — a cost gap that's the difference between a PhD project shipping and not shipping.
Multi-embodiment fleets without a single vendor lock-in. OpenVLA supports multiple robot platforms out of the box and is designed to be adapted to new robot setups via parameter-efficient fine-tuning. That matters for anyone running a mixed fleet of arms, grippers, and mobile bases who doesn't want a separate closed model contract per robot body.
Sites that can't or won't send camera feeds anywhere. The same logic that keeps factory-floor LLMs air-gapped — process recipes and order histories being a legal and competitive-intelligence risk, not just a technology one, as one industrial deployment writeup put it via dev.to — applies directly to a robot arm's camera stream. A manipulation policy watching a proprietary assembly line is watching something a competitor would pay to see. Running the VLA on local Jetson-class compute instead of a cloud API means that footage never leaves the cell.
Field robots without reliable connectivity. Warehouse pickers, agricultural robots, and disaster-response platforms operate exactly where the "just call an API" model breaks: intermittent or nonexistent connectivity, and a task-level decision that has to happen in under a second regardless. The split-brain architecture — slow VLA for intent, fast local controller for motion — is the only version of "AI-driven robot" that survives a dead network connection.

The ceiling, and where closed models still lead

Be honest about the gap. Google DeepMind's Gemini Robotics 2 announcement demonstrated a humanoid — Apptronik's Apollo — walking to a table, picking up a watering can, and placing it on a shelf from a single natural-language instruction, with a companion "Gemini Robotics On-Device 2" model that DeepMind says can adapt to a new bi-arm embodiment in a few hours using fewer than 200 examples. That's a vendor claim about a closed model, not an open one, and whole-body humanoid control is a meaningfully harder problem than tabletop manipulation — but it's a real signal of where the frontier is moving, and it's the bar open VLA projects are implicitly racing toward.

The honest trade-off: open-weight VLAs give you inspectable, fine-tunable, deployable-anywhere manipulation policies today, at the 7B-parameter, single-arm, tabletop scale that OpenVLA operates at. They don't yet give you out-of-the-box whole-body humanoid coordination or multi-robot teaming. If your job is teaching a Franka arm a new grasp without sending video to anyone, open weights already do it. If your job is a humanoid crossing a room, the frontier is still mostly closed.

Credits & sources

Technical detail on the VLA control-loop split and edge compute constraints comes from promptquorum.com. The compute-bound argument for why VLAs need real GPUs rather than microcontrollers is from semianalysis.com. Model architecture, benchmark, and fine-tuning figures for OpenVLA are drawn from the project's own page at openvla.github.io, maintained by its research authors. Details on Gemini Robotics 2 and its on-device variant come from <a href="https://deepmind.google/blog/gemini-robotics-2-br

Appendix — the field in one chart

Open-Weight AI in Cars: Local LLMs Built for a 15-Year Vehicle Life — Day 21/30

AI Explore — Thu, 10 Sep 2026 13:16:06 +0000

TL;DR — Carmakers are quietly choosing open-weight models over API-based assistants because a car has to work in a signal-dead parking garage and keep working for 15 years after the vendor stops caring. This episode looks at real deployments — from a Raspberry Pi running a 35B model as an in-car chat agent to Phi-2 running diagnostics on ECU-class hardware — and why owning the weights, not renting them, is becoming the default for in-cabin AI.

Your phone gets replaced every two or three years. Your car doesn't. The average vehicle on the road today is over 12 years old, and a new one sold this year is expected to still be running in 2041. Now ask yourself: which AI vendor is confidently offering an API contract that lasts until 2041? None of them. That mismatch — hardware with a multi-decade lifespan, paired with an AI industry that deprecates endpoints every 18 months — is the quiet reason open-weight models are showing up under the hood.

The parking garage problem

Every cloud-based car assistant has the same failure mode: the underground garage, the mountain tunnel, the rural dead zone, the country where the carrier partnership didn't get signed. A voice assistant that goes silent the moment signal drops isn't a degraded experience, it's a broken product. That's the design constraint a hobbyist project called CarWatch takes seriously: a Raspberry Pi 5 riding in the car, running a 35B-parameter Qwen3.6-A3B model fully locally, quantized down to 14.3GB with Unsloth's dynamic UD-Q3_K_S format. Measured on real hardware — a ~€300 Pi 5 with 16GB of RAM — it generates at 3.5 tokens/second with 25+ tok/s on prompt processing, sustaining 65°C with no cloud, no internet, no subscription, as detailed in the project's own benchmarks and independently written up by gokawiil.com.

The architecture is the interesting part: local is the product, online is the enrichment. Voice input, the assistant's answers, and the owner's-manual RAG (built from the actual 745-page manual, shipped on the SD card with page citations) all work with zero connectivity. Room posts and clip uploads queue in a persistent on-disk outbox and deliver late rather than getting lost. Only reachability, self-updates, and escalation to a bigger cloud model are online-only — and even the escalation path checks a local LAN model server first. That's not a hobbyist quirk; it's the same hierarchy any carmaker would need to build for a feature that has to survive fifteen years of connectivity gaps.

Diagnostics on hardware that already exists

Not every in-car AI needs 35B parameters and a dedicated Pi. A separate project, tinyllm-auto, targets the opposite end: can a conversational assistant run on existing ECU-class silicon, under 8GB of RAM, with no GPU at all? The answer, benchmarked on a Raspberry Pi 4 (8GB), was Phi-2 at 2.7B parameters, 4-bit GGML quantized down to 1.6GB on disk and 3.2GB at runtime, delivering 450ms first-token latency and 45 tokens/second generation — beating a target of 30 tok/s and staying under a 4GB memory ceiling. The project's own model bake-off is worth noting: TinyLlama (1.1B) scored only 62% on a 500-question automotive QA benchmark, Mistral-7B scored highest at 84% but needed 4.1GB, and Phi-2 landed at 78% accuracy as the best fit for the size and latency budget. That's a real trade-off table, not a vendor slide: bigger models answer more diagnostic questions correctly, but the ECU doesn't have the RAM to run them.

Use cases: where this actually gets deployed

Strip away the demo projects and three workloads keep showing up in both hobbyist repos and industry roadmaps:

Diagnostic code translation. "What does P0420 mean?" is exactly the kind of narrow, high-value query a small quantized model handles well — tinyllm-auto's benchmark shows 82% accuracy on diagnostic-code questions specifically, the strongest category in its 500-question test. A cloud LLM does this fine too, but only if the car has signal in the exact moment the check-engine light comes on, which is precisely when a driver is often in a garage or on a rural shoulder.
Owner's-manual Q&A with refusal behavior. CarWatch's RAG pipeline is built to refuse questions the manual doesn't answer, rather than hallucinate a torque spec. For a safety-adjacent domain like vehicle maintenance, a model that says "I don't know" is worth more than one that guesses fluently — and that refusal logic is something you control when you own the weights and the retrieval pipeline, not something you can guarantee from a black-box API.
Cabin monitoring and system self-awareness. CarWatch's assistant reads its own temperature, throttling state, fan, memory, disk, and network status live from the machine before answering questions about itself — the system prompt is built so an unmeasured value can never silently present as a known fact. That's a small but telling design pattern: grounding the model in what the hardware can actually sense, rather than letting it narrate confidently about things it has no sensor for.

The honest trade-off across all three: local models on car-grade hardware are slow by desktop standards — 3.5 tok/s on a Pi 5 running a 35B model is fine for a conversational reply, unusable for anything requiring fast back-and-forth. Carmakers accept that latency in exchange for an assistant that never goes dark.

Why carmakers own the weights, not rent the API

This isn't just a hobbyist pattern — it maps onto where the automotive industry is actually spending. According to an analysis from sbdautomotive.com on Li Auto's custom silicon push, computing capability is already the second-largest cost component in a vehicle after the battery, with per-vehicle compute costs projected to climb from around $2,000 today toward $8,000–$10,000 as multi-modal, always-on AI features expand. The piece frames the automaker's own chip investment as a bet that outsourcing the "digital soul" of the car — meaning the models and the silicon that run them — is as risky as outsourcing the engine was in the combustion era. Waymo has taken the parallel path on the perception side: a custom 5nm chip delivering roughly 1,000 TOPS of dedicated edge compute per vehicle, purpose-built to run neural networks on raw sensor data before it ever leaves the car, according to a technical breakdown from ai2.work. Different companies, same conclusion: if the model is core to the product, you want to own the weights and, increasingly, the silicon they run on.

Open weights fit that logic cleanly. A model you can download, quantize, and freeze onto a chip is a component you control for the vehicle's full service life — no API deprecation notice, no pricing change, no dependency on a vendor's roadmap outliving your warranty period. A closed API is a subscription with a kill switch fifteen years out; an open-weight checkpoint on an SD card is a part you can still order.

Credits & sources

Thanks to the maintainer of xen1024/CarWatch for the detailed, benchmarked write-up of a real offline in-car assistant, and to gokawiil.com for the independent Show HN coverage of that project. Thanks to the maintainer of sreekarvamsi/tinyllm-auto for publishing the model comparison and benchmark tables that made the ECU-scale trade-offs concrete. Thanks to sbdautomotive.com for the analysis of Li Auto's in-house chip strategy, and to ai2.work for the breakdown of Waymo's custom edge-compute silicon.

Tomorrow, Day 22 moves from the dashboard to the chassis: open-weight AI in robotics, and what changes when the model has to control a body instead of just talking to one.

Appendix — the field in one chart

Open-Weight AI in Healthcare: HIPAA-Compliant LLMs for Clinical Notes — Day 20/30

AI Explore — Wed, 09 Sep 2026 13:16:01 +0000

TL;DR — Healthcare is one of the clearest cases for open-weight AI: HIPAA's Security Rule and PHI liability push clinical teams toward models they can run entirely inside their own network. This episode covers what's actually deployed today — DeepSeek-R1 in a German university hospital, clinical documentation drafting, medical coding assist, and literature triage — plus the regulatory mechanics that make on-prem the default rather than the exception.

Most industries treat data residency as a preference. Healthcare treats it as a legal exposure with your name on it. That difference is why open-weight AI has quietly become the practical default in clinical IT, not because open models are smarter, but because closed APIs put a third party in the middle of protected health information, and nobody wants to be the compliance officer explaining that decision after an audit.

The mechanics: why PHI forces the on-prem question

HIPAA's Security Rule, codified at 45 CFR Part 164, applies to any system that transmits or maintains electronic PHI — and an LLM pipeline that ingests a visit transcript and returns a draft note is squarely that system. As one detailed breakdown of the rule puts it, "every component of that pipeline is subject to the Security Rule's safeguards," and organizations that bolt cloud AI tools onto existing workflows without updating their Security Risk Analysis are, in practice, already out of compliance the moment an auditor looks closely (arjunjaggi.com).

Self-hosting an open-weight model changes the shape of that liability. If PHI never leaves infrastructure you control, there's no third-party model vendor processing the data — which means there's no Business Associate Agreement gap to manage with that vendor in the first place. One HIPAA-focused engineering write-up frames it plainly: "there is no third-party BAA gap to manage and no foreign-jurisdiction exposure; your own legal perimeter is the only boundary that matters" (stribog.com). You still own the entire compliance burden — the access controls, the audit logging, the risk analysis — but you remove one processor from the PHI flow, and in a regulated industry that's not a small thing.

What's actually running today

The clearest deployment on record comes from a German university hospital that put DeepSeek-R1, served through vLLM, inside a fully sealed radiology environment — no outbound internet connectivity, host-enforced egress filtering, and automated isolation tests that ship with the deployment package so other institutions can verify the seal before going live (pivotnews.ai). The system cleared a one-week prospective pilot with 22 residents and radiologists using ten predefined prompt templates in daily clinical work, and it's now running as an official hospital service.

The pilot results are worth sitting with because they're not a marketing win — they're an honest capability map. Text-anchored tasks, like correcting or simplifying existing radiology reports and surfacing guideline-based recommendations, scored highest for clinical utility. Open-ended generation — specifically drafting conclusions from a raw list of findings — produced the most critical errors, including hallucinations and omitted significant findings. That's a real signal for anyone deploying open-weight models in clinical settings: the model is good at rewriting and retrieving, and worse at generating clinical judgment from scratch. Design your workflow around that split, not around what a demo makes look easy.

The use cases that are actually working

Across multiple independent write-ups on HIPAA-compliant AI deployment, the same three use-case categories keep surfacing, and the common thread is deliberate: the task is well-defined, a qualified human verifies the output before it touches patient care, and the PHI surface is bounded.

Clinical documentation drafting. Turning a visit transcript into a structured note, suggesting ICD-10/CPT codes from the note text, and drafting prior-authorization letters from structured patient data. The model generates a draft; a clinician reviews, edits, and signs before anything enters the record or a billing system (arjunjaggi.com).
Medical literature triage. Summarizing PubMed abstracts, synthesizing treatment guidelines for a specific condition, or generating differential-diagnosis literature summaries. Crucially, this class of work touches no patient PHI at all — it's public scientific text — so it can run on smaller models with a dramatically simpler compliance posture, which makes it a good on-ramp for practices not ready to build a full PHI-grade pipeline (arjunjaggi.com).
Administrative automation with PHI kept out of the prompt. Scheduling messages, referral letter templates, and eligibility-verification query drafting can often be restructured so the LLM works on a template and patient identifiers get inserted afterward — keeping the model PHI-free while still saving staff time (arjunjaggi.com).
Internal knowledge search over policies and coding references. Letting staff ask plain-language questions against internal protocols and payer rules, with citations back to the source document, so answers are checkable rather than trusted blindly (layer3labs.io).

Notice what's absent from that list: nothing here makes a clinical decision unattended. That's not caution for its own sake — it's the actual shape of what's deployed, because the failure mode of an unreviewed hallucinated finding in a radiology report is not a bug ticket, it's a patient safety incident.

What the architecture actually requires

A compliant on-prem LLM pipeline isn't just "download the weights and run vLLM." The recurring architectural checklist across independent sources includes a PHI detection layer at the application boundary, an encrypted and write-once audit log tied to §164.312(b)'s audit-control requirement, a network boundary that keeps inference traffic off the public internet entirely, and identity management that ties every query to an authenticated user (arjunjaggi.com, stribog.com). None of these controls are exotic on their own — role-based access and audit logging are standard IT hygiene — but mapping them explicitly to a new AI inference layer is the part most organizations skip until an audit forces the issue.

Federal and sovereign-data contexts push the same direction for a slightly different reason: air-gap capability. A model that can run with zero outbound network connectivity is the only option when the deployment environment is a closed enclave by policy, not just by preference (conselara.dev).

The trade-off nobody should skip

Open-weight deployment removes a vendor from your PHI flow, but it does not remove the compliance work — it transfers it to you. Cost logic favors self-hosting once you're processing thousands of documents or messages a month, since fixed infrastructure beats metered per-token billing at volume (layer3labs.io). Below that volume, or without staff to own the Security Risk Analysis and audit infrastructure, a well-contracted API with a signed BAA may honestly be the more responsible choice for a small practice. Open weights are the right answer when the PHI volume, the customization need, or the sovereignty requirement is high enough to justify owning the whole stack — not by default just because the word "open" sounds safer.

Credits & sources

Reporting and analysis drawn from layer3labs.io's HIPAA guide to open-weight healthcare deployment, Arjun Jaggi's detailed breakdown of open-source LLMs in HIPAA-compliant architectures at arjunjaggi.com, the on-prem DeepSeek-R1 radiology deployment reported by pivotnews.ai, the data-boundary framing from conselara.dev, and the BAA-gap analysis from stribog.com. Thanks to the maintainers of vLLM, whose serving stack sits at the center of the hospital deployment described above.

Tomorrow, Day 21 heads somewhere with a very different set of constraints: open-weight AI in cars.

Appendix — the field in one chart

Xiaomi MiMo-V2.5: Open-Weight Omnimodal AI for Self-Hosted Voice and Video — Day 19/30

AI Explore — Tue, 08 Sep 2026 13:16:26 +0000

TL;DR — MiMo-V2.5 is Xiaomi's open-weight model built to take audio, image, and video natively in one context rather than piping a transcript through a separate text model. Today's probes only exercised its text reasoning, code, and JSON-extraction abilities — all clean — while the omnimodal side rests on vendor description, not measured benchmarks. The real draw for builders is a 1.05M-token context and self-hosting, which matters most for meeting intelligence and call analytics in privacy-bound fields.

Most "multimodal" open-weight models are a text model with a transcription step duct-taped to the front door: Whisper turns audio into words, an OCR pass turns an image into words, and the LLM never actually hears or sees anything. MiMo-V2.5 is pitched differently — Xiaomi's release notes describe it as taking audio, image, and video into the same context as native tokens, not as a pre-processed transcript. That distinction matters for anything where tone, timing, or visual detail carries information text can't capture: a sigh mid-sentence on a support call, a whiteboard sketch in a meeting recording, a gesture in a video clip.

What's actually verified here

The metadata I have permission to quote is narrower than the marketing story, so let's be precise about it. MiMo-V2.5 (Hugging Face ID XiaomiMiMo/MiMo-V2.5) ships with a context length of 1,050,000 tokens — big enough to hold a multi-hour meeting recording's worth of tokens without chunking, assuming the audio pipeline tokenizes efficiently. Pricing on the hosted route I probed is $0.14 per million prompt tokens and $0.28 per million completion tokens. Filling that entire context once would cost roughly $0.147 just in prompt tokens — cheap enough that long-context use isn't the expensive part; the inference compute to actually decode a long video is.

What the probes actually tested (and what they didn't)

Here's the honest part: the probe suite run against this model today was text-only — a coding task, an arithmetic reasoning task, and a structured-extraction task. I have no audio, image, or video transcripts to grade, so I can't verify the omnimodal claim against measured evidence. What I can verify is the text reasoning stack underneath it, since that's the same model doing the "thinking" regardless of which modality fed it.

On the coding probe — write merge_intervals and state its time complexity — the model sorted by start time, merged correctly, and returned [[1,6],[8,10],[15,18]] for the test case, which is right. It also correctly named the complexity as O(n log n), driven by the sort. That took 6.6 seconds for 239 completion tokens, about 36.4 tokens/sec.

The reasoning probe — a two-pump tank problem — is where I look hardest for silent arithmetic drift, and there wasn't any: net fill rate of 30 L/min for 20 minutes gives 600 L, leaving 1,800 L, which at 90 L/min alone takes exactly 20 more minutes. Correct, and shown in four clean steps, in 4.5 seconds at 47.7 tokens/sec — the fastest of the three runs.

The structured-output probe asked for strict JSON with three keys pulled from an invoice. The model returned valid JSON with the right vendor and date, though it kept the dollar sign in the total field rather than emitting a bare number — technically faithful to the source text, but the kind of detail that will bite you if a downstream system expects 445.50 as a float rather than a string. Worth a schema note or a stricter prompt if you're piping this into anything that parses currency.

The use case: ears and eyes that don't leave the building

The editorial reason to care about MiMo-V2.5 isn't the invoice-parsing accuracy — plenty of smaller text models do that fine. It's that a native omnimodal model, self-hosted, gives you a single inference stack for jobs that currently require stitching together three separate services: a speech-to-text vendor, a vision API, and an LLM, each with its own data-handling agreement.

Meeting intelligence: feed the raw audio (or video, if screen shares matter) directly into a 1.05M-token context and ask for action items, decisions, and who committed to what — without a transcript vendor ever touching the recording.
Call center analytics: tone, hesitation, and cross-talk carry signal that a clean transcript throws away. A model that ingests audio natively can flag frustration or confusion patterns a transcript-only pipeline misses entirely.
Voice interfaces: on-device or on-prem voice assistants that need to reason about what was said and how it was said, in one pass, without round-tripping through a cloud STT API.

What ties all three together is the same constraint: none of them can send raw audio or video to a third-party API in a regulated environment. Call recordings, meeting footage, and voice logs are exactly the kind of artifact that legal, healthcare, and financial teams are told not to hand to an external vendor. A self-hostable model with the weights on Hugging Face — at $0.14/$0.28 per million tokens if you do use a hosted endpoint, or free of that fee entirely if you run it yourself — is the shape of tool those teams actually need, which is precisely the setup tomorrow's episode leans on.

The honest gap

I want to flag this plainly rather than bury it: everything above about audio and video is the model's stated design, not something I measured today. The probes prove the text reasoning core is solid — correct algorithms, correct arithmetic, valid JSON — which is a reasonable proxy for "the brain works," but it's not proof that the audio or video encoders perform well on real recordings, handle background noise, or hold up on long video with fast cuts. If you're evaluating MiMo-V2.5 for a voice or video workload, budget time to run your own probes on your own audio before you commit infrastructure to it. Treat the omnimodal capability as a vendor claim worth testing, not a fact worth assuming.

Should you deploy it

If your workload is genuinely multimodal and privacy-bound — meeting recordings, call audio, video review — MiMo-V2.5's pitch is worth a pilot precisely because the alternative is three vendors and three data-sharing agreements instead of one model you control. If your workload is really just text with a transcript step in front of it, you don't need this model; a smaller, cheaper text-only model fed a transcript will do the same job for less compute. The 1.05M-token context is the single most interesting number in the metadata — it's the difference between chunking a two-hour recording into fragments and reasoning over the whole thing at once.

Tomorrow's episode turns to open-weight AI in healthcare — a field where "self-hostable" isn't a nice-to-have, it's the whole reason a model gets considered at all.

The numbers (measured, not quoted)

I ran three quick probes against Xiaomi MiMo-V2.5 via a hosted API today before writing this. Small sample, one snapshot in time, routed hardware I don't control — treat these as a smell test, not a benchmark:

Probe	Wall-clock	Output tokens	Tokens/sec (effective)	Result
Code	6.6s	239	36.4	completed
Reasoning	4.5s	215	47.7	completed
Structured output	2.9s	95	32.9	completed

Effective tokens/sec includes queueing and time-to-first-token — it's what you actually experience, not peak decode speed.

Model card: context window 1,050,000 tokens · hosted pricing $0.14/M input · $0.28/M output · weights: XiaomiMiMo/MiMo-V2.5 on Hugging Face

Credits — where it's due

Xiaomi — for training Xiaomi MiMo-V2.5 and releasing the weights openly: XiaomiMiMo/MiMo-V2.5. Open releases like this are why a series like this can exist at all.
OpenRouter — the hosted API used for today's live probes and the pricing/context figures.
The quantizers and runtime maintainers — the mostly-unpaid people who turn every open release into something that runs on real hardware within days.

Open-Weight AI in Space: Gemma 3 Runs Onboard NASA's YAM-9 Satellite — Day 18/30

AI Explore — Mon, 07 Sep 2026 13:16:02 +0000

TL;DR — NASA's JPL flew a 4-bit quantized, unmodified Gemma 3 4B on a Loft Orbital satellite this year, running vision-language inference on live Earth imagery with no ground-based datacenter in the loop. This episode looks at why open weights, not APIs, are what makes onboard AI in orbit possible, and where satellite-class edge compute is already running smaller open models today.

Every model you've read about in this series so far assumes you can, in principle, ping a server. Space breaks that assumption on purpose. A satellite in Low Earth Orbit has a ground station window measured in minutes, a power budget measured in watts, and a round trip to mission control that can take hours. If your AI needs a datacenter, it doesn't fly. This is the one environment where "open-weight and runs locally" stops being a preference and becomes the only option on the table.

The proof point: Gemma 3 in orbit

On April 16, 2026, NASA's Jet Propulsion Laboratory ran what appears to be the first in-orbit demonstration of a vision-language model doing autonomous multi-modal inference entirely onboard a spacecraft. The system, called NAVI-Orbital, deployed a 4-bit quantized version of Google's open-weight Gemma 3 4B — the same checkpoint anyone can pull from Hugging Face — aboard a Loft Orbital YAM-9 satellite, with no fine-tuning performed before or during the mission, according to the arXiv paper describing the demonstration.

The numbers are the story here. The model hit 88.16% accuracy on a curated 7,960-image version of the AID aerial imagery benchmark. It ran on an Nvidia Jetson Orin AGX drawing power from a satellite bus generating 150 to 500 watts depending on orbital position, and the 4-bit quantized model needed only about 8GB of memory — small enough to fit that edge hardware without modification, as reported by TPS's coverage of the mission. Detection outputs — a predicted label, a free-text scene description, and metadata — occupy 700 to 1,060 bytes in JSON. The corresponding full-resolution captures from the YAM-9 imager run 9 to 14MB. That's roughly four orders of magnitude of compression, achieved not by shrinking the image but by replacing it with language.

TechCrunch's reporting quotes Loft Orbital's head of AI, Paul Lasserre, describing the implication plainly: "It opens the door to always-on, patrol layers in space... you can have logic — like 'monitor this border for me, and let me know when something is suspicious,' and interact back and forth with the satellites." The orchestration layer running underneath Gemma 3 is a LangGraph state machine coordinating separate detection and dialogue agents — the same open tooling you'd use to build an agent on the ground, just deployed 500 kilometers up.

It's not the only satellite running a model

NAVI-Orbital is the headline case, but it's part of a small, growing pattern rather than a one-off stunt. On the International Space Station, the IMAGIN-e payload has demonstrated onboard inference with compact variants of a Vision Transformer-based geospatial foundation model, purpose-compressed to fit resource-constrained flight hardware while preserving downstream task accuracy across five Earth-observation tasks, per a separate arXiv writeup of that ISS demonstration.

Separately, Syntiant and Novi Space demonstrated real-time, ultra-low-power AI object detection in orbit in March 2026, running quantized neural networks on an AMD Versal SoC with a dual-core ARM Cortex-A72 aboard Novi's SP240 onboard computer, according to SatNews's coverage. That mission reported the ability to retrain and hot-swap models in under 24 hours — switching a satellite's job from wildfire detection to maritime tracking without a hardware change. Different scale, different model family, same underlying principle: small, self-contained, retrainable-on-the-ground models beat anything that needs a live connection to a bigger brain.

Why open weights specifically, not just "small models"

You could, in theory, build a small proprietary model for this. What open weights buy you in orbit is control over the whole lifecycle, not just inference. A satellite is a closed system for months or years at a time; when something needs to change — a new benchmark, a new sensor calibration, a new mission objective — you can't wait on a vendor's API roadmap or eat a licensing renegotiation while the spacecraft is already flying. With an open-weight model like Gemma 3, JPL's team could quantize it to 4-bit, verify accuracy on their own curated benchmark, and load it onto flight hardware with full knowledge of exactly what's running and why it behaves the way it does. No vendor terms of service govern what a satellite is allowed to do with a locally-run, locally-owned model. That distinction matters more in orbit than almost anywhere else in this series, because there is no fallback path if a dependency changes mid-mission.

The use case: semantic compression as bandwidth strategy

The actual job being solved here isn't "AI in space" as a novelty — it's a specific, unglamorous bandwidth problem. Earth observation satellites generate more imagery than they can ever downlink. A ground station pass is short, contested across multiple payloads sharing the same bus, and every megabyte costs real money and real time. The traditional model is "acquire everything, downlink everything, let analysts sort it out on the ground" — which means most of what a satellite sees never gets looked at in any useful time frame.

NAVI-Orbital inverts that: the satellite classifies each scene onboard, writes a text description, and only flags the image itself for full downlink when the description warrants a closer look. An operator can send a plain-English prompt — "look for infrastructure near railway hubs" or "flag where natural terrain meets new development" — instead of writing a bespoke detection pipeline for each new question. That's the practical payoff of a general-purpose vision-language model over a narrow, single-task classifier: mission teams don't need a new custom-trained model every time the target changes. The trade-off is honest, too — this is a proof of concept on one satellite, one imaging instrument, and one benchmark. It's not yet a general autonomy platform, and nobody involved claims otherwise; JPL's team is explicit that this is a narrow demonstration, not a leap toward independently acting spacecraft.

Who should care about this pattern beyond aerospace: anyone building triage systems for expensive-to-move data — drone swarms, remote industrial sensors, offshore rigs, disaster-response imaging — where the constraint is bandwidth and connectivity, not compute. The lesson generalizes even if the hardware doesn't: run a small open model at the edge, downlink language instead of pixels, and only pull the expensive payload when the summary earns it.

Credits & sources

Technical details and benchmark figures on NAVI-Orbital are drawn from the arXiv paper on the in-orbit VLM demonstration. Additional reporting and quotes from Loft Orbital's Paul Lasserre come from TechCrunch's coverage of the mission. Hardware and model specifications are sourced from TPS's report on the deployment. The IMAGIN-e ISS geospatial foundation model demonstration is described in this companion arXiv paper, and the Syntiant/Novi Space low-power inference demonstration is covered by SatNews. Credit to the teams at NASA's Jet Propulsion Laboratory, Loft Orbital, Google DeepMind's Gemma team, Novi Space, and Syntiant for the actual engineering behind these missions.

Tomorrow, Day 19 comes back down to Earth for a look at Xiaomi's MiMo-V2.5 — one more open-weight model worth knowing about, no orbital hardware required.