DEV Community: Yong Yu

Muse Spark Open Is Out: Reasoning You Can't Turn Off, KV-Cache Math at 128K vs 1M, and Hosted vs Mac vs GPU

Yong Yu — Fri, 16 Oct 2026 21:08:26 +0000

Attributed compile + two small offline scripts

Primary sources (all 2026-10-15): Meta AI Research, Introducing Muse Spark Open; Hugging Face, Welcome Muse Spark Open by Pedro Cuenca; Baseten, Muse Spark Open, an open-weight model for coding agents by Marylise Tauzia; Meta developer docs: Get the model, Run inference, Prompting guide, Recommended settings; OpenRouter: meta/muse-spark-open.

I have not downloaded or served the weights (the HF repo is gated, and this box has no GPU). The scripts below are arithmetic on published architecture values and list prices.

Meta released the weights for Muse Spark Open: an 816B-total / ~42B-active mixture-of-experts model with a 1M-token context. It takes text, image, audio and video as input and returns text. Meta calls it "the most capable US open-weight model to date" and aims it at coding and agentic work. Hosted APIs went live on day 0 (Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter, Together AI), and weights are on Hugging Face in BF16 and NVFP4.

Benchmarks will get argued about all week. This post covers three things that will actually bite you on the first day: reasoning can't be switched off, the KV cache is much smaller than the parameter count suggests, and choosing between hosted, Mac and GPU.

The architecture numbers that matter

From the HF blog and Meta's docs:

Item	Value
Parameters	~816B total, ~42B active per token (Baseten: 42.7B)
Experts	256 per MoE layer, 8 routed per token
Decoder	62 layers, hidden 8192, 128 query heads, 16 KV heads, head dim 64
Attention schedule	46 sliding-window layers (2,048 tokens) + 16 full-attention layers (Meta settings doc; Baseten adds that the global layers use NoPE)
Context	`max_position_embeddings` = 1,048,576
Checkpoints	BF16 ~1.65 TB (Meta) / NVFP4 ~537 GB

Baseten's post also describes a compressed MoE block. A shared pooler shrinks the vector before it reaches the experts, and a shared unpooler expands it afterwards. This halves each expert, which Baseten says removes roughly 800B parameters compared with an uncompressed design.

Gotcha 1: reasoning is always on, and it eats `max_tokens`

From Meta's Run inference doc: the reasoning goes into reasoning_content and the answer goes into content, and "both are drawn from the same max_tokens budget." If the budget is too small, it all goes to reasoning and you get an empty content with finish_reason: "length".

Three more details from the docs:

The default is xhigh. The chat template maps low / medium / high / xhigh to strengths 8 / 32 / 256 / 512.
There is no off switch. "none" and "max" are rejected. If you want cheap calls, use low.
A system-prompt line wins. Reasoning strength: 256. in the system prompt overrides reasoning_effort. Streaming is recommended for high/xhigh so you don't hit timeouts.

A request that won't silently return nothing:

{
  "model": "meta-models/Muse-Spark-Open-816B-A42B",
  "messages": [{"role": "user", "content": "Find the race condition in this worker-pool design."}],
  "reasoning_effort": "high",
  "max_tokens": 16384,
  "stream": true
}

runs/20-muse-spark-open/budget.py simulates the shared budget. The reasoning lengths in it are hypothetical, chosen only to show the failure modes, not measured:

max_tokens=  1024 reasoning=3000 answer=400: EMPTY content, finish_reason=length
max_tokens=  4096 reasoning=3000 answer=400: ok
max_tokens=  4096 reasoning=3800 answer=400: TRUNCATED answer
max_tokens= 16384 reasoning=3000 answer=400: ok

In practice: set reasoning_effort explicitly on every call. Treat an empty content with length as a budget bug, not a model failure, and log completion_tokens_details.reasoning_tokens (Meta's reasoning doc says reasoning tokens are billed at the output rate).

Gotcha 2 (a good one): the KV cache is small for an 816B model

Each token stores K and V for 16 heads × 64 dims per layer: 4,096 bytes per layer in BF16. The HF blog multiplies that across all 62 layers and gets ~250 KB/token, or ~266 GB for a 1M context. That is the upper bound. Meta's settings doc says only the 16 full-attention layers keep the whole context. The 46 sliding-window layers are capped at 2,048 tokens.

runs/20-muse-spark-open/kv_cache.py computes both, per sequence:

LAYERS, LOCAL, GLOBAL, WINDOW = 62, 46, 16, 2048
KV_HEADS, HEAD_DIM = 16, 64

def kv_bytes(tokens, bpe, hybrid=True):
    p = 2 * KV_HEADS * HEAD_DIM * bpe          # K+V per layer per token
    if not hybrid:
        return LAYERS * tokens * p
    return GLOBAL * tokens * p + LOCAL * min(tokens, WINDOW) * p

Actual output (Python 3.13.5, run 2026-10-16):

   context dtype  all-global GiB  hybrid GiB  ratio
    32,768  BF16            7.75        2.36   3.28x
   131,072  BF16           31.00        8.36   3.71x
   131,072   FP8           15.50        4.18   3.71x
   262,144  BF16           62.00       16.36   3.79x
 1,048,576  BF16          248.00       64.36   3.85x
 1,048,576   FP8          124.00       32.18   3.85x

The 128K row matches Baseten's published 31 GiB → 8.4 GiB (~3.7×) exactly. A full 1M context costs about 64 GiB in BF16 (about 32 GiB if your runtime stores KV in FP8), not 248 GiB. With 100 GiB free after weights, that's about 11 sessions at 128K and one at 1M, before paging overhead. This is why Meta recommends setting --max-model-len / --context-length to what your workload needs instead of the full window.

Hosted vs Mac vs GPU

Hosted (start here). OpenRouter lists $1.25 / $4.25 per 1M input/output tokens, $0.15/M cache reads, five providers, a 1,048,576-token context and up to 262,144 completion tokens. Baseten lists the same prices and serves NVFP4 with the full 1M context. My budget.py arithmetic at list price: a 60K-in / 4K-out agent turn costs $0.092. With 50K of that served from cache it's $0.037, so 1,000 such turns come to about $37. Remember that the 4K output includes reasoning.

from openai import OpenAI
import os
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"])
r = client.chat.completions.create(
    model="meta/muse-spark-open",
    messages=[{"role": "user", "content": "Review: def div(a, b): return a / b"}],
    reasoning_effort="low", max_tokens=4096)
print(r.choices[0].finish_reason, r.choices[0].message.content)

(Not run here. I don't have an OpenRouter key on the box.)

Mac (for tinkering, not serving). HF tested it on an M3 Ultra with 512 GB:

llama.cpp Q2_K: ~304 GB, ~18 tok/s
oMLX oQ3e: ~344 GB, ~225 tok/s prefill and ~21 tok/s generation

Even the 3-bit quant needs more than 256 GB of RAM. The model card also says it's optimized for two Mac Studios with M5 Ultra using MLX. Fine for private experiments. Too slow for agent loops at xhigh.

GPU (when you need control or volume).

NVFP4: Meta's recipe uses TP4/EP4 on 4× B300. SGLang's reference runs it on 4× GB300.
BF16: TP8/EP8 on 8× B300.
HF Inference Endpoints has a preset recipe on 8× RTX Pro 6000 (128K context) or 8× H200 (1M).

NVFP4 needs Blackwell. Size the context to the KV table above, not to the marketing number.

My rule: start hosted, at low or medium with a large max_tokens. Move to dedicated GPUs only when you need data residency, fine-tuning (HF shows LoRA via TRL on 64× H100) or steady volume that beats the per-token price. Use the Mac when you want to poke at it offline.

Why this matters

The useful lesson here goes beyond this one model. A hybrid local/global attention schedule makes long context about 3.7–3.9× cheaper in memory, and an always-on reasoning budget turns max_tokens into a correctness setting. Both are worth checking in your own harness before you look at any leaderboard.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Grok Voice 3.0 Just Landed: First Realtime Session, Full Duplex, and Why $0.08/min Beats Guessing Latency

Yong Yu — Thu, 15 Oct 2026 13:19:32 +0000

Attributed compile + one small offline smoke test

Primary sources (xAI / SpaceXSI, ~2026-10-14/15): Introducing Grok Voice 3.0 (announcement), Speech to Speech docs, Grok Voice 3.0 model page, and Release notes (October: Voice 3.0 available; Voice 2.0 deprecated)

Benchmark rows below are Artificial Analysis (Speech-to-Speech Index, dated October 13, 2026 on the announcement), xAI product claims, and ServiceNow EVA-Bench (Nitin Aggarwal quote on the announcement). No XAI_API_KEY on this box — I did not open a live WebSocket. The run below is an offline model-id / session.update / mock-event linter only.

xAI shipped Grok Voice 3.0 into the Voice Agent API at the same list price as Voice 2.0: $0.08 / min ($4.80 / hr). If your agents already use the grok-voice-latest alias, the announcement says they were switched to 3.0 automatically — same endpoint, same session fields, same events.

The product pitch is speech-to-speech for support-style agents: under-one-second class latency (xAI), Full Duplex Bench at 97% (Artificial Analysis), and Tesla / Starlink production numbers on the announcement. This post is the practical first session: WebSocket URL, a clean session.update, duplex/server_vad, tools, cost arithmetic at list, and what the alias auto-upgrade means for pinned clients.

What the announcement actually claims

Artificial Analysis numbers as printed on the Grok Voice 3.0 post (source line: Artificial Analysis, October 13, 2026):

Benchmark	Grok Voice 3.0	Grok Voice 2.0	GPT-Live-1 (Astra, Medium)	Gemini 3.8 Live (Ext. Thinking, High)
Overall AA Speech-to-Speech Index	86.9 (#1)	81.3	83.2	82.6
Speech Reasoning Big Bench Audio	98.5%	97.2%	90.1%	97.7%
Human Preference Speech Agent Arena Elo	1070	1011	1048	990
Task Success Rate	94.8%	94.6%	87.4%	89.1%
Full Duplex Bench	97.0%	95.1%	94.9%	91.9%

τ-Voice (support calls with accents, noise, interruptions): 71.6%, described as comparable to GPT-Live-1 (Astra) at roughly 40% lower latency. On ServiceNow's EVA-Bench, the announcement cites 74% accuracy and a Pareto frontier vs accuracy + customer experience; Nitin Aggarwal (ServiceNow) is quoted there saying 3.0 pairs top accuracy with a natural, responsive conversation.

Production color from the same post (vendor numbers, not mine): millions of conversations a day across Tesla and SpaceX; Starlink ~20,000 calls/day, roughly 70% no human, 1 in 4 sales calls ending in a purchase. Tesla framing splits Connectors (calendar / email hands-free) from Grok Bot (bigger tasks). Anecdotes, not a public SLA. I did not reproduce any benches.

First realtime session (WebSocket)

Docs endpoint:

wss://api.x.ai/v1/realtime?model=grok-voice-3.0

Auth: Bearer XAI_API_KEY on the server, or an ephemeral token (xai-client-secret.…) for browsers. Model query params that matter:

Model id	Meaning
`grok-voice-latest`	Alias → currently 3.0
`grok-voice-3.0`	Pin this release
`grok-voice-2.0`	Deprecated (same price; upgrade to 3.0 / latest)

A minimal session.update after connect (shape from the Speech to Speech docs):

{
  "type": "session.update",
  "session": {
    "voice": "eve",
    "instructions": "You are a concise support agent. Ask before issuing credits.",
    "turn_detection": { "type": "server_vad" },
    "tools": [
      { "type": "web_search" },
      {
        "type": "function",
        "name": "issue_credit",
        "description": "Issue a billing credit",
        "parameters": {
          "type": "object",
          "properties": { "amount_usd": { "type": "number" } },
          "required": ["amount_usd"]
        }
      }
    ],
    "reasoning": { "effort": "high" },
    "replace": { "Acme Mobile": "Acme Mobull" }
  }
}

With turn_detection.type = server_vad, stream mic audio via input_audio_buffer.append (or binary if audio.input.transport is binary) and let the server handle turn boundaries — the duplex / barge-in path. Without VAD you commit/clear yourself.

Untested live sketch (websockets + a real key; not run against the API):

import asyncio, json, os
import websockets  # untested live against api.x.ai

async def voice_agent():
    url = "wss://api.x.ai/v1/realtime?model=grok-voice-3.0"
    headers = {"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}
    async with websockets.connect(url, additional_headers=headers) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "eve",
                "instructions": "You are a helpful assistant.",
                "turn_detection": {"type": "server_vad"},
            },
        }))
        # ... append audio / conversation.item.create / response.create ...
        async for raw in ws:
            event = json.loads(raw)
            print(event.get("type"))

# asyncio.run(voice_agent())  # do not run without a key and audio path

Same-session tools: web_search, x_search, file_search, mcp, and client function tools. Server-side tools run on SpaceXSI; only custom functions need your conversation.item.create + response.create loop. Pronunciation replace fixes spoken audio without changing the visible transcript.

Cost arithmetic (list price, not a bill)

Announcement list: $0.08 / min = $4.80 / hr, unchanged from Voice 2.0.

Illustrative only:

1,000 support calls × 3 minutes = 3,000 minutes → $240 at list.
Starlink-scale illustration using the announcement's ~20k calls/day: at 2 / 4 / 6 minute averages → $3,200 / $6,400 / $9,600 per day at list.

Arithmetic on published list rates and call volume — not an invoice. If duplex cuts average handle time by a minute, list saving is $0.08 per call before quality metrics.

What I actually ran (offline)

No xAI key on the box, so runs/19-grok-voice-3/preflight.py only checks shapes:

grok-voice-3.0 / grok-voice-latest → OK; grok-voice-2.0 → deprecated warning; prompt-shaped and fake ids → rejected.
Clean session.update (voice eve, server_vad, web/x/function tools, replace, reasoning effort, json transport) → OK.
Missing voice / bad turn_detection / unknown tool type → flagged.
List-price math prints the $240 and Starlink-scale day figures above.
Hand-built mock stream (session.updated → response.output_audio.delta → response.done) parses; missing delta and unknown event types raise ValueError.

Smoke test of your client contract, not of Grok Voice 3.0. Full stdout: run_output.txt (2026-10-15, Python 3.13, exit 0).

Alias auto-upgrade vs pinning

Release notes and the Speech to Speech upgrade section agree: grok-voice-latest already points at 3.0, Voice 2.0 is deprecated, API surface unchanged, price unchanged. Two practical implications:

If you used the alias, you are already on 3.0 — verify behavior (reasoning default is "high" on 3.0 per the migration notes) rather than assuming a silent quality-only bump.
If you pinned grok-voice-2.0, change the query param to grok-voice-3.0 (or latest). Deprecation is not the same as free forever; pin deliberately for prod, track release notes for the next alias flip.

The Tesla product split is a useful design hint: Connectors-style tools for short reversible actions (calendar, email) vs Grok Bot-style loops for multi-step work with approvals. Same voice model, different authority.

How I'd try it this week

Pin grok-voice-3.0 in a staging WebSocket and send one session.update with server_vad + a single function tool you already trust.
Measure barge-in and tool overlap audiowise — docs warn that sending response.create before prior audio finishes causes overlap; wait for playback (or nearly) after function outputs.
Budget at $0.08/min with a hard max session length before you chase Elo screenshots.
Keep irreversible tools behind confirmation — credits, shipments, and outbound messages are still your harness's job.

Why this matters

Speech-to-speech rankings move weekly. What is copyable here is boring and useful: a stable realtime URL, unchanged session schema, an AA Full Duplex number, flat per-minute list price, and an alias that already moved. Build the duplex + tool harness once; swap pins when the next index drops.

I have not called the Voice Agent API. Start with a keyed ping and one scripted support turn with approvals on.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Coinbase's 9B Fraud Model Beat Opus 4.5: Copy the Pattern With a Verifiable Reward, Tier-Only Output, and Block-Only Harness

Yong Yu — Wed, 14 Oct 2026 13:27:20 +0000

Attributed compile + one small offline run on synthetic data

Primary sources (Coinbase Engineering, "Owning Intelligence" series): Part 1/3 — Build by Yao Ma, Bo Lei, Xuwei Tan (2026-10-06), Part 2/3 — Evaluate by Yao Ma, Xuwei Tan (2026-10-07), and Part 3/3 — Own by Yao Ma, Xuwei Tan, Akshit Trehan, Aman Choudhary with the Anyscale team (Eric Tang, Sumanth Hegde, Seiji Eicher, Jess Kong) (2026-10-08)

Also: Brian Armstrong on X (2026-10-11); secondary coverage by Dealroom, Crypto Briefing (Diego Almada Lopez), CoinCentral, and HDATF AX case 137

Every benchmark, latency and cost number about Coinbase's system is Coinbase's, measured on their private data. I did not train anything. The run below is a tiny eval harness on a synthetic, seeded transaction set.

The headline that went around X this week: Coinbase post-trained open-weight Qwen3.5-9B on its own Onramp fraud data, and it beat Claude Opus 4.5 on Coinbase's fraud benchmark. Brian Armstrong's take was that specialist models will "out perform on more narrow tasks."

The model swap is the least copyable part. The useful part is the system around it. That's what this post covers, plus a harness you can run on a laptop in under a second (the optional small-model reviewer takes a few minutes on CPU).

What Coinbase actually built

From Part 1, in my own words:

Layered, not replaced. Onramp sends a transaction to a Risk Service. The existing ML model and rules engine screen it first. The LLM agent sits downstream and reviews only selected transactions, using recent transaction history plus written review guidance and calibration examples.
Tier-only output. The model returns a risk level (their example is RISK_LEVEL: 5) with no explanation. With a ~7,000-token prompt, Opus had a P50 of about 1.5 s.
The harness decides. Application code builds the context, parses the tier, and applies the decision policy. The model never executes a payment action.
Block-only authority. The agent can block something upstream allowed. It cannot override an upstream block.

In a 50% traffic A/B test, the agent-enabled arm had 30% fewer fraudulent transactions and 22% less fraud value. Coinbase estimates fraud savings at roughly 3× the inference cost.

Part 2 built a fixed benchmark: 16,140 transactions from 7,293 users, 813 confirmed fraud, taken from nine weeks before the agent rolled out, so the agent's own blocks couldn't bias which outcomes got observed. Newer Opus, Sonnet and GPT versions each scored lower on recall, F1 and dollar-weighted recall than the version before. GPT-5.6 (sol) gained 11.5 points of precision but lost 20.7 points of recall and 21.8 of dollar-weighted recall.

Part 3 is the post-train: RLVR with GRPO and LoRA on SkyRL (on Anyscale), run on one RTX PRO 6000 96GB for under $100. Against Opus 4.5 as a single-pass decision agent: +7.5 precision, +12.0 recall, +9.6 F1, +35.4 dollar-weighted recall. The figure caption says these are percentage-point differences. Serving uses Ray Serve LLM + vLLM on L40S GPUs (2–8 replicas). P50 was 0.683 s vs 1.515 s, P99 1.036 s vs 3.312 s. Coinbase notes that latency and detection were measured in separate evaluations. They also say the unadapted Qwen base "performed poorly."

Why the reward is "verifiable" here

Most RL-for-LLM setups need a judge model to say whether an answer is good. Fraud doesn't. Historical transactions eventually get an outcome: chargeback or confirmed fraud, or not. So the reward is just a deterministic function of (model output, known label). Nothing gets graded on how persuasive it sounds.

Part 3 adds two details worth copying:

Format checks. Unparseable output gets penalised.
Class balance. If fraud is rare, "always say low risk" looks accurate. Their reward weights fraud and legit examples so a constant answer can't farm reward.

GRPO then samples a group of answers to the same prompt and pushes the model toward the ones that scored above the group mean. You don't need a value network or a judge.

Why dollar-weighted recall, not just F1

F1 counts incidents. One missed $5,000 cash-out counts the same as one missed $12 card test. Dollar-weighted recall asks what share of the fraud value you caught. Part 2's point is that a model can miss relatively few transactions and still miss a large share of the money.

The run: a laptop-sized version of the pattern

I wrote fraud_tier_eval.py, about 200 lines of plain Python, plus an optional small-model reviewer. It runs on a synthetic set (seed 18): 600 transactions, 47 labelled fraud worth $68,484.50 in total. I wrote the generator with three fraud shapes: card testing (many small buys), cash-out (one large buy, fresh card, new wallet), and quiet (mid-size, geo mismatch only). I designed the shapes, so the results show how the metrics behave. They don't show which reviewer is good.

The flow mirrors the pattern. Upstream rules block first. Routing picks 579/600 for review. Each reviewer returns only a string like RISK_LEVEL: 4. The harness parses it with a strict regex and blocks at tier ≥ 4:

def decide(upstream, reviewer_text):
    if upstream == "BLOCK":
        return "BLOCK", "upstream_block_kept"   # reviewer can never release
    if reviewer_text is None:
        return "ALLOW", "not_routed"
    tier = parse_tier(reviewer_text)            # ^RISK_LEVEL: ([1-5])$
    if tier is None:
        return upstream, "parse_error"          # log it, keep upstream decision
    return ("BLOCK", "reviewer_block") if tier >= 4 else ("ALLOW", "reviewer_allow")

The reward function is the part a GRPO environment would call:

w_f, w_l = n / (2 * n_fraud), n / (2 * n_legit)
def reward(text, is_fraud):
    tier = parse_tier(text)
    if tier is None:
        return -1.0
    correct = (tier >= 4) == is_fraud
    return (1.0 if correct else -1.0) * (w_f if is_fraud else w_l)

Printed results (2026-10-14, 8-vCPU Xeon, Python 3.13, CPU only):

Reviewer	P	R	F1	$-recall	Legit $ blocked
upstream rules only	1.000	0.319	0.484	0.598	0.00
velocity (counts recent buys)	0.714	0.638	0.674	0.603	846.55
context (new wallet/card, amount vs history)	0.379	0.702	0.493	0.979	49,411.22
d1-omni-600M, unadapted, CPU	0.455	0.319	0.375	0.598	12,521.50

What the run showed:

F1 and dollar-weighted recall picked different winners. The velocity reviewer caught 24/29 card tests and none of the cash-outs that upstream let through: $-recall 0.603 vs 0.598 for upstream alone. The context reviewer caught 12/12 cash-outs ($-recall 0.979) but blocked $49,411.22 of legit purchases. Neither metric alone is enough. Track precision and legit-dollars-blocked next to them.
A generic small model added nothing here. LiquidAI's d1-omni-600M, prompted with a plain-English description of each transaction, added 18 blocks and every one was a false positive. Its recall stayed at the upstream 0.319. Median review time was 193.6 ms. That's one model on toy data, but it fits Coinbase's note that the unadapted base was weak and adaptation did the work.
The class-balanced reward caught the lazy policy. "Always RISK_LEVEL: 1" had 0.922 plain accuracy and a mean reward of −0.000. The d1 reviewer scored −0.033.
The guard held, but it limits direction, not damage. A deliberately buggy reviewer emitted RELEASE, RISK_LEVEL: 9 and prose. That caused 344 parse errors, and every upstream block stayed blocked (asserted). It still added 111 random blocks, 104 of them legit. Block-only authority stops wrongful releases. It doesn't stop a broken model from blocking good customers, so alert on block rate too.
GRPO-style group demo: for one $4,733.38 cash-out, RISK_LEVEL: 5 and RISK_LEVEL: 4 scored +6.383 (advantage +0.935) and RISK_LEVEL: 2 scored −6.383. A correct tier with an explanation appended scored −1.0, so the format contract is part of the reward.

Not done: no fine-tuning, no GRPO training run, no Qwen3.5-9B, no Coinbase data, no latency comparison with any frontier model. The set is tiny and synthetic. Files are in runs/18-coinbase-specialist/ (fraud_tier_eval.py, run_output.txt).

When not to copy this

No trustworthy labels. RLVR is only as good as the outcome label. If fraud confirmation is slow, noisy or missing, the reward becomes guesswork. Coinbase notes labels arrive late even for them.
Low volume. Coinbase's benchmark alone had 813 confirmed fraud cases. If you have dozens, a prompted frontier model with a fixed eval set is cheaper to run and to reason about. That's my rule of thumb, not a Coinbase threshold.
Policy that shifts weekly. Part 1 points out that you can change natural-language guidance without retraining. Baked-in weights can't do that. If the rules move faster than you can retrain and re-evaluate, keep the policy in the prompt and the harness.
No owner for the lifecycle. Part 3's own checklist is to evaluate on domain outcomes, version weights, prompts and policy together, and keep a tested rollback. If no one owns those, don't self-host a model that sits in a payment flow.

How I'd start this week

Freeze a labelled eval set from a period before any model touched decisions.
Make the model return one parseable token. Put the policy in code.
Score every candidate on precision, recall, F1, dollar-weighted recall and legit-dollars-blocked.
Only then consider RL. The reward function you need is already in your eval harness.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Muse Wonder Just Landed: First muse-wonder-1.0 Call, Active Memory vs Active Focus, and Why max Isn't Wonder Full

Yong Yu — Tue, 13 Oct 2026 13:30:14 +0000

Attributed compile + one small offline smoke test

Primary sources (Meta, 2026-10-12): Muse Wonder: A Frontier Model For Personal & Professional AI Agents (AI at Meta research blog), Put your ideas to work with Muse Wonder and Muse Wonder Full by Josh Walters (Meta), Muse Wonder Model Card, and the Meta Model API docs (overview, models, chat completions quickstart)

Secondary: Indian Express explainers (2026-10-13)

Benchmark and safety numbers are Meta's (or third-party numbers they cite). No MODEL_API_KEY on this box — I did not call Muse Wonder. The run below is an offline request/response linter on hand-built mocks only.

Meta shipped Muse Wonder and Muse Wonder Full on 2026-10-12, into a feed already full of coding-agent head-to-heads. The pitch is not "another chat model." It is a frontier stack for personal and professional agents: keep context across weeks (Active Memory), keep working through blockers (Active Focus), and survive long coding / computer-use jobs past turn three.

Access is split on purpose. Muse Wonder is on the Meta Model API and Muse Code at $4 / $20 per million tokens. Muse Wonder Full is Muse Code subscription only (Power and new Maximum Usage). That split is the catch when people paste "reasoning_effort": "max" and expect Full.

What Meta actually shipped

From the research post and model card:

	Muse Wonder	Muse Wonder Full
Where	Model API + Muse Code	Muse Code subscription only
Model API id	`muse-wonder-1.0`	(not on Model API)
Context	1,048,576 tokens	1M (same family)
Modalities in	text, image, video, audio	same family
Modalities out	text	text
List price (API)	$4 in / $0.20 cached in / $20 out per 1M	included in Power / Maximum Usage

Docs say Wonder and Spark take the same request shapes. Any Spark example becomes a Wonder call by changing the model string to muse-wonder-1.0. Wonder is for longer, harder agent loops; Spark stays the cheaper default (and Contributor mode on Model API).

Meta's product language for the two agent knobs:

Active Memory — carry what matters about you across conversations, and retrieve the bits that are relevant to the current task (not "dump the whole history into the prompt and hope").
Active Focus — keep pushing on a hard goal: try alternatives, work through blockers, ask when stuck, check in before consequential actions.

Coding is where they claim those two meet: multi-file edits, long test/fix cycles, sub-agents, computer use. Treat DeepSWE / SWE-Marathon / PSI / Harvey LAB-AA rows as vendor or cited third-party scores. I did not reproduce them.

First call (Chat Completions)

Base URL is https://api.meta.ai/v1. Auth is a Bearer MODEL_API_KEY. OpenAI SDK drop-in:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.meta.ai/v1",
    api_key=os.environ["MODEL_API_KEY"],
)

resp = client.chat.completions.create(
    model="muse-wonder-1.0",
    messages=[
        {"role": "developer", "content": "Be concise. Ask before irreversible actions."},
        {
            "role": "user",
            "content": (
                "I am wiring a personal agent that must remember lease clauses "
                "across chats. In five bullets, what should go in durable memory "
                "vs what should stay in the current turn only?"
            ),
        },
    ],
)
print(resp.choices[0].message.content)
print(resp.usage)

Or curl:

curl -X POST "https://api.meta.ai/v1/chat/completions" \
  -H "Authorization: Bearer $MODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "muse-wonder-1.0",
    "messages": [
      {"role": "user", "content": "Reply with the single word: ready"}
    ]
  }'

Three things the docs make easy to miss:

Prefer developer over system for steering. system still shows up in old OpenAI snippets; Meta's quickstart steers with developer.
Do not send stop, logit_bias, or reasoning_effort: "none". Reasoning Muse models return HTTP 400 on those.
max on Wonder ≠ Wonder Full. Model card and models page say Full is subscription-only in Muse Code; cranking reasoning effort on muse-wonder-1.0 does not promote you.

For serious agent loops (tools, search grounding, file IDs, server-managed state), Meta points you at the Responses API rather than raw Chat Completions. Chat Completions is still the right first ping and the path for simple multi-turn arrays you already have.

Cost arithmetic (list price, not a bill)

Josh Walters' post lists Wonder at $4 input / $20 output per million tokens, with $0.20 cached input on the pricing card.

Illustrative only: 50,000 calls × 4,000 prompt tokens × 800 completion tokens ≈ $800 input + $800 output = $1,600 at list (ignores retries, tools, browsing, Muse Code plan floors). Same call with 3,500 of those 4,000 prompt tokens cached drops one-call list cost from about $0.032 to about $0.019. That is why prompt caching matters once your agent system prompt and tool schemas stop changing every request.

Wonder Full's economics are different: Muse Code Power / Maximum Usage quotas, not a public per-token meter. Launch-day quota bugs got a Meta reset — check your usage bar before a weekend marathon.

What I actually ran (offline)

No Meta key on the box, so I wrote a small preflight that only checks shapes:

Clean muse-wonder-1.0 Chat Completions body → OK; Spark model id → OK (swap-compatible).
Fake muse-wonder-full-1.0 + reasoning_effort: "max" → flagged as not on Model API.
Legacy prompt (no messages) → rejected; system + stop + reasoning_effort: "none" → doc warnings / 400s.
Hand-built mock choices[0].message.content parses; completions-v1-shaped mock raises ValueError.
The $1,600 / $0.032 / $0.019 figures above are the same list-price arithmetic the linter prints.

Smoke test of your client, not of Muse Wonder. Scripts: runs/17-muse-wonder/ (preflight.py, run_output.txt).

The Full / API catch (and the memory catch)

Two product traps:

1. Full is not an API model id. If your eval harness hard-codes "use max reasoning to get the best Muse," you are still on Wonder for API traffic. For Full you need Muse Code on Power or Maximum Usage. Compare apples to apples when you paste Twitter win rates.

2. Active Memory is not free durability. The research framing is "carry important context across conversations and retrieve what matters." In your agent, that still needs a store, a write policy, and a retrieve policy. Wonder does not magically make a stateless Chat Completions call remember last month's lease PDF. Put durable facts in a store you control; use the model to decide what to write and what to fetch. Meta's own card recommends human-in-the-loop confirmation for irreversible agent actions — that is the right default even with Active Focus.

Also noted on the card: visual / 3D spatial work is a known weakness; harness and quantization change results; not intended for under-18 use. Read Meta's safety section before giving an agent real credentials.

How I'd try it this week

Ping Model API with the curl above — confirm key, model id, and usage before rebuilding a harness.
Port one Spark agent loop by changing only the model string; keep tools identical for a fair A/B.
Separate memory writes from the chat transcript. Log what the agent decided to remember.
Gate irreversible tools. Active Focus keeps trying — good for long SWE tasks, bad for rm, payments, and outbound email.

Why this matters

This month's frontier noise is loud. Muse Wonder is Meta's clearest "personal agent" bet: same family behind a consumer Muse app and a developer API. For builders the useful part is boring — OpenAI-compatible base URL, stable model id, documented Full/API split, and two named behaviors (memory + focus) you can design around after you measure them yourself.

I have not run Wonder live. Start with the ping, then one real repo task with approvals on.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Microsoft-Decision-1 Just Landed: First systemone Call, Free Output Tokens, and the Qwen-to-MAI Rebase Catch

Yong Yu — Mon, 12 Oct 2026 13:43:30 +0000

Attributed compile + one small offline smoke test

Primary sources (Microsoft, 2026-10-09): Microsoft-Decision-1: Our model for fast decision-making by Achint Srivastava (Command Line / Microsoft), Deploy and use Microsoft-Decision-1 in Microsoft Foundry (Microsoft Learn), and the OpenRouter model page for microsoft/microsoft-decision-1

Secondary: Microsoft launches a decision model built on Alibaba’s Qwen by The Next Web (2026-10-12)

Accuracy, latency multiples, flip rates, and internal Xbox/Copilot anecdotes are Microsoft's. I have no Foundry or OpenRouter key on this box, so I did not call Microsoft-Decision-1. What I ran is an offline request/response linter against hand-built mocks, described below as exactly that.

Microsoft shipped Microsoft-Decision-1 into Foundry and OpenRouter on 2026-10-09, three days after Liquid open-weighted d1. Satya Nadella's launch post on X cleared a million impressions within hours. Same category shift: stop asking a chat model for JSON and ask a decision model for typed probabilities.

The access difference matters. Liquid gave you weights. Microsoft gave you a hosted systemone endpoint, $0.042 per million input tokens with output free, and a Qwen3.5-9B base with an explicit plan to rebase onto MAI and OpenAI later. Same API shape either way. That rebase note is the catch.

What it actually returns

From Microsoft Learn and the OpenRouter page, a request is not a chat completion. You send a state (string or JSON) plus named questions. Each question is one of three types:

type	you give	you get back
`noul`	a yes/no condition	`P(yes)` in `[0, 1]`
`choice`	named options with descriptions	selected option + per-option probabilities
`score`	an ordered rubric (2+ levels)	probability-weighted level index

There is no free-form text and no written rationale. That is the point. Your code thresholds the probability, routes the ticket, or blocks the tool call. Learn's endpoint path on Foundry is:

<your-foundry-resource-endpoint>/providers/microsoft/v1/systemone

The request's model field is your deployment name, not the string Microsoft-Decision-1. On OpenRouter the model id is microsoft/microsoft-decision-1, and OpenRouter documents that chat-completions SDKs will not work — you need the Decisions / System One surface.

What Microsoft is claiming

From Srivastava's post, labeled as vendor numbers:

Highest accuracy in Microsoft's 36-benchmark comparison (~150k questions kept blind from training).
~35× faster (P50) than GPT-6 Sol in their latency test; 2.5× faster than H2O-Lightning-4B v1.1.
Under eight input perturbations, decisions flipped 1.3% of the time on average; zero flips when option descriptions were paraphrased or options were reversed/shuffled.
Xbox Research: competitive with GPT-6 Sol on labeling 10k+ feedback items, >14× faster and ~200× cheaper (Microsoft's framing).
Copilot quality scoring: competitive with GPT-5.6 Luna and ~100× faster (Microsoft's framing).

I did not reproduce any of that. Treat it as a launch claim sheet, then measure on your own labeled tickets.

First request shape (Foundry-style)

A single call can ask three questions over the same state. Pattern from Learn + OpenRouter:

import json, os, urllib.request

body = {
    "model": os.environ["DEPLOYMENT_NAME"],  # your Foundry deployment name
    "state": "Help! My payouts have been failing for 3 days.",
    "questions": {
        "is_urgent": {
            "type": "noul",
            "instructions": "Does this message convey urgency?",
            "criteria": {
                "true": "Explicitly time-sensitive",
                "false": "No urgency expressed",
            },
        },
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this?",
            "criteria": {
                "billing": "Payments, invoicing, refunds",
                "technical": "Bugs, outages, integrations",
                "sales": "Pricing, upgrades, new accounts",
                "cannot_tell": "Not enough information, or not a support request",
            },
        },
        "frustration": {
            "type": "score",
            "instructions": "How frustrated is the customer?",
            "criteria": ["Calm", "Frustrated", "Very angry"],
        },
    },
}

req = urllib.request.Request(
    os.environ["AZURE_ENDPOINT"].rstrip("/") + "/providers/microsoft/v1/systemone",
    data=json.dumps(body).encode(),
    headers={
        "Content-Type": "application/json",
        "api-key": os.environ["AZURE_API_KEY"],
    },
    method="POST",
)
# with urllib.request.urlopen(req) as resp: print(resp.read().decode())

Three things before production:

cannot_tell is deliberate. A choice always picks one option. Without an abstention label, gibberish still gets a team. Learn says to include abstention when the model should not be forced.
Option descriptions are the policy. "Payments, invoicing, refunds" does not cover "is this phishing email from you?". Write phishing into the criteria if it belongs in fraud/security.
Auth is Entra ID or api-key. Learn prefers DefaultAzureCredential; the snippet uses an API key because it is shorter to read.

Cost arithmetic (list price, not a bill)

Microsoft and OpenRouter both list $0.042 / 1M input and $0 / 1M output. Output is free because there are no completion tokens in the chat sense — you get structured answers.

Illustrative only: 100,000 requests × 2,000 billable input tokens ≈ $8.40 of model input (ignores retries, retrieval, review, platform fees). That is why decision models look attractive for the boring 80% of routing.

OpenRouter lists a 32,768-token context, text in, structured decision out. No images or audio here (unlike Liquid's multimodal d1-omni).

What I actually ran (offline)

No Azure and no OpenRouter key on the box, so I wrote a small preflight that only checks shapes:

A legacy chat request (messages + temperature=0) is flagged as the wrong API.
A multi-question Foundry body with noul / choice / score lints clean.
A two-option choice without an abstention key gets a warning.
A hand-built mock answers object parses; a chat-completions-shaped mock raises ValueError: missing answers object.
The $8.40 figure above is the same list-price arithmetic the linter prints.

That is a smoke test of your client, not of Microsoft-Decision-1. Scripts live under runs/16-microsoft-decision-1/ (preflight.py, run_output.txt).

The Qwen-to-MAI rebase catch

Microsoft says Decision-1 was post-trained from Qwen3.5-9B, and that it will soon rebase onto other bases including MAI and OpenAI. The API shape is supposed to stay put while the weights under it change.

Practical consequences:

Do not bake "this is a Qwen model" into monitoring, eval harnesses, or compliance docs as a permanent fact. Pin the deployment / model version you evaluated.
Re-run your labeled set after any silent base swap. Calibration and flip rate are product claims; they are not a contract that survives a rebase without measurement.
If you needed open weights or air-gapped inference, this product is the wrong shelf. Liquid d1 (or another open decision model) is the local path; Decision-1 is the hosted path inside Foundry/OpenRouter.

How I'd wire it for an agent

Gate tool calls with noul. "Does this tool argument touch production data?" — auto-allow only above a high threshold; otherwise ask a human or a stronger model.
Route with choice + abstention. Escalate when cannot_tell wins or when top probability is below your threshold (start around 0.7 and tune).
Grade agent steps with score. Rubric the proposed next action before you execute it. Microsoft's Discovery anecdote is exactly this pattern.
Keep Decision-1 off consequential people decisions. Microsoft's own card and Learn page say not to use it as the sole basis for employment, credit, healthcare, and similar.
Log state, criteria, answers, and the action you took. When something routes wrong, you need to know whether the bug was missing evidence, a bad option description, or your threshold.

Why this matters

Agent stacks are full of "call an LLM, beg for JSON, parse it." A hosted decision endpoint that returns calibrated probabilities, charges nothing for output, and already sits next to Foundry deployments is a concrete alternative for routing, gating, and grading. It does not replace a generative model for writing. It stops you from using one for a multiple-choice question.

Keep Liquid's open d1 for edge and offline. Use Microsoft-Decision-1 when you already live in Azure or OpenRouter. Measure both on the same labeled set before you trust either.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Liquid AI's d1 Decision Models Went Open: Triage Support Tickets on a CPU With the 600M One (and Where It Fools You)

Yong Yu — Fri, 09 Oct 2026 13:45:49 +0000

Attributed compile + one small real CPU run

Primary sources (Liquid AI, 2026-10-07): Open d1: Edge decision models for text, vision, and audio (Liquid AI blog), Multimodal open d1 decision models for the edge (Hugging Face blog), and the model cards for d1-3B and d1-omni-600M

Background: Introducing d1: The most capable decision model, now with vision (Liquid AI, 2026-10-05)

All benchmark and latency figures for d1-3B are Liquid AI's. The numbers I report for d1-omni-600M come from one run on a shared 8-vCPU Linux box with 12 tickets I wrote myself. That is a smoke test, not an evaluation.

This week Liquid AI put two of its d1 "decision models" on Hugging Face as open weights: d1-3B (text + images) and an experimental d1-omni-600M (text + images, or text + audio). It landed the same week OpenAI put its Decisions API into public beta, and the conversation on X quickly turned to running these decisions locally instead of paying per call.

The idea is simple and, for backend people, more interesting than another chat model. A decision model does not write text. You hand it a state (a string, JSON, an image) and a set of named questions, and it returns typed, calibrated answers in one forward pass, with zero output tokens. Three question types cover most pipeline glue:

type	you give	you get back
`noul`	a yes/no question	`P(yes)`
`choice`	named options with descriptions	`choice`, `confidence`, `probabilities`
`score`	2 to 10 ordered levels	expected level, `confidence`, `probabilities`

That is exactly the shape of the ticket routers, intent classifiers, and moderation checks that a lot of us currently build with an LLM call, a prompt that says "answer only with JSON", and a parser that hopes for the best.

What Liquid AI is claiming

From the blog and model cards, in short:

d1-3B (3.12B params, built on LFM2.5-VL-3B) scores 48.57 on Decision Index v0.2.1, which Liquid says is the best under 10B and on par with a 35B-A3B decision model. Liquid reports 8 ms per decision on an RTX 4090, 30 ms on an Apple M5 Pro, and 50 ms on a Jetson Orin Nano.
d1-omni-600M is built on a 350M bidirectional encoder plus vision and audio encoders. Liquid calls it an early research release, scores it 15.95 on the same index, and deliberately publishes no speed numbers for it.
Both have day-one transformers support via trust_remote_code, and Liquid mentions llama.cpp support for the family.

The 3B model is the one Liquid is pushing. I ran the 600M one, for a boring reason: d1-3B wants about 12 GB in float32 on CPU, and the box I used had about 6 GB free. If you have a GPU or an M-series Mac, start with the 3B.

Run it yourself (CPU, about 10 minutes)

The 600M model card asks for transformers>=5.15 (the 3B card says >=5.14). so pin it explicitly:

python3 -m venv .venv && . .venv/bin/activate
pip install --index-url https://download.pytorch.org/whl/cpu torch torchvision
pip install "transformers>=5.15" pillow soundfile

Then one call answers three questions over the same ticket:

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "LiquidAI/d1-omni-600M", trust_remote_code=True, dtype=torch.float32
)

questions = {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
            "billing": "Charges, refunds, invoices",
            "technical": "App or site faults",
            "fraud": "Suspected unauthorised use",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["Can wait", "Today", "Blocking the customer now"],
    },
}

out = model.system_one(
    "I was charged twice for my October plan. Please refund the duplicate.", questions
)
print(out["answers"])

On my run, that ticket came back as billing with confidence 0.994, P(refund) = 0.999, and an urgency score of 1.69 on the 0 to 2 scale, using 157 input tokens and 0 output tokens. Two notes from the card: use float32 on CPU (the model was trained in it), and avoid bfloat16 on GPU, which Liquid says flipped the top answer on a small share of rows. trust_remote_code also downloads Python files from the repo, so pin a revision before you ship this anywhere.

What happened on 12 tickets

I wrote 12 short support tickets and their expected team and refund labels before running anything: four billing, four technical, four fraud-ish, including a phishing email question and a hijacked account.

Team: 12 of 12 matched my labels. Refund (P >= 0.5): 12 of 12.
Latency, three questions in one pass, warm: median 208 ms, range 160 to 471 ms. One question alone: median 114 ms.
Batch: 48 single-question requests through system_one_batch took 3.63 s, about 13 tickets per second.

Those are CPU numbers on a shared Xeon box, so treat them as "fast enough for a background queue", not as a latency benchmark. Twelve easy tickets also say nothing about your data; the point is that the API shape works and the probabilities are usable.

The more useful part was where confidence dropped. The phishing question ("I got an email asking me to confirm my card number on a weird link. Is that you?") still went to fraud, but at 0.477. The hijacked-account ticket went to fraud at 0.633. Those are exactly the tickets I'd want a human to see, and the confidence field flags them without any prompt engineering.

Where it fools you: forced choices

A choice question always picks one of your options. I sent two inputs that are not tickets at all:

"What's your office address in Toronto?" went to billing at 0.496. The low confidence gives it away.
"asdf asdf test test" went to technical at 0.907. High confidence, and wrong.

So a confidence threshold alone does not catch junk. The obvious fix is a catch-all option, so I added "other": "None of the above, or not a real support request" and a separate noul gate, "Is this a genuine customer support request?". That fixed the junk: the gibberish moved to other (0.663) with P(request) = 0.004, and the address question moved to other (0.896).

It also moved two real tickets. With other available, the phishing question went to other (0.603) and "Do you have a nonprofit discount?" went to other (0.699) instead of billing. The clear tickets (double charge, crashing app) stayed put at 0.90 or higher. Adding a catch-all moves the decision boundary for every borderline input, not just the junk.

How I'd actually wire it

Gate first. Ask a noul "is this a real request?" question and drop or queue anything with very low P(yes).
Route with explicit options, no catch-all, and send anything under a confidence threshold (I'd start around 0.7 and tune it on real tickets) to a human or a bigger LLM.
Write option descriptions like policy. "Suspected unauthorised use" did not obviously cover "someone asking if a phishing email is real". If phishing reports should go to fraud, say so in the description.
Label 100 real tickets before trusting any of this. My 12 are a smoke test. Measure agreement, and check which tickets land between 0.4 and 0.7.
Pin the revision of a trust_remote_code model, and try d1-3B if you have the memory. It's the one Liquid benchmarks.

Why this matters

For years the pattern has been to call an LLM, beg for JSON, and parse it. A small model that returns probabilities in one pass, runs on a CPU, and costs nothing per call is a real option for the boring 80% of classification work. It doesn't replace judgment on the hard 20%, but it tells you which tickets those are.

The scripts and raw output from this run (triage.py, other_option.py) need only the pip install above and no API key.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Claude Haiku 5.5 Just Landed: Migrating a Haiku 4.5 Classifier Without the 400s (temperature, Prefill, Thinking Blocks)

Yong Yu — Thu, 08 Oct 2026 13:20:08 +0000

Attributed compile + one small offline smoke test

Primary sources (Anthropic, 2026-10-07): Introducing Claude Haiku 5.5 (Anthropic); the Claude Platform docs What's new in Claude Haiku 5.5, Migration guide, and Prompting Claude Haiku 5.5

Secondary: Introducing Claude Haiku 5.5 on AWS by Aamna Najmi, Dani Mitchell, Alfredo Castillo, and Sofian Hamiti (AWS Machine Learning Blog, 2026-10-07)

Prices, benchmark claims, and the "~30% more tokens" figure are Anthropic's. I have no Claude API key on this box, so I did not call Haiku 5.5. What I did run is an offline check of the current Python SDK and a small request linter against hand-built mock responses, described below as exactly that.

Anthropic shipped Claude Haiku 5.5 yesterday, and its launch post on X drew close to 5 million views within a day. The headline is price. Anthropic lists $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100K tokens ($0.50 / $2.50 above that), against $1 / $5 for Haiku 4.5, and says it costs "around 75% less to run" on average. It also gets a 1M-token context window, 128K max output, and, for the first time on a Haiku, an effort setting.

Most coverage is about benchmarks. My view as a backend engineer is narrower: Haiku is the model people put behind ticket routers, intent classifiers, and extraction jobs. Those are exactly the integrations written in the Haiku 3/4.5 style, with temperature=0, a prefilled {"label": ", and max_tokens=20. Every one of those habits now breaks.

What actually changed for a classifier

Anthropic's migration guide lists several breaking changes. Three hit a typical classifier directly, and three more change behavior without an error.

Hard errors (HTTP 400 per the docs):

Sampling parameters. temperature must be omitted or exactly 1. top_p must be omitted or exactly 0.99 (even 1 fails), top_k is rejected at any value, and sending temperature and top_p together also fails. The "set temperature to 0 for deterministic labels" reflex is now an error.
Assistant prefill. A final assistant turn for the model to continue is rejected, even with thinking off. The docs recommend structured outputs or a tool with an enum field for classification instead.
Manual thinking budgets. thinking: {"type": "enabled", "budget_tokens": N} is gone. Use {"type": "adaptive"} and steer it with output_config.effort.

Silent changes:

Thinking is on by default, so a response can start with a thinking block even if you never asked for one. By default that block arrives with an empty thinking field and only a signature. Code that reads content[0].text gets nothing.
Thinking tokens count toward max_tokens. A max_tokens=20 request can stop with stop_reason: "max_tokens" after the thinking block and before any text.
The tokenizer changed. Anthropic says the same text counts as about 30% more tokens than on Haiku 4.5. Token-based budgets, truncation limits, and cost dashboards all shift.

There is also a new failure path: Haiku 5.5 runs safety classifiers that can return stop_reason: "refusal" with a stop_details.category, and the docs say retrying usually refuses again. If you run Priority Tier on Haiku 4.5, the guide also notes Priority Tier isn't supported on 5.5.

The smoke test: what I could check without an API key

SDK check. I created a fresh virtualenv on a Linux box (Python 3.13) and ran pip install anthropic, which resolved to anthropic 1.12.1 today. Inspecting client.messages.create showed thinking, output_config, and tool_choice parameters, and no temperature, top_p, or top_k parameters at all. Calling it with temperature=0 raised:

TypeError: Messages.create() got an unexpected keyword argument 'temperature'

That happens client-side, before any network call. So depending on which SDK version your service pins, the same legacy code fails either as a Python TypeError at call time or, per Anthropic's docs, as a server-side 400. Both look like "the classifier is down," and neither tells you which label you would have gotten. I didn't check older SDK versions or the extra_body escape hatch.

Request linter. I wrote a ~150-line preflight.py that encodes the migration-guide rules and points them at a classic Haiku 4.5 ticket classifier:

legacy = {
    "model": "claude-haiku-4-5",
    "max_tokens": 20,
    "temperature": 0,
    "system": "Classify the support ticket. Reply with JSON only.",
    "messages": [
        {"role": "user", "content": "I was charged twice this month, please refund one."},
        {"role": "assistant", "content": '{"label": "'},   # prefill
    ],
}

The linter flagged the model ID, temperature=0, the prefill, and the 20-token max_tokens (as a warning, since thinking eats into it). The migrated version replaces the prefill with a forced enum tool:

fixed = {
    "model": "claude-haiku-5-5",
    "max_tokens": 1024,
    "system": "Classify the support ticket. Reply with JSON only.",
    "messages": [{"role": "user", "content": "I was charged twice this month, please refund one."}],
    "tools": [{
        "name": "set_label",
        "description": "Record the single best label for the ticket.",
        "input_schema": {"type": "object",
                         "properties": {"label": {"type": "string",
                                                  "enum": ["billing", "technical", "account", "other"]}},
                         "required": ["label"]},
    }],
    "tool_choice": {"type": "tool", "name": "set_label"},
    "output_config": {"effort": "low"},
}

Why a forced tool? Per the migration guide, with a forced tool_choice the response starts with the tool call and carries no thinking block. For a single-label router, that's the behavior you want: no reasoning tokens to pay for, and an answer constrained to your enum. If the label needs judgment, use tool_choice: {"type": "auto"} and let adaptive thinking work.

Response reader. Finally, I built four mock responses by hand, shaped the way the docs describe: thinking-then-text, forced tool call, thinking-then-max_tokens, and a refusal. They validated against the SDK's own Message pydantic type. A type-aware reader handled all four:

def read_answer(resp):
    if resp["stop_reason"] == "refusal":
        return {"status": "refused", "category": (resp.get("stop_details") or {}).get("category")}
    for b in resp["content"]:
        if b["type"] == "tool_use":
            return {"status": "ok", "label": b["input"]["label"]}
    text = "".join(b["text"] for b in resp["content"] if b["type"] == "text")
    if resp["stop_reason"] == "max_tokens" and not text:
        return {"status": "truncated_after_thinking"}
    return {"status": "ok", "text": text}

The Haiku 4.5-era content[0].text returned None on three mocks and raised IndexError on the refusal mock, where I'd given it an empty content list. These are mocks, not model output. They show what your parsing code does with the documented shapes, not how often Haiku 5.5 actually produces each one.

On the cost math

The pricing is compelling, but recompute it with the new token counts instead of reusing Haiku 4.5 numbers. As an illustration using Anthropic's list prices and its ~30% estimate: a prompt measured at 400 input tokens on 4.5 becomes roughly 520 on 5.5. Per million such requests, input cost goes from about $400 to about $52. That's still a large cut, but it's smaller than the per-token price drop suggests. Effort matters too. Anthropic's prompting guide says moving from low to medium roughly halved early stopping in long agent prompts and more than doubled output tokens per attempt. Medium is the default on the API, so a high-volume classifier that never sets effort pays for thinking it may not need.

Run count_tokens with model="claude-haiku-5-5" on a sample of real prompts rather than trusting the 30% figure for your content.

Where it fits, and where something else might

Anthropic and AWS both pitch Haiku 5.5 as the fast layer: routing, classification, extraction, summaries, and subagents under Opus or Sonnet. Anthropic says Sonnet and Opus remain the better choice for complex agentic coding. For pure bounded-choice decisions, it's worth knowing that OpenAI put a Decisions API into public beta the day before, which returns typed predicates, choices, and scores from gpt-6-luna. I haven't tested either. If your workload is "pick one of N labels," evaluate both on your own labeled set before committing.

Migration checklist

Swap the model ID (claude-haiku-5-5; Bedrock uses anthropic.claude-haiku-5-5 or a regional inference profile).
Delete temperature, top_p, and top_k, and check whether your pinned SDK still accepts them.
Replace prefills with an enum tool or structured outputs.
Replace budget_tokens with adaptive thinking plus an explicit effort. Start a classifier at low.
Raise small max_tokens values, or force the tool call so there's no thinking block.
Read content blocks by type, never by position.
Handle stop_reason: "refusal" without blind retries.
Recount tokens and redo cost estimates on real prompts.
Before switching traffic, compare labels from 4.5 and 5.5 on a few hundred real inputs.

Anthropic's guide also mentions a /claude-api migrate skill inside Claude Code that applies these edits across a repo. I didn't try it, but the checklist above tells you what to check in its diff.

The SDK check and the response reader can be reproduced from the snippets above with pip install anthropic and no API key. Again, no live model calls were made.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

EmbeddingGemma 2 Just Dropped: Local Code Search on a CPU, the Install Gotcha, and Two Ways It Fails Silently

Yong Yu — Wed, 07 Oct 2026 13:15:16 +0000

Attributed compile + one small real run

Primary sources (all Google, 2026-10-06): EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings by Sahil Dua and Henrique Schechter Vera (Google DeepMind); EmbeddingGemma 2: The Developer Guide by Maarten Grootendorst and Ian Ballantyne; Bring multimodal semantic search to the edge with EmbeddingGemma 2 (Google AI Edge and Android ML teams); the google/embeddinggemma-2 model card on Hugging Face

Secondary coverage: Google releases a 740M-parameter embedding model for local multimodal search by Ryan Merket (Runtime Wire, 2026-10-06)

Model specs, benchmark scores, and on-device RAM figures below are Google's own numbers. The only thing I measured is the tiny text-only code-search test in this post, run once on a CPU-only Linux box. It is a smoke test, not a benchmark. I did not test images, video, or audio.

Google released EmbeddingGemma 2 yesterday: an Apache 2.0 embedding model, built on Gemma 4, that puts text, code, images, video, and audio into one 768-dimensional vector space. The first EmbeddingGemma was text-only and, per Google, passed 20 million downloads. This one is the "search my photos with a sentence, without uploading them" version.

Most coverage focuses on the multimodal demos. As a backend engineer, I care about a narrower question: can I drop the text-and-code part into a RAG pipeline today, on a CPU, and what bites first? I ran it to find out.

What shipped (vendor-reported)

Modular size. 740M parameters in total, but you load only what you need. Text and code alone is 270M. Adding vision makes it 440M, adding audio instead makes it 570M, and everything together is 740M. Google's AI Edge team reports about 191 MB of active RAM for text-only and about 567 MB for the full model on a Pixel 11 Pro.
One shared space. A text query can be compared directly with an image, a video frame, or an audio clip. All inputs share an 8,192-token budget. Google prices an image at 280 tokens, a video frame at 140 tokens (sampled at 1 fps by default), and audio at 25 tokens per second.
Matryoshka truncation. You can keep only the first 512, 256, or 128 dimensions of each vector to save storage. Google's table shows the cost: the code benchmark drops from 78.68 at 768d to 71.41 at 128d, and the overall multimodal score drops from 59.01 to 45.65. Google recommends 128d mainly for text-only workloads.
Code is the headline text gain. On MTEB Code, Google lists 78.68 versus 68.76 for the first EmbeddingGemma. Multilingual MTEB barely moved (61.36 versus 61.15). So if you only embed prose, the upgrade is mostly about multimodality, not text quality.
Task prefixes matter. The model was trained with short instructions such as task: code retrieval | query: ... for queries and title: "... | text: ... for documents. Sentence Transformers exposes these as named prompts."

The run: natural-language code search on a CPU

My setup: a CPU-only Linux box with 8 vCPUs, Python 3.13, torch 2.14.1+cpu, sentence-transformers 6.1.0, and transformers 5.19.0. The weights are not gated on Hugging Face, so there is no license click-through or token.

I wrote eight small Python snippets (retry with backoff, slugify, JWT signature check, text chunker, token-bucket rate limiter, CSV dedupe, a FastAPI health check, cosine similarity). Then I asked six plain-English questions that deliberately avoid the function names, like "back off and try again when a call fails."

import torch
from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "google/embeddinggemma-2",
    config_kwargs={"vision_config": None, "audio_config": None},  # text + code only
    model_kwargs={"torch_dtype": torch.float32},                   # never float16
)

docs = [f"title: "{name} | text: {code}\" for name, code in snippets.items()]"
d = model.encode(docs, truncate_dim=768, normalize_embeddings=True)
q = model.encode(queries, prompt_name="CodeRetrieval",
                 truncate_dim=768, normalize_embeddings=True)
scores = q @ d.T   # cosine similarity, since both sides are unit length

What happened on this run:

The text-only model loaded in 6.4 seconds and reported 271M parameters, which matches Google's 270M text configuration.
All six queries put the right snippet first, at both 768d and 128d. For example, "check that a signed auth token wasn't tampered with" returned jwt_check.py, and "remove duplicate customers from a spreadsheet export" returned csv_dedupe.py.
Encoding the eight snippets plus six queries took about half a second on the CPU.

Eight documents is far too few to say anything about quality at scale. What it does show is that the text path runs comfortably on an ordinary CPU, and that the CodeRetrieval prompt works as documented.

Gotcha 1: "text-only" still needs the image stack to load

My first two attempts crashed before any embedding happened. Even with vision_config=None and audio_config=None, loading failed first with EmbeddingGemma2Processor requires the PIL library, and after installing Pillow, with No module named 'torchvision'. In this version combination the processor imports Gemma 4's image-processing code no matter which encoders you enable.

The fix that worked for me:

pip install -U sentence-transformers transformers pillow
pip install --index-url https://download.pytorch.org/whl/cpu torch torchvision

If you are building a slim Docker image for a text-only RAG service, budget for both packages, or pin versions and check again when a later release decouples them.

Gotcha 2: slicing vectors without re-normalizing

Matryoshka truncation tempts you to store 768d vectors and slice them later. The model card warns that sliced vectors are no longer unit length, and that cosine scores then degrade "silently." I checked what that looks like in practice.

After slicing my normalized 768d document vectors to their first 128 values, their lengths were about 0.58 to 0.60 instead of 1.0. The score for the retry query against retry.py fell from 0.849 (properly re-normalized 128d) to 0.295 (raw slice). Nothing errors. If your retriever has a similarity cutoff such as "drop anything under 0.5," the raw slice silently returns nothing.

The safe version is to let the library do it:

model.encode(texts, truncate_dim=128, normalize_embeddings=True)

Also keep query and document dimensions identical. A 768d query can't be scored against a 128d index.

A related surprise from the same run: the correct 128d scores were all higher than the 768d ones (0.82 to 0.87 versus 0.75 to 0.79). The rankings matched, but the absolute numbers moved. Any threshold you tuned at one dimension needs re-tuning at another.

Gotcha 3: float16 (per Google, not tested)

Google's guide says the model's activations overflow float16, which produces NaN or degraded vectors without raising an error. Use bfloat16 on GPUs that support it and float32 on CPUs. I only ran float32, so I'm passing this along rather than confirming it. It matters because "cast to fp16 to save memory" is a common reflex.

A small prompt-name note

Google's current docs use names like SearchQuery, Document, and CodeRetrieval. The checkpoint I loaded also still exposes the older names (Retrieval-query, Retrieval-document, STS) from EmbeddingGemma 1. Pick one convention and use it for both indexing and querying. Mixing a query prefix from one family with documents embedded under another is the kind of mismatch that never throws an error.

When to reach for it

Private, local search over notes, code, screenshots, or recordings that shouldn't leave the machine. This is the case Google is designing for.
Code search inside a dev tool, where the 270M text configuration is small enough to ship with the tool.
Zero-shot intent routing. Google's edge post pitches matching user input directly against label descriptions, with no training data, as a cheap router in front of a bigger model.

If you already pay for a hosted embedding API and your data can leave the device, Google's own cloud counterpart, Gemini Embedding 2, is the comparison to run on your own corpus before switching.

Checklist

Install pillow and torchvision even for text-only use.
Load only the encoders you need via config_kwargs.
Use float32 on CPU and bfloat16 on GPU. Avoid float16.
Use the right task prefix: CodeRetrieval or SearchQuery for queries, and title: "... | text: ... for documents."
Truncate with truncate_dim plus normalize_embeddings=True, never by slicing.
Re-tune similarity thresholds whenever you change dimensions.
Validate on 50 to 100 of your own queries before trusting the 128d setting.

Byline: YongBo Yu — Toronto AI engineer (agents, LLM workflows, RAG). GitHub: YongBoYu1.

Gemini Skills Just Landed in Workspace: Your First SKILL.md, Moving a Gem, and Why a Migrated Gem Isn't a Working Skill

Yong Yu — Tue, 06 Oct 2026 13:23:42 +0000

Attributed compile (not original research)

Primary sources: Let skills in Gemini tackle your most repetitive tasks by Deven Tokuno, Senior Product Manager, Gemini App (Google, 2026-09-30); Introducing skills in the Gemini app and Workspace, plus what's next for Gems (Google Workspace Updates, 2026-09-30, with the Workspace rollout starting 2026-10-05); Google's help pages Create & manage skills for Gemini Apps and Write effective skills for Gemini Apps; the open Agent Skills specification

Secondary coverage: Google Gemini Skills Start Rolling Out Today, Replacing Gems by Abhilasha Raut (Affitronix, 2026-10-05); Google starts retiring Gemini Gems in November (On The Wire, 2026-10-03); Google launches Skills in Gemini by Brian Beckmann (2026-10-01)

This post restates Google's announcements and help docs in my own words, with a practical first-skill checklist. I did not create, upload, or migrate a skill in Gemini or Workspace for this article. Feature descriptions, dates, and limits are vendor-reported or attributed to the named source. The SKILL.md and folder examples are adapted patterns based on the spec. The small Python checker was run locally against a sample folder only. It is not a Google tool and was not tested against Gemini's uploader.

Google is retiring Gems. Their replacement, skills, started reaching Workspace Rapid Release domains on October 5. Personal accounts in the Gemini app have been getting them gradually since the September 30 announcement.

The headline is convenience: save an instruction set once, then call it with / in any chat. The more interesting part for engineers is the format. Google says skills are "built entirely on the open, Markdown-native SKILL.md standard." That is the same folder-plus-Markdown format used by the Agent Skills spec, which Beckmann notes is also read by Claude, GitHub Copilot, and Google's own Gemini CLI.

So a skill you write for Gemini can be a plain file in a Git repo. It is no longer a setting trapped in one product.

What changed (and when)

Here are the dates from Google's Workspace Updates post and blog:

Date	What happens (vendor-reported)
Sep 30, 2026	Skills announced; rolling out "directly into Gemini chat globally" for personal accounts, 18+
Oct 5 – Oct 12	Skills in Workspace apps, Rapid Release domains
Oct 13 – mid-Nov	Skills in the Gemini app for Workspace domains
Oct 19 – mid-Nov	Skills in Workspace apps, Scheduled Release domains
Nov 17, 2026	Gems move into the Gemini app's Settings panel (still usable)
November 2026	Gem support removal begins for personal accounts
No sooner than Mar 1, 2027	Business/enterprise: Gems stop working; leftovers auto-migrate to draft skills
No sooner than Jun 1, 2027	Education: same, plus Classroom and LTI integrations

If your Gemini looks different from your coworker's, this staggered schedule is probably why. Personal account, Workspace domain, and release channel each land on a different date.

The mental model: skills are loaded lazily

A Gem was a separate assistant with its own chat. A skill is closer to a plugin that the model pulls into the current conversation. Google's help page describes three stages, which line up with the "progressive disclosure" section of the Agent Skills spec:

Match. Gemini reads only each skill's name and description to decide whether it is relevant.
Load. If it matches, Gemini reads the full instructions.
Fetch. Attached files (templates, style guides, data) are opened only when a step needs them.

The practical consequence: your description is your router. Gemini can apply skills automatically, and a vague description means the skill never fires. Google's writing guide says to start with a third-person capability statement, add "Use when…" triggers, and stay under 1,024 characters. The spec sets the same 1,024-character limit.

You can also stack skills. Google's example pairs a writing-style skill with a brand-guidelines skill in one prompt. Because each skill should do one job, this pushes you toward small, composable files rather than one giant system prompt.

Your first SKILL.md (adapted pattern)

Per the spec, a skill is a folder whose name matches the skill's name, with SKILL.md at the root:

draft-weekly-update/
├── SKILL.md
└── references/
    └── tone.md

---
name: draft-weekly-update
description: Drafts a weekly status update from meeting notes and task lists. Use when the user asks for a weekly update, status report, or Friday summary.
metadata:
  author: your-name
  version: "1.0"
---

# Weekly update

1. Read the notes the user provides (or the attached doc).
2. Group items under: Shipped, In progress, Blocked, Next week.
3. Use the tone rules in [references/tone.md](references/tone.md).

## Common mistakes
- Do not invent dates or owners. If one is missing, ask.
- Keep it under 200 words unless the user asks for more.

The "common mistakes" section and the "ask instead of guessing" rule both come straight from Google's best-practices page. They are the cheapest anti-hallucination measures you can add.

Pre-upload check (adapted pattern)

Google's upload rules: a SKILL.md in the root folder, a lowercase-hyphenated name, plain-text files only (no .docx or .xlsx), 100 MB total, and a tip to strip hidden files like .DS_Store and .pyc. Scripts can't reach external websites. One surprise: .env appears on the supported-file list, so nothing stops you from uploading a secret by accident. This checker flags all of that before you click Upload:

# check_skill.py: pre-upload sanity check for a SKILL.md folder
import re, sys
from pathlib import Path

NAME_RE = re.compile(r"^[a-z0-9]+(-[a-z0-9]+)*$")    # lowercase, single hyphens
BLOCKED = {".DS_Store", ".env"}                        # junk + secrets
BINARY_EXT = {".docx", ".doc", ".xlsx", ".pyc"}         # not plain text
MAX_BYTES = 100 * 1024 * 1024                          # Gemini's 100 MB total

def check(folder: Path) -> list[str]:
    errs, skill = [], folder / "SKILL.md"
    if not skill.is_file():
        return ["SKILL.md missing from folder root"]
    m = re.match(r"^---\n(.*?)\n---\n", skill.read_text(), re.S)
    meta = dict(re.findall(r"^(\w[\w-]*):\s*(.+)$", m.group(1), re.M)) if m else {}
    name, desc = meta.get("name", ""), meta.get("description", "")
    if not NAME_RE.fullmatch(name) or len(name) > 64:
        errs.append(f"bad name: {name!r}")
    if name != folder.name:
        errs.append(f"folder {folder.name!r} != name {name!r}")
    if not 0 < len(desc) <= 1024:
        errs.append("description must be 1-1024 chars")
    files = [p for p in folder.rglob("*") if p.is_file()]
    for p in files:
        if p.name in BLOCKED or p.suffix in BINARY_EXT:
            errs.append(f"remove before upload: {p.relative_to(folder)}")
    if sum(p.stat().st_size for p in files) > MAX_BYTES:
        errs.append("total size over 100 MB")
    return errs

if __name__ == "__main__":
    problems = check(Path(sys.argv[1]))
    print("\n".join(problems) or "OK")
    sys.exit(1 if problems else 0)

Run python check_skill.py draft-weekly-update. On my sample folder it printed OK. On a copy renamed to Weekly_Update with a stray .DS_Store, it flagged the folder/name mismatch and the hidden file. It is a lint, not a full YAML parser. For spec-level validation, the Agent Skills project points to skills-ref validate ./my-skill.

The trap: a migrated Gem is a draft, not a working skill

Google says remaining Gems will be auto-migrated, and the Workspace post calls the result draft skills. Read the fine print before you count on that:

Some Gem tools don't carry over yet. As relayed by On The Wire, Google's help page says most default Gem tools don't work with skills for now, naming Create video, Create music, Canvas, Deep Research, and Guided Learning.
Skills don't sync. Google says skills in the Gemini app and skills in Workspace apps are separate. You have to create each one twice.
Sharing and Drive files are "coming weeks." Both are promised but not shipped yet, per Google's blog.
Labs Gems and Opal don't migrate. Google's footnote says Gems by Google Labs will not become skills, and Opal shuts down in November.
Editing files means re-uploading. To change a skill's files, you upload the whole folder again (Skill actions → Replace skill).

Checklist for moving a Gem early

Open each Gem and download its Knowledge files into a folder named exactly like the future skill (lowercase, hyphens).
Create the skill in Settings → Skills → Create manually, then paste in the name, description, and instructions. Use the web app; Google lists manual creation on mobile as a known bug.
Rewrite the description as a router: what it does, then "Use when…".
Download the new skill's .zip, move SKILL.md into your folder, run the checker, then use Replace skill to upload the folder.
Commit the folder to a repo. That copy is your source of truth for Workspace, for Gemini CLI, and for anything else that reads SKILL.md.
Test any Gem that relied on Deep Research or Canvas before November, not after.

Why this matters

Gems were product configuration. Skills are files. Once Google, Anthropic, and GitHub all read the same SKILL.md layout, your prompt engineering stops belonging to one vendor's settings page and becomes something you can version, review, and lint. The work that pays off is the description, because it decides when a skill fires, and the "ask instead of guessing" rules, because they decide whether the output can be trusted.

Byline: YongBo Yu — Toronto AI engineer (agents, LLM workflows). GitHub: YongBoYu1.

Codex CLI's Big Refresh Just Landed: First Session, Parallel Agents, and Why /fork Isn't a Worktree

Yong Yu — Mon, 05 Oct 2026 19:01:35 +0000

Attributed compile (not original research)

Primary source: ChatGPT & Codex changelog by OpenAI, Codex CLI releases 0.156.0 (2026-09-22) through 0.160.0 (2026-10-01), plus OpenAI's Codex CLI command reference

Announcement context: OpenAI Developers announced the "major refresh" on 2026-09-29; the text was reposted in Major Codex CLI refresh, with a new look and powerful new capabilities (OpenAI Developer Community, 2026-09-30)

Secondary coverage: OpenAI Refreshes Codex CLI with Full-Screen UI and Parallel Task Management by Hiren Dave (TheNextGenTechInsider, 2026-10-04); Codex CLI Update: Voice, the /agents View, Resume and Worktrees by Rogier Muller (Codex Workshop, 2026-09-29)

This post restates OpenAI's release notes and docs in my own words, with a first-session checklist. I did not run the new Codex CLI build for this article. Feature descriptions are vendor-reported or attributed to the named secondary source. Shell snippets are adapted patterns using documented commands, not output from a real session.

OpenAI is calling it a "major refresh" of Codex CLI. The announcement pitches three things: a full-screen terminal interface, better readability, and "ways to manage parallel work" without leaving the terminal. Under the hood it is not one release. It is a run of builds, 0.156.0 through 0.160.0, shipped between September 22 and October 1.

That matters for a practical reason. Which version you have decides what you actually see. And one headline phrase, "/fork into a worktree," is easy to misread in a way that can cost you a messy checkout.

What actually changed (by release)

Here is the changelog, grouped by what you will notice:

Release	Date	What changed (vendor-reported)
0.156.0	Sep 22	Optional full-screen UI via `/tui` (transcript search, mouse selection, right-click copy); voice on by default (F8); `/usage` analytics dashboard; worktree sessions from the agent command center, with worktree support on by default; Mermaid diagrams and equations rendered in the terminal; `--no-daemon`
0.157.0	Sep 25	Full-screen transcript on by default; automatic background-server startup for eligible sessions; GPT-6 Sol and Luna in the catalog
0.159.0	Sep 29	Opt-in `instant_interrupt` so new input can steer mid-response; compact welcome screen; `.aws` directories protected by default under writable roots
0.159.1	Sep 29	GPT-6.1 Sol becomes the default in the bundled catalog
0.160.0	Oct 1	"Show more" history in the agent command center; projectless sessions (when policy permits); unsent queued messages resume after a reconnect without being sent twice

The community repost of the announcement points to two quality-of-life fixes. Older history now loads as you scroll, past your terminal's own scrollback limit. And the composer stays pinned at the bottom, so you no longer scroll back down to type during a long session. One commenter in the thread called the old behaviour a bug.

The mental model: the TUI is now a control plane

Before this refresh, Codex CLI was basically a chat loop in your terminal. Now it is closer to a supervisor for several agent threads:

The agent command center (OpenAI's announcement calls it the /agents view) lists your sessions. You can filter them by status and start worktree sessions from there.
Subagent threads are things you switch into with /agent or /subagents, per the current command reference. Codex Workshop notes that the announcement and the docs use different names here, so type / and check what your build actually offers.
Steering vs queueing. Press Enter while Codex is working to inject instructions into the current turn. Press Tab to queue a follow-up for the next turn. Press Escape twice with an empty composer to edit your last message and fork the chat from that point.

The trap: /fork is a chat fork, not automatically a checkout

As relayed in press coverage, the announcement says you can "/fork a conversation into a worktree with the same context and a separate checkout." That sounds like /fork always gives you filesystem isolation.

OpenAI's own command reference describes /fork more narrowly. It clones the current chat into a new chat with a fresh ID and leaves the original transcript alone. The thing that creates a separate checkout is a worktree session. Per the 0.156.0 notes, you start one from the agent command center, and worktree support is on by default.

If you fork a chat and both threads keep writing to the same directory, you have two agents editing one working tree. The isolation you assumed isn't there. So check before you let a second agent write anything:

# Adapted pattern: confirm isolation before parallel edits
git worktree list            # the second session should have its own path
# inside each Codex session:
#   /status  -> check writable roots point at the path you expect

Codex Workshop, citing OpenAI's worktree guide, adds two caveats. Managed worktrees sit under $CODEX_HOME/worktrees on a detached HEAD. And ignored files such as .env are not copied into them automatically. Git's usual rule also still applies: a branch can be checked out in only one worktree at a time.

First session checklist

Upgrade. Run npm install -g @openai/codex@latest, or pin @openai/codex@0.160.0 if you want the October 1 build exactly. codex update works when your installed release supports self-update.
Health check. Run codex doctor --summary to check install, config, auth, Git, and terminal.
Start with sane permissions. The docs suggest codex --sandbox workspace-write --ask-for-approval on-request for low-friction local work. Avoid --yolo outside an isolated VM.
Confirm the session. Run /status to see the active model, approval policy, writable roots, and context usage. If GPT-6.1 Sol isn't what you expected, switch with /model.
Parallelize deliberately. Start the second task as a worktree session, then verify with git worktree list. Codex Workshop also warns that approval prompts can come from a thread you aren't looking at, so read which thread is asking before you approve.
Cap concurrency. Codex Workshop documents a per-project limit you can set:

   # .codex/config.toml (pattern from Codex Workshop's restatement of the docs)
   [agents]
   max_concurrent_threads_per_session = 4

Watch the bill. Subagents each do their own model and tool work, so parallel sessions use more tokens. Use /usage (added in 0.156.0) to see token activity.
End each thread with a review. Run /diff, then /review, before you merge anything a parallel thread produced.
Escape hatch. If the full-screen UI fights your tmux or SSH setup, /tui picks the UI mode for your next launch, and --no-alt-screen turns off alternate-screen mode for a single run.

Why this matters

Terminal coding agents are turning into orchestrators. The full-screen UI is the visible part. The real change is that one terminal now supervises several threads, each with its own context, permissions, and maybe its own checkout. That makes "which directory is this agent writing to?" a question you have to answer on purpose. Before you run two agents at once, check git worktree list and look at the writable roots in /status.

Byline: YongBo Yu â€” Toronto AI engineer (agents, LLM workflows). GitHub: YongBoYu1.

Claude Code Mods Just Landed: First Hook, Blast Radius Guard, and Why They're Not Sandboxed

Yong Yu — Fri, 02 Oct 2026 13:20:24 +0000

Attributed compile (not original research)

Primary source: Getting started with Claude Code mods by Addy Osmani (claude.dev), published 2026-10-01

Secondary roundup: Anthropic Launches Claude Code Mods, TypeScript Agent Hooks (AI Weekly), published 2026-10-02

This post restates Anthropic’s Claude Code Mods launch packaging in my own words, with a builder checklist for first hooks and install hygiene. I did not run Claude Code Mods for this article. Hook shapes, example mods, version floors, and trust claims below are vendor-reported (or clearly attributed to the named secondary source). Code snippets adapt the public API patterns from Osmani’s guide — not a dump of the full sample modules.

Anthropic shipped Claude Code Mods on October 1, 2026. Per Osmani’s guide, a mod is a small JavaScript or TypeScript module that lives inside a plugin, registers hooks into Claude Code’s event loop, and can rewrite prompts, hold or deny tool calls, approve or refuse permissions, redact secrets, and draw or replace UI. Claude Code 2.1.287+ is the floor; mods are on by default.

That last detail is the product decision. You are not opting into a toy extension API. You are sitting on a middleware layer that already powers first-party features — and that runs with the same machine reach as Claude Code itself.

What a mod actually is (middleware, not a settings shell)

Osmani frames mods as hooks that ship inside plugins. The folder is a normal plugin (.claude-plugin/plugin.json). hooks/hooks.json points at one module under modules. That module exports register(on, options), and inside it you attach handlers with on(event, matcher?, hook).

Every hook shares one shape (API pattern from Osmani’s post):

on("tool.call", { tool: "Bash" }, async ($, e, next) => {
  // $ = mods API (ui, session, state, fs, process, …)
  // e = this event's plain data
  // next = pass e down the chain to other mods / Claude Code
  return next(e);
});

Hooks chain like Express middleware. The first-loaded mod sees the event first and the result last — AI Weekly restates the same load-order rule from Anthropic’s packaging. Inside a hook you pick one of three moves:

Move	Pattern	Typical use
Observe	`const r = await next(e); /* inspect */; return r`	Log edits, meter context, build a replay
Rewrite	`return next({ ...e, command: safer })`	Change what the rest of the chain sees
Answer	`return { deny: "…" }` without calling `next`	Refuse a tool call; serve a command yourself

Events cover tool calls, prompt submit, turn start/finish, session lifecycle, slash commands, and ui.render. Osmani notes the module itself has no DOM and no Node — everything outside goes through $. That is not the same as “sandboxed from your laptop.” $ exposes fs, process, http, and friends. Treat install like npm: trusted source or skip.

First-party features are already mods

Claude Code dogfoods the system. Osmani says AGENTS.md support and the /diff pane beside the conversation ship as mods; their sources live under mods/ in the public anthropics/claude-code repo. AI Weekly adds that Team/Enterprise plans load a built-in sec-default mod first, aimed at blocking risky overrides of permission denials.

Practical takeaway: if Anthropic’s own UX is a mod, your team’s conventions, cost meters, and Bash guards belong in the same layer — not as one-off shell hooks that restart on every event.

Three example mods (vendor demos, not my harness)

Osmani walks three public demos. I am summarizing behavior only; numbers and line counts are from the guide.

Token Weather (~80 lines) — After each main-loop turn, reads $.session.usage(), keeps history in $.state, and draws a one-line forecast in the AbovePrompt band (Clear → Cloudy → Showers → Storm → Compact soon). Teaching point: hot reload wipes module variables; put session history in $.state.
Blast Radius — On risky Bash (rm -rf, git reset --hard, force push, migrations, …), classifies the command, dry-runs impact via $.process.run, opens a Proceed/Cancel pane, and either next(e) or returns { deny: "…" }. Teaching point: the Answer move is how you hold a call without rewriting Claude Code.
Replay Theater — Observes Edit/Write calls, registers /replay, and steps diffs in a pane. Teaching point: observe-only mods never call deny; they pair turn.start / turn.complete and filter out subagents via e.agentId.

If you only build one thing this week, build a Blast Radius–style guard tuned to your stack (production kubectl contexts, terraform apply, drop-table SQL). Permission rules are still the hard block; Blast Radius is a safety net that reads command text and can miss $(…), aliases, and nested scripts — Osmani says that explicitly.

Minimal Blast Radius core (adapted pattern)

The heart of holding Bash is the Answer move. Adapted from Osmani’s Blast Radius sketch (not the full module):

on("tool.call", { tool: "Bash" }, async ($, e, next) => {
  const risk = classify(String(e.command ?? ""));
  if (risk === null) return next(e); // safe path: pass through

  // measure($, …) dry-runs via git status / git clean -n / du, etc.
  const report = await measure($, risk, await $.session.cwd());
  const decision = await askUser($, report); // pane or AbovePrompt band

  if (decision === "proceed") return next(e);
  return {
    deny: `Held by Blast Radius: ${report.summary}`,
  };
});

Validate with claude plugin validate ./your-mod before you share it. Osmani’s guide shows the validator listing hooks, $ calls, and declared PluginState keys — fail closed on undeclared state.

Install path and the trust model

Shipping is the existing plugin channel. From Osmani:

/plugin marketplace add your-org/my-mods
/plugin install token-weather@your-mods
/reload-plugins

Claude directory submissions go through claude.ai/directory/manage. Hot reload watches the folder in a session started with --plugin-dir; Claude may ask once to enable it.

Trust is the feature, not a footnote. AI Weekly quotes Anthropic bluntly: mods run with the same access to your machine as Claude Code; they are not sandboxed; install only sources you trust. Osmani’s share section says the same in packaging language — publisher code, not Anthropic-signed isolation. sec-default helps Team/Enterprise at the front of the chain; it does not make a random marketplace plugin safe.

Shortcut Osmani highlights: describe the mod to Claude Code and let it scaffold the plugin. You still own the review of what it wrote before /plugin install hits a shared machine.

Builder checklist (this week)

Upgrade to Claude Code 2.1.287+ and confirm claude --version.
Do not mass-install marketplace mods until you have a trusted-source list (internal org marketplace first).
Scaffold one observe mod (Token Weather–style) so you learn $.state + ui.render / AbovePrompt without touching Bash.
Scaffold one Answer mod (Blast Radius–style) for the three commands your team fears most; keep permission rules as the hard deny.
Run claude plugin validate and at least one claude plugin test before sharing.
Prefer $.state over module let — hot reload re-runs register and session.start.
On Team/Enterprise, confirm sec-default is loaded first; document whether local users can dilute it (AI Weekly flags this as an open admin question).
Review any mod Claude writes for you the same way you review a PR that gets fs + process.

Why this post exists

Coding agents are becoming extension hosts. Mods make Claude Code’s loop a public middleware surface: rewrite, hold, redraw. The durable skill is not memorizing AbovePrompt — it is treating install hygiene as part of agent security, and shipping one guard that matches your blast radius before the directory fills with convenience plugins.

Byline: YongBo Yu — Toronto AI engineer (agents, LLM workflows). GitHub: YongBoYu1.

Gemini 4 Argon Just Dropped — But You Can't Call It Yet: Pricing, 1M Output Tokens, and What to Do Until the API Lands

Yong Yu — Thu, 01 Oct 2026 13:46:06 +0000

Attributed compile (not original research)

Primary source: Gemini 4 Argon: our next era of frontier intelligence by Koray Kavukcuoglu (Google DeepMind / Google blog), published 2026-09-30

Secondary roundup: Google announces Gemini 4 Argon as its new frontier model (9to5Google), published 2026-09-30

Optional context: Google's first Gemini 4 model is 'Argon' (Engadget), published 2026-10-01

Contrast only (not the headline): Introducing GPT-6.1 Sol (OpenAI) and GPT-6.1 Sol in GitHub Copilot (GitHub Changelog), both 2026-09-29

This post restates Googles launch packaging in my own words, with a builder checklist for what to do before a public model ID exists. I did not run Gemini 4 Argon it is not publicly available yet. Every price, benchmark, and access claim below is vendor-reported (or clearly attributed to a named secondary source).

Google announced Gemini 4 Argon on September 30, 2026, as its next frontier model after the Gemini 3.x line. The capability story is long-horizon software engineering, enterprise knowledge work, and defensive cybersecurity. The access story matters more for builders this week: Argon is live only inside Google’s Fairwind Program for trusted cyber defenders. Broader rollout is “soon,” starting with paid API customers and Google AI Ultra subscribers. There is no public model ID and no first-call snippet for general developers yet.

That phased release is the difference between bookmark the blog” and “ship a production default tomorrow.”

What Google announced (and why Fairwind-first matters)

Per Kavukcuoglu’s post, Argon is built for deep reasoning across complex, multi-step workflows. Google says it is already using Argon internally (thousands of Googlers, vendor claim) for specialized coding, research, and writing. External access starts with Fairwind — and for trusted defenders plus Google’s own teams, Google will ship a version without cyber guardrails so they can use full defensive capability.

For everyone else, Google is still iterating on safeguards (misuse including CBRN/cyber, prompt injection, misalignment monitoring of chain-of-thought and actions, and hardened sandboxes) while engaging the U.S. government’s voluntary pre-release process. Treat Fairwind-only as a hard product constraint: do not plan a customer-facing pipeline on Argon until you have a documented API model ID and a paid access path.

Pricing trap: intro $2/$10 vs post-intro $4/$20

Googles stated launch pricing (vendor):

Per 1M tokens (Google)	Introductory	After intro period
Input	$2	$4
Output	$10	$20
Cached input	95% off input price	same discount structure stated at launch

Intro sticker matches the familiar $2 / $10 band many teams already budget for mid-frontier coding models. After the introductory period, Google says rates move to $4 / $20 — Opus-class list territory. Cached input at 95% off input price is the quiet lever: long agent loops that re-send the same system prompt, repo summary, or tool schema will live or die on cache hit rate once Argon is in the API.

Build cost models for both columns now. An eval that looks cheap at intro rates can double on input/output when the promo ends. Do not bake “Argon is $2/$10 forever” into a board deck.

Why 1M output tokens changes agent design

Google expands Argon’s output token limit to 1M, up from 64K — framed as industry-leading generation headroom for long trajectories. Context-window talk is saturated. Output headroom is different: one trajectory can think, tool-call, revise, and emit a large artifact (migration plan, multi-file patch set, research memo) without mid-flight truncation.

Loops designed for 64K128K completions assume early summarization or multi-turn “continue. A 1M ceiling (once callable) means fewer artificial breakpoints — and more runaway-spend risk. Add budgets, stop conditions, and per-step logs before you get an API key.

Vendor benchmarks (labeled as such)

These figures come from Google’s announcement and secondary reporting. They are not my harness results.

Benchmark (vendor-reported)	Gemini 4 Argon	Notes
DeepSWE v1.1	77.9%	Long-horizon SWE; 9to5Google cites Claude Opus 5.5 at 74.2% and GPT-6 Astra at 74.1%
AutomationBench (Zapier)	51.3%	Google claims #1
LVBench (long video)	91.7%	Google claims SOTA
CWE-bench v1	68%	Google claims tied for first (vuln remediation)

Google also points to leading placement on the Vals Index and strong domain results on Vals Finance Agent v2 and Harvey’s Legal Agent Benchmark. Orientation only: different benches measure different jobs. Your golden set still wins.

Internal Google anecdotes (vendor examples, not your SLA)

Capability hints, not SLAs: quantum subroutine optimization (40% better than a published baseline in one example); fleet memory work freeing 300+ TiB; CRust migrations from re2 / libgav1 up to 800K+ lines for Fuchsia Zircon (under heavy audit); and a libgav1 Rust SIMD rewrite claimed 2.7× faster than the prior Rust port with identical video output.

Cyber angle: Argon is trained for defense; Wiz’s Scan for Good is cited for a critical healthcare exposure previous frontier models missed. Vendor narrative — not an independent pen-test for compliance.

What builders should do this week

OpenAI shipped GPT-6.1 Sol on September 29 at the same sticker band ($2 / $10) and it is already in the API and GitHub Copilot per OpenAI and GitHub’s changelog. That contrast is actionable: do not stall shipping while Fairwind widens.

Keep shipping on models you can call today: GPT-6.1 Sol, Claude Opus / Sonnet 5.5, Grok where it fits.
Prepare long-horizon SWE evals now — multi-file edits, flaky tests, migration-style tasks. When Argon’s model ID lands, you want a 2050 trajectory harness ready.
Watch for the public model ID and paid API / Ultra notes. No ID → no production default.
Do not block a release on Fairwind-only access unless you are in that program.
Price for the post-intro column ($4/$20) in any Argon ROI slide; assume cache misses until you measure them.
Security posture: if you ever get a no-guardrail cyber build, isolate it. Fairwind is defense capability plus staged safeguards not “same sandbox as the public chatbot.”

When Argon might become your default (once the API lands)

Only after a callable model ID and billing access:

Workloads are long-horizon and you are hitting today’s completion caps.
A golden eval shows Argon beating your current default on your failures — not DeepSWE screenshots alone.
Cost model survives post-intro $4/$20 and realistic cache hit rates.
Prompt-injection / tool-exfiltration tests pass under your threat model (Google claims Gray Swan IPI leadership — verify on your agents).
You can rollback to Sol / Opus / Sonnet without rewriting orchestration.

Until then, Argon is a roadmap signal. Treat the blog as planning input; treat GPT-6.1 Sol and the Claude 5.5 pair as this week’s shipping surface.

Why this post exists

Frontier launches now split into two products: the capability blog and the availability calendar. Argon is a serious capability claim with a Fairwind-first calendar. The durable skill is refusing to confuse them — and having harnesses, budgets, and rollbacks ready for the day a public model ID appears.

Byline: YongBo Yu — Toronto AI engineer (agents, LLM workflows). GitHub: YongBoYu1.

DEV Community: Yong Yu

Muse Spark Open Is Out: Reasoning You Can't Turn Off, KV-Cache Math at 128K vs 1M, and Hosted vs Mac vs GPU

The architecture numbers that matter

Gotcha 1: reasoning is always on, and it eats max_tokens

Gotcha 2 (a good one): the KV cache is small for an 816B model

Hosted vs Mac vs GPU

Why this matters

Grok Voice 3.0 Just Landed: First Realtime Session, Full Duplex, and Why $0.08/min Beats Guessing Latency

What the announcement actually claims

First realtime session (WebSocket)

Cost arithmetic (list price, not a bill)

What I actually ran (offline)

Alias auto-upgrade vs pinning

How I'd try it this week

Why this matters

Coinbase's 9B Fraud Model Beat Opus 4.5: Copy the Pattern With a Verifiable Reward, Tier-Only Output, and Block-Only Harness

What Coinbase actually built

Why the reward is "verifiable" here

Why dollar-weighted recall, not just F1

The run: a laptop-sized version of the pattern

When not to copy this

How I'd start this week

Muse Wonder Just Landed: First muse-wonder-1.0 Call, Active Memory vs Active Focus, and Why max Isn't Wonder Full

What Meta actually shipped

First call (Chat Completions)

Cost arithmetic (list price, not a bill)

What I actually ran (offline)

The Full / API catch (and the memory catch)

How I'd try it this week

Why this matters

Microsoft-Decision-1 Just Landed: First systemone Call, Free Output Tokens, and the Qwen-to-MAI Rebase Catch

What it actually returns

What Microsoft is claiming

First request shape (Foundry-style)

Cost arithmetic (list price, not a bill)

What I actually ran (offline)

The Qwen-to-MAI rebase catch

How I'd wire it for an agent

Why this matters

Liquid AI's d1 Decision Models Went Open: Triage Support Tickets on a CPU With the 600M One (and Where It Fools You)

What Liquid AI is claiming

Run it yourself (CPU, about 10 minutes)

What happened on 12 tickets

Where it fools you: forced choices

How I'd actually wire it

Why this matters

Claude Haiku 5.5 Just Landed: Migrating a Haiku 4.5 Classifier Without the 400s (temperature, Prefill, Thinking Blocks)

What actually changed for a classifier

The smoke test: what I could check without an API key

On the cost math

Where it fits, and where something else might

Migration checklist

EmbeddingGemma 2 Just Dropped: Local Code Search on a CPU, the Install Gotcha, and Two Ways It Fails Silently

What shipped (vendor-reported)

The run: natural-language code search on a CPU

Gotcha 1: "text-only" still needs the image stack to load

Gotcha 2: slicing vectors without re-normalizing

Gotcha 3: float16 (per Google, not tested)

A small prompt-name note

When to reach for it

Checklist

Gemini Skills Just Landed in Workspace: Your First SKILL.md, Moving a Gem, and Why a Migrated Gem Isn't a Working Skill

What changed (and when)

The mental model: skills are loaded lazily

Your first SKILL.md (adapted pattern)

Pre-upload check (adapted pattern)

The trap: a migrated Gem is a draft, not a working skill

Checklist for moving a Gem early

Why this matters

Codex CLI's Big Refresh Just Landed: First Session, Parallel Agents, and Why /fork Isn't a Worktree

What actually changed (by release)

The mental model: the TUI is now a control plane

The trap: /fork is a chat fork, not automatically a checkout

First session checklist

Why this matters

Claude Code Mods Just Landed: First Hook, Blast Radius Guard, and Why They're Not Sandboxed

What a mod actually is (middleware, not a settings shell)

First-party features are already mods

Three example mods (vendor demos, not my harness)

Minimal Blast Radius core (adapted pattern)

Gotcha 1: reasoning is always on, and it eats `max_tokens`