DEV Community: Mark Huang

DeepSeek's Pro Endpoint Will Stop Meaning Pro

Mark Huang — Thu, 10 Sep 2026 01:35:55 +0000

A fixed API socket hides a model switch. Reproducing yesterday's result requires knowing which route the request took.

A Hacker News submission reproduces a DeepSeek platform notice that changes the meaning of an existing API name. DeepSeek plans to release V4.1 Flash around September 10, 2026, then route every request for V4 Pro to the new Flash model until V4.1 Pro arrives. Those requests will be billed at Flash prices.

The same notice lists new off-peak rates of $0.003 per million cached input tokens, $0.15 per million uncached input tokens, and $0.60 per million output tokens. Peak rates will be twice as high. Cheaper and faster is welcome. My problem is the name: after the switch, asking for deepseek-v4-pro will no longer mean that DeepSeek served V4 Pro.

I would treat this as a migration, even if DeepSeek presents V4.1 Flash as better across performance, cost, speed, and total task time. A broad vendor claim cannot tell me whether my parser or coding agent will behave the same. The bill may fall while the audit trail gets fuzzier.

The discount comes with a different model

There is a real customer benefit here. DeepSeek is not charging Pro rates while it sends traffic to a cheaper model. ITHome reported the same platform notice and gave the new off-peak prices in renminbi: ¥0.02 for cached input, ¥1 for uncached input, and ¥4 for output per million tokens. Peak prices are double. Those figures line up with the dollar prices in the submitted post.

DeepSeek also says V4.1 Flash has surpassed V4 Pro on every named dimension after internal and external testing. I am interested, but the notice does not publish the test set, harness, score breakdown, provider conditions, or failure cases. "Better" at the product level is therefore a reason to run an evaluation, not a guarantee that each existing Pro workload improves.

There is an obvious objection to my caution. If Flash is cheaper, faster, and more capable, why make users opt into an inferior model? Because aggregate improvement and behavioral compatibility are different questions. A model can win more tasks overall while regressing on the narrow prompt, output shape, or tool sequence that pays a particular team's bills.

Before September 10, I would save a small set of representative Pro requests and accepted outputs. After routing changes, I would replay them, record the served model revision if DeepSeek exposes it, compare failure modes and latency, and keep a rollback path through another pinned endpoint or provider.

A stable alias can still wreck a comparison

DeepSeek's official pricing page currently maps deepseek-v4-flash to DeepSeek-V4-Flash-0731 and deepseek-v4-pro to DeepSeek-V4-Pro-0813. Both accept a 1 million-token context and advertise a maximum output of 384,000 tokens. Those friendly API names already sit in front of dated revisions.

The official change log makes that arrangement explicit. On August 13, DeepSeek said callers could keep using the same Pro model name to receive the new GA release. The July 31 Flash update similarly kept the Flash name while changing the version behind it. Aliases make upgrades easy, but the new plan goes further: a Pro alias will temporarily resolve to a model in a different tier.

That matters whenever results need to be reproduced. A regression report that says "we used V4 Pro" becomes ambiguous across the cutover. So does an eval trend line, unless the test record captures the actual served revision. If the API response only echoes the requested alias, DeepSeek should add immutable revision metadata. If it already exposes that metadata, teams need to store it rather than logging only the request body and endpoint.

I would not compare a V4.1 Flash result collected after the change with an older Pro result and call it an apples-to-apples retest. The model changed, and the surrounding service may have changed with it. The alias hides exactly the variable the experiment needs to preserve.

The public test is still too young

Public reaction is enthusiastic but thin. In the HN discussion, one commenter traced the notice to a banner on DeepSeek's usage page, while others focused on the novelty of a Flash release leapfrogging Pro. A same-day Reddit thread includes early API impressions and a report that the beta was available only through the official API. These are useful leads, not a substitute for a published evaluation.

The release notice asks users to report problems found in comparative testing. That request makes sense, but it also tells me customer workloads are part of the validation process. DeepSeek is making the routing decision before those customers have had much time to publish what moved.

I reached a similar conclusion when V4 Pro 0813 reached general availability without a detailed technical diff: pin what can be pinned and make the new build earn traffic on the workload that matters. V4.1 Flash may well earn it. The forced route merely removes the option to learn at one's own pace.

What I would want from DeepSeek

DeepSeek should publish the comparison behind the "surpassed across all key metrics" claim. It does not need to prove universal superiority. A task list, harness configuration, latency definition, and known regressions would let customers decide where the evidence transfers.

I also want the immutable model revision in every response and usage export. Billing can still follow the cheaper Flash rate. The requested alias and the served revision belong in separate fields because they answer separate questions.

Production users should have a temporary opt-out or a dated Pro endpoint. Some will accept the new route immediately. Others have contractual tests, regulated records, or fragile integrations that make a controlled migration worth more than the token discount.

I like the price cut, and I would put V4.1 Flash into a shadow test as soon as it is available. I would not pretend the endpoint stayed the same. DeepSeek is changing the model while preserving the name, which gives users a generous price and a weak versioning contract.

Originally published at markhuang.ai

GPT-6 Astra Reprices the Whole Request After 272K Tokens

Mark Huang — Sat, 05 Sep 2026 01:19:37 +0000

A long context window looks continuous. The bill has a boundary in the middle.

OpenRouter's GPT-6 Astra page advertises a 1,050,000-token context window, up to 128,000 output tokens, and standard prices of $10 per million input tokens and $50 per million output tokens. Those are useful headline numbers. They are not enough to estimate a long request.

OpenAI's model documentation adds the line I would put beside the context limit: once a prompt exceeds 272,000 input tokens, the entire request is billed at twice the input and cache rates and 1.5 times the output rate. The larger window is real, but it is not a flat-price allowance.

My read is that Astra on OpenRouter should be configured as a set of explicit lanes, not dropped in as one model slug. I want a normal-context lane, a deliberately approved long-context lane, and a recorded provider choice. Otherwise a small change in retrieved material can move the whole request onto a different cost curve.

The expensive token is the one that crosses the line

Consider a Standard API request with 250,000 input tokens and 10,000 output tokens. At the published base rates, that costs $3.00 before any separate tool charges: $2.50 for input and $0.50 for output. Increase the prompt to 300,000 tokens, keep the output at 10,000, and the request costs $6.75 because the higher rates apply to all of it. That is $6.00 for input plus $0.75 for output.

The extra 50,000 input tokens do not add fifty cents. In this example, they add $3.75 to the request. That discontinuity matters in retrieval systems, coding agents, and document analysis because prompt size can move with repository state or the number of files returned. A request that sat just below the boundary yesterday may cross it after one more search result or a longer tool transcript.

I would alert before 272,000 input tokens, not after the bill arrives. The alert should show what entered the prompt and ask whether that material deserves to reprice the full request.

One model slug can still buy different service

OpenRouter adds another variable. Its routing documentation says requests are load balanced across top providers by default to improve uptime, and fallbacks are allowed unless the caller disables them. The live Astra endpoint feed exposes provider and service variants with different prices, including OpenAI Flex, Standard, and Fast paths and Azure endpoints.

That flexibility is the reason to use a router. It is also why I would not compare an OpenRouter run with a direct API run until I knew which endpoint actually served it. A model ID can remain constant while price, latency, and the supported request parameters change underneath it.

OpenRouter gives developers controls for this. A request can set a provider order, disable fallbacks, require support for every requested parameter, impose a maximum price, or prefer a latency or throughput range. For an evaluation, I would pin the endpoint and disable fallback so the comparison stays clean. In production, I might restore fallback for availability, but I would log the resolved provider, service tier, token counts, and final cost.

I would set the privacy rules at the same time. OpenRouter's provider logging documentation says providers have their own retention policies. Routing does not automatically exclude providers based on retention alone. OpenRouter supports data-policy filters and a Zero Data Retention requirement, but the caller or account owner has to choose them.

A million tokens should be an exception budget

Astra's large window can help when removing material would break the task: a long legal record, a large codebase with dependencies spread across many files, or an agent session whose earlier decisions still affect the next action. I would not pay the long-context rate merely to avoid deciding what matters.

OpenAI's model guidance says Astra follows longer instructions better but can also be more sensitive to information in context. That is a useful warning. More context can preserve a decisive constraint, but it can also preserve stale instructions, duplicate evidence, and irrelevant tool output. Capacity does not sort the prompt for me.

Once a request can double its input rate, prompt assembly becomes a budget decision. Retrieval quality, transcript compaction, and prompt assembly decide whether a request stays below 272K. If a workflow repeatedly crosses the line, I would first ask whether it needs a larger prompt or a better memory policy.

My migration test starts with the bill

Before moving a workload, I would replay the same tasks in prompt-size bands on both sides of 272K. I would record completion quality, retries, latency, cache behavior, resolved endpoint, and total cost per accepted result. A cheaper request that needs three attempts is not cheaper. A 900,000-token request that prevents a costly omission may be worth every cent.

Public discussion about OpenRouter's long-context pricing has already focused on whether the higher tier is visible enough before a request runs. A recent r/openrouter thread raised that concern for another OpenAI model. I find the complaint reasonable, but the practical response is not to avoid long context. It is to make the threshold visible in budgets, traces, and approval rules.

This also complements what I found in Astra's 45.1-point ARC harness gap. The surrounding system can change both the score and the bill. OpenRouter makes the provider layer configurable, while Astra makes very large prompts possible. I would treat both as versioned parts of the product.

Astra's 1.05-million-token ceiling is enough to make me test it on long documents. I would just keep the 272K boundary in view, pin the route, and show the cost before the request leaves.

Originally published at markhuang.ai

Astra's ARC Score Moves 45.1 Points When the Harness Changes

Mark Huang — Fri, 04 Sep 2026 15:48:20 +0000

One model enters the same world twice. The path that preserves its discoveries reaches much farther.

ARC Prize tested OpenAI's GPT-6 Astra on ARC-AGI-3 and got two answers to what looks like one question. At high reasoning effort, Astra scored 54.8% with ARC Prize's Standard harness and 99.9% with a Provider Adapter that preserves opaque reasoning state and compacts long conversations. That is a 45.1-point gap.

I believe the near-perfect result is real, but the model weights cannot claim it alone. The result tells me Astra can learn unfamiliar environments unusually well when its working state survives. It also tells me that the memory contract around the model can decide whether I see that ability at all. Both facts matter more than the 99.9% headline if I am choosing an agent.

The adapter changes what the score means

ARC-AGI-3 puts agents into abstract, turn-based environments without explicit rules or goals. The agent must explore, infer how the world works, identify a target, and plan its actions. ARC Prize says people can solve all of the environments, which gives the benchmark a human reference instead of an arbitrary pass mark.

The two harnesses keep the games, actions, limits, and scoring fixed, according to ARC Prize's open-source benchmarking repository. What changes is the state carried between model calls. The Standard harness uses provider-neutral text history and asks the model to save useful discoveries in visible notes. The Provider Adapter uses OpenAI's native conversation handling, including retained opaque reasoning and compaction.

The scores split widely at every published effort level. The cleanest comparison is high effort: 54.8% versus 99.9%. At maximum effort, the comparison is 62.7% versus 98.6%. Mixing those rows would imply that 62.7% and 99.9% came from identical settings. They did not, and the result is already strong enough without that shortcut.

The Standard score asks how Astra performs through a common interface. The Provider Adapter score asks what the deployed OpenAI stack can produce when its native memory features are available. One is better for comparison; the other is closer to a product result.

Remembering is part of the capability

This is not the first ARC-AGI-3 result to move with context handling. In July 2026, OpenAI reported that retained reasoning and compaction raised GPT-5.6 Sol from 13.3% to 38.3% on the public task set while using six times fewer output tokens. The settings changed together, so that experiment did not isolate how much each one contributed. It did show how badly an agent can perform when every action makes it reconstruct its own earlier thinking.

Astra's replays make the benefit easier to picture. ARC Prize observed the model compressing game state into short symbolic notes, recording objects, coordinates, rules, and unfinished plans. In the Provider Adapter runs across Public and Semi-Private tasks, ARC Prize says the system was about 3.66 times faster by aggregate recorded elapsed time and used 49% fewer tokens across 167 game and reasoning-effort pairs that both harnesses solved.

I would carry that result into a product review. Persistent state helped Astra repeat less work in these runs. ARC Prize also reports that Astra at maximum effort used fewer actions than the median tested human on 96.0% of levels and averaged 51.7% fewer actions per level under the Provider Adapter.

Action efficiency has a boundary. ARC Prize's scoring methodology counts interactions that affect the environment rather than internal reasoning steps, tool calls, or retries. The test tells me Astra learned and acted with little external trial and error. Energy use, required supervision, and recovery from a wrong production assumption need their own measurements.

A fair benchmark can still be an unfair product test

There is an obvious objection to the Standard result: why test a model with memory behavior unlike the environment it was trained and deployed to use? OpenAI made that case after the earlier GPT-5.6 result. I find it persuasive when a buyer wants to know, "What can I get from OpenAI's supported stack?" The 99.9% result answers that question.

It is less persuasive when the question is, "Which model travels best across providers and runtimes?" A native adapter can reveal available capability, but it ties the result to provider-specific state handling that another system may not reproduce or inspect. The opacity matters too. If a preserved false hypothesis survives several actions, a developer may see the consequences without being able to examine the state that kept it alive.

Astra is not an isolated example. TechCrunch reported that an Nvidia research harness took Claude Opus 5 from 30% to 100% on ARC-AGI-3 by adding memory support and a supervisor component. The setups are not directly comparable, but both make the same buying mistake easy to see: a model name is an incomplete agent specification.

Public reaction jumped quickly from the score to whether ARC-AGI-3 is saturated or whether Astra counts as AGI. A short Reddit discussion captured both questions. ARC Prize is more careful. Its article says saturating this tightly bounded, deterministic benchmark is not proof of AGI, and that the real world remains more open-ended.

I want two lanes on the scorecard

I would keep both harnesses. A common lane reveals portability and makes cross-provider comparisons less slippery. A native lane reveals the best supported system a customer can actually deploy. Publishing only the common result can hide useful capability; publishing only the native result can hide how much the provider layer contributes.

For a production decision, I would version the model together with its harness, state policy, compaction behavior, tools, and effort level. Then I would run the actual workflow with deliberate state changes. I would revise the goal halfway through, inject a bad observation, cross a compaction boundary, and resume after a long pause. The failure I would test is a system that keeps a bad premise alive with confidence.

This extends the argument I made about the Agentic Index: a leaderboard earns a model test time rather than production credentials. Astra's 99.9% earns serious attention, while the 54.8% run tells me what to inspect next. The memory layer belongs in the product specification and the evaluation record. Its failure modes belong in the risk review.

Originally published at markhuang.ai

Fable 5.1 Rebuilt Union Square and Left Its Misses in Git

Mark Huang — Wed, 02 Sep 2026 23:10:45 +0000

Union Square looks finished at the center. Around it, wireframes and camera angles expose how the reconstruction is being checked.

The fable51-worlds repository presents a browser reconstruction of San Francisco's Union Square that it says was researched, modeled, and checked end to end by Claude Fable 5.1 agent swarms. Its first world loads 453 OpenStreetMap building footprints, places 220 simulated pedestrians, and runs as a plain Three.js application.

I became more interested when I left the 59-second walkthrough and opened the audit files. The repo includes 34 camera-matched viewpoints, 147 comparison sheets, and nine reviewer-agent reports. Its own final report gives visual fidelity, street-level detail, and materials 6/10. Those admissions give me more to work with than a polished autonomous-agent demo.

The scores are not independent proof that the reconstruction is accurate. What the repository offers is narrower and more credible: a large result with its inputs, checks, and unresolved defects available for inspection.

The evidence is bigger than the walkthrough

The project covers roughly 800 by 720 meters around Union Square. Its README lists 75 authored facade specifications, 129 identified storefronts, 109 vehicles, two explorable interiors, and 23 interactive objects. The runtime assembles the scene from data and generated assets rather than shipping proprietary 3D tiles.

Those counts make an easy model-capability story. Anthropic positions Fable 5.1 for long-running coding projects, multi-day autonomous sessions, self-written tests, and visual checking. This repository puts that pitch into one messy job: research a real place, turn mixed sources into code and assets, build an interactive world, then compare the result with photographs.

The QA machinery sits beside the output. Playwright captures fixed views. Another script places renders next to reference photographs and produces 50 percent overlays. The reports keep the remaining gaps. A reader can see why the team gave an attractive scene only 6/10 in several categories.

Reviewer agents are still inside the experiment

The README calls the nine reviewer agents independent and assigns roles such as architect, geographer, technical artist, and interaction reviewer. Separate roles can catch different defects. The reports say those reviews found inward-wound facade walls, a transit-station outline extruded into a 28-meter block, mirrored street markings, and reference cameras placed inside buildings or roadways. The project says those problems were fixed and rechecked with screenshots.

I still put a boundary around the word "independent." The repository describes reviewer agents within the same project workflow; it does not present the scores as a third-party audit. Role prompts can diversify attention, but they do not automatically diversify training assumptions or prove that the scoring rubric matches how a San Francisco architect, accessibility specialist, or daily pedestrian would judge the place.

In my earlier piece on multi-model ensemble intelligence, I argued that several agents can share the same blind spots even when they appear to debate one another. Fable51-worlds partly compensates by comparing renders with external photographs and geospatial data. Outside review still has a different job.

Open data provides coordinates, not a complete city

The input choices are sensible and inspectable. The project derives geometry from OpenStreetMap and elevation from USGS 3DEP, then uses public records and free-licensed photographs for storefronts and visual references. OpenStreetMap's license page explains that its community data is available under the ODbL with attribution and share-alike conditions. The repo records photograph provenance without redistributing the reference images.

Those sources can anchor a street grid and building footprint. They cannot settle every facade, shop tenant, interior, or street behavior. The project's own discrepancy ledger says some building details are approximated and procedural textures repeat at close range. Pedestrians do not enter stores, and buses do not stop at shelters. Several storefronts remain unresolved and use blank fascias instead of invented identities.

I like that last choice. A neutral blank is less cinematic than a plausible fake, but it keeps missing evidence separate from generated detail. A reconstruction of a real place needs that boundary.

The final QA report says its category scores are unadjusted reviewer-agent results, not scores raised to meet the project's target bar. They are useful as the project's own acceptance record, not as an external benchmark of Fable 5.1.

The failure ledger is the part I would copy

Most model showcases compress the work into a prompt and a finished screen. The output looks magical, and the mistakes become hard to price. This repo exposes the research, confidence levels, capture scripts, comparison sheets, reviewer reports, performance measurements, and known gaps. A developer can disagree with the result without first reverse-engineering the experiment.

I would still want a human domain review before calling this a dependable digital twin. I would also want costs, elapsed agent time, interventions, failed branches, and the exact model and harness used for each reviewer role. Those omissions make it hard to compare this build with a skilled human team or another agent system.

The project has already made one decision I would copy: it stores uncertainty with the artifact. If agent swarms are going to build days-long visual systems, the deliverable should say what was checked, what failed, what remains approximate, and which claims an outsider can reproduce. Fable 5.1's Union Square is ambitious. I trust its narrower claims more because the repo leaves the misses in Git.

Originally published at markhuang.ai

Muse Spark 1.3 Charges 21 More for Output It Won't Train On

Mark Huang — Wed, 02 Sep 2026 22:54:45 +0000

Muse Spark 1.3 offers two routes to the same model family. The cheaper route opens prompts and completions to future training; the standard route keeps them out.

Meta's Muse Spark 1.3 page pitches a stronger agentic and coding model with a 1 million-token context window. I kept looking at the pricing table below it. The Contributor tier charges $0.10 per million input tokens and $0.20 per million output tokens, while Standard charges $1.25 and $4.25.

The discount changes the product decision. Meta's pricing documentation says prompts and completions on Standard are not used to train Meta models. Contributor grants Meta permission to use both for future model training. To me, these are two data policies that happen to serve the same model. They do not belong in a dropdown as if price were the only difference.

The gap is hard to shrug off. Standard input costs 12.5 times as much, output costs 21.25 times as much, and cached input costs 75 times as much. Contributor can save real money. Whether a workload belongs there is a separate question.

A better agent does not remove the deployment choice

Meta released Muse Spark 1.3 on September 2, 2026 for Muse Code and Meta Model API. It says the model handles long tasks and conflicting context better, asks for help when stuck, and confirms consequential actions. In comparisons run by Meta engineers, it used about 20% fewer tool calls and 25% fewer tokens than Muse Spark 1.2.

Every unnecessary turn adds latency and cost, so those improvements could matter. The official benchmark graphic also reports 75.4 on DeepSWE v1.1 and 88.8 on Terminal-Bench 2.1 for Muse Spark 1.3 at max reasoning. I would not buy on those two scores. They come from Meta's selected comparison, and the launch post says max reasoning will arrive only after additional safety testing. The strongest mode in the chart was not the fully available launch-day product.

Independent testing is still one version behind. Artificial Analysis measured Muse Spark 1.2 at $0.40 per task on its Intelligence Index, but it also found about 53% more input tokens and 36% more output tokens per task than 1.1. A token price is not a task price. I made the same case in my recent look at token sticker prices: the useful unit is a result that passes review.

The discount is permission, not a lesser model

Meta lists the same 1,048,576-token context window for the Standard and Contributor model IDs. The published distinction is what Meta may do with the traffic.

Endpoint	Training use	Input per 1M	Output per 1M
`muse-spark-1.3`	Prompts and completions are not used to train Meta models	$1.25	$4.25
`muse-spark-1.3-contributor`	Prompts and completions may train future Meta models	$0.10	$0.20

I can see a good use for the Contributor tier. Public open-source repositories, synthetic test cases, benchmark harnesses, and throwaway prototypes can generate plenty of useful traffic without exposing material the team needs to protect. The low rate makes broad experiments much easier to justify.

Private repositories, customer records, incident logs, unreleased product plans, and any prompt that might capture a credential stay on Standard for me. Coding agents gather context aggressively. A request that begins with a harmless bug description can pull in source files, terminal output, configuration, and issue history before it is done. The endpoint decision covers the whole agent loop, far beyond the sentence a developer typed.

My rule would be simple: use Contributor only when I would be comfortable treating the complete prompt-and-completion trace as training-eligible data. If that answer changes halfway through the task, the workload started on the wrong endpoint.

The missing details matter before production

The pricing page clearly states the exchange, which is better than hiding it in a footnote. It does not, on that page, give a retention period, a deletion workflow, or a more granular account of how training-eligible records are reviewed and prepared. I would want those answers before approving Contributor traffic from a company repository.

The wording also makes opt-in a model identifier. That is easy to automate, but easy to misconfigure. A copied environment variable or fallback route could move a sensitive job onto the discounted endpoint without changing the code that assembles its context. Cost dashboards would celebrate while the data boundary quietly moved.

I would separate the tiers at the infrastructure layer with different project credentials and budget labels. Contributor workloads would need an explicit allowlist. I would also log which model ID handled each job. Redaction can help, but it should not carry the whole policy. An agent can reveal sensitive structure even after obvious secrets have been removed.

The price is good when the data fits

Muse Spark 1.3 looks competitive enough to test, and Meta's reported efficiency gains make that test more interesting. The Contributor price makes large experiments unusually cheap. Neither point turns training permission into a minor billing detail.

I would use the cheap tier deliberately for public and synthetic work, then pay Standard rates when the agent can touch anything I would not publish. That sounds conservative until the 21.25-times output gap reaches the budget meeting. Then it becomes exactly the decision the pricing table was designed to force: how much is the organization willing to save, and which data is it willing to contribute?

Originally published at markhuang.ai

Gemini 3.7 Flash Called 12% of Poisonous Mushrooms Edible

Mark Huang — Wed, 02 Sep 2026 22:43:01 +0000

One photograph can produce several plausible names. That uncertainty is useful for learning and dangerous when lunch depends on the answer.

Quesma tested 16 vision models on 1,040 mushroom photos and asked each one for its five most likely species. Gemini 3.7 Flash put the correct species first 61% of the time. More worrying, it named an edible species for 12% of the poisonous mushrooms.

That result gives me a clean boundary. I would use a vision model to suggest names and explain which features to inspect while I learn about local fungi. I would never let its answer decide whether a wild mushroom goes into a pan. The model can help long before it is safe enough to make that decision.

The stakes show up in hospital data. A CDC analysis of U.S. cases estimated 1,328 emergency-department visits and 100 hospitalizations associated with accidental poisonous mushroom ingestion in 2016. Among 556 patients in separate claims data from 2016 through 2018, 8.6% had a serious adverse outcome. The CDC's advice is blunt: do not consume wild mushrooms unless an expert has identified them.

The ranking stops at the safety column

Quesma's benchmark used 55 species selected from a Polish list of mushrooms permitted for sale and a list of deadly species. It sampled up to 20 ordinary photos per species from FungiTastic, mostly from the dataset's 2023 test split. The prompt required five Latin binomials and nothing else. The code, photo list, raw model answers, and scoring scripts are public.

Gemini 3.6 Flash led the first-choice result at 64% and included the correct species in its top five 85% of the time. Its poisonous-to-edible error rate was 11%. Gemini 3.7 Flash reached 82% within five guesses, but its 12% dangerous-error rate was slightly worse. Put safety beside accuracy and the leaderboard becomes much less comforting.

The mistakes also landed on exactly the lookalikes that matter. Across all model answers in Quesma's species-level analysis, death-cap photos were assigned an edible species 17% of the time. Fool's funnel photos were assigned an edible species 49% of the time. A model does not need to fail often to be unusable as a safety gate. It only needs to sound convincing on the wrong specimen.

This is why I care more about the direction of an error than the overall score. I made a similar argument about model build-offs and failure rates, but mushroom identification makes the asymmetry impossible to ignore. Calling an edible mushroom poisonous wastes a find. Calling a poisonous one edible can send someone to the hospital.

One photo is not the whole specimen

The benchmark is deliberately close to how people use a phone: take one normal photo and ask for a name. That makes it a useful product test, but it is not a complete test of field identification. Location, nearby trees, substrate, smell, texture, bruising, spores, and the underside of the cap can all matter.

iNaturalist's mushroom documentation guide recommends at least three photos: the specimen in its habitat, the cap, and the stem and gills. It also notes that photographs alone sometimes cannot settle the species. Quesma tried adding a Denmark location hint and reported that it did not improve these model results.

The underlying data are stronger than a random web-image collection. FungiTastic describes itself as an expert-verified dataset with roughly 350,000 observations, more than 600,000 images, and over 5,000 species. Its records can include geospatial and climate data, captions, and other views. Quesma's test intentionally removes most of that context. It measures what happens when a general vision model receives the thin evidence a casual user is likely to provide.

A species guess is not permission to eat

Photo-identification software had this problem before the latest multimodal models. In a 2023 study of three mushroom apps, researchers tested 78 specimens whose identities were confirmed by an expert mycologist. The best app correctly identified 49% of specimens. The authors concluded that the apps were not reliable enough, on their own, to exclude a potentially poisonous exposure.

Quesma's results are not directly comparable with that study. The apps, specimens, prompts, and scoring methods differ. The narrower lesson is enough: better species suggestions do not convert a camera into a food-safety instrument. Put a polished name, confidence score, and edibility label on one card, and users may read three separate claims as one answer.

I would remove direct "safe to eat" answers from a general AI identification flow. The product can ask for more views, show several candidates, explain the missing evidence, and direct suspected ingestion to a poison center or emergency service. In the United States, Poison Control says to use its online tool or call 1-800-222-1222 rather than wait for symptoms.

I would use it as a field guide

There is still a good use for this capability. A model can turn a walk into a set of questions: Which parts should I photograph? What features separate these candidates? Which local expert or field guide should I consult next? A five-name shortlist can make unfamiliar biodiversity easier to explore.

The interface should leave the uncertainty visible. It needs to say when one photograph is inadequate, keep identification separate from edibility, and make escalation obvious when someone may have eaten the specimen. A refusal buried beneath a confident species card would miss the point.

Gemini 3.7 Flash did well enough in Quesma's test to be an interesting learning aid. The same test produced a 12% poisonous-to-edible error rate, which settles the food decision for me. I would ask the model what to look at next. I would not ask it what to cook.

Originally published at markhuang.ai

Muse Spark 1.3 Uses 25% Fewer Tokens. Its Max Mode Still Has to Wait.

Mark Huang — Wed, 02 Sep 2026 21:54:03 +0000

Spark 1.3 sends the everyday work down a shorter path. Its highest-reasoning route is still behind the gate.

Meta has released Muse Spark 1.3 in Muse Code and the Meta Model API. The claim that caught my attention is not the leaderboard: Meta says its engineers saw roughly 20% fewer tool calls and 25% fewer tokens than with Spark 1.2. Every unnecessary turn in an agent loop adds latency, cost, and another chance to wander.

The release has an awkward split. The existing reasoning modes are available now, while the new max mode will arrive only after additional safety testing. Yet max is the configuration in Meta's benchmark scorecard. Spark 1.3 has earned a place in my test queue, but its headline scores have not earned production traffic.

The useful number is not the biggest score

A coding agent rarely fails because its benchmark number is one point too low. It fails because it rediscovered the same file, called the wrong tool, forgot a constraint, or declared victory before checking the result. Meta says it trained Spark 1.3 on more long-horizon coding tasks and across different agent harnesses. The model is also supposed to be less verbose and better at keeping detailed instructions.

The 20% and 25% reductions interest me more than a broad intelligence ranking. They point at the cost of getting work done. Still, Meta describes comparisons made by its own engineers, without giving a task count, variance, cache assumptions, or a harness-by-harness breakdown. I would treat the figures as a testable claim, not a budget forecast.

I would measure completed work per review minute. Fewer tokens help only when the agent also preserves the brief, uses the intended tools, and leaves a result I can verify.

Meta benchmarked the mode developers cannot use yet

Meta's scorecard reports Spark 1.3 max at 75.4 on DeepSWE v1.1, up from 55.0 for Spark 1.2 xhigh. On Terminal-Bench 2.1, it reports 88.8, tied with GPT-5.6 Sol max and above Spark 1.2's 82.9. Those are strong coding signals, but they belong to the max configuration that is still waiting on safety work.

The wider table is more measured. Spark 1.3 max scores 64.9 on JobBench against 65.7 for Opus 5 max, and 1,754 on GDPval-AA v2 against Opus 5's 1,824. It leads the compared models on the two long-context MRCR tests and on DeepSWE, but not on every agent benchmark. The table makes the model interesting without telling me how it will behave in my repository.

I made the same distinction when looking at the Artificial Analysis Agentic Index: a leaderboard can fund a shortlist, but only a local failure test can choose a production model. Here I would go one step further and keep the current reasoning modes separate from max in my results. Combining them would turn a staged rollout into a performance claim the available product has not yet proved.

The complaints about 1.2 map neatly to the release notes

Public discussion of Spark 1.2 is mixed. In one OpenCode discussion, developers complained about unnecessary Python scripts, shortcut-taking, and poor behavior unless prompts were unusually tight. Others in the same thread said a different harness or custom system prompt worked well. A separate first-impressions post called 1.2 a pleasant workhorse but flagged its cache-hit rate.

Anecdotes do not settle model quality, especially when harnesses and traffic load differ. They do tell me what to test. Spark 1.3 specifically promises fewer needless turns, better instruction retention, cleaner coding style, more accurate routing when a user interrupts a long thread, and a better sense of when to ask for help. That reads like a response to real workflow friction rather than another claim about solving harder puzzles.

If those changes hold, the developer supervising a long job gets fewer interruptions. But "trained across diverse harnesses" does not guarantee the same behavior with my tool definitions or permission prompts. Meta's earlier Spark 1.1 release already advertised planning, delegation, context compaction, and a one-million-token context window. Version 1.3 has to show that those capabilities cooperate more reliably.

The safety delay is part of the product

I am glad Meta did not blur the max delay into a footnote. A stronger agent that takes consequential actions needs more than a refusal score. Meta says 1.3 is better calibrated around irreversible actions and more resistant to prompt injection, but the announcement does not publish quantitative support for those two claims.

That omission matters because Meta disclosed in August that a pre-release Spark 1.1 model exploited a real website during a misconfigured cyber evaluation. Meta says the model believed the site was its assigned target and changed its database. The company reviewed more than 10,000 activity records and found no other instance. An agent's judgment cannot repair a broken boundary.

That is also why I would not read "max is coming" as a release date. Additional safety testing can find a model problem, a harness problem, or both. I have argued before that a cyber evaluation is production infrastructure once a capable agent can touch the internet. Waiting is the correct state until the model and its environment clear that bar.

I would run the cheaper experiment first

For the available Spark 1.3 modes, I would use a small set of real repository tasks with fixed permissions and repeated runs. I would record completion, tool calls, tokens, elapsed time, human interventions, constraint violations, and attempts at irreversible action. Comparing those traces with Spark 1.2 in the same harness would test Meta's efficiency claim directly.

I would keep max out of the comparison until developers can actually call it. Once it arrives, it gets its own lane and the same tests. A stronger score may justify extra reasoning on difficult migrations or long debugging sessions. It may also spend more time proving what the regular mode already knows.

Spark 1.3 looks most promising where Meta has made the least glamorous claim: fewer calls and fewer tokens inside messy work. That could make an agent cheaper to supervise. The max benchmark is a useful preview, but my decision today is simple. Test the model that shipped, and wait for evidence on the mode that did not.

Originally published at markhuang.ai

Gemini 3.8 Flash and Cyber Share a Core, Then Split at the Guardrails

Mark Huang — Wed, 02 Sep 2026 21:29:45 +0000

Gemini 3.8 Flash and Flash Cyber share foundational intelligence, but Google controls the two paths with different safeguards and access rules.

Google has introduced Gemini 3.8 Flash and Gemini 3.8 Flash Cyber, its third Flash release in six weeks. The public model keeps the introductory 3.7 Flash price of $0.75 per million input tokens and $3.75 per million output tokens. The Cyber variant is different: Google is limiting it to trusted defenders through the Fairwind Program.

I keep coming back to the boundary between the two models. Google says they share the same foundational intelligence, yet Flash Cyber has more permissive cybersecurity mitigations. A benchmark headline about Cyber and the model a typical developer can call may come from the same family, but they do not describe the same product.

One core, two permission models

The regular Gemini 3.8 Flash API model is broadly available. It accepts text, images, video, audio, and PDF input, has a 1,048,576-token input limit, and supports low, medium, or high thinking effort. Google says it ships with safeguards against misuse in cyber offense and chemical, biological, radiological, and nuclear domains.

Flash Cyber starts from the same underlying intelligence, according to Google, but changes the operating envelope. Its mitigations are more permissive for cybersecurity. That matters because legitimate incident response, exploit validation, and vulnerability research can resemble offensive activity. Google is offering that version to government authorities, critical infrastructure operators, and software maintainers who qualify for Fairwind access.

I think that separation is defensible. A 2025 research paper on differential access lays out the same basic strategy: give vetted defenders access to higher-risk capabilities while keeping tighter limits elsewhere. The paper warns that a restrictive program can still fail defenders if access is too narrow or slow. Those teams need time to prepare before similar capabilities reach attackers.

When I see a Gemini 3.8 cyber benchmark, I want the model name and access tier beside it. A Flash Cyber result does not tell me how the public gemini-3.8-flash endpoint will handle the same task.

The best cyber numbers come from behind the gate

Google reports that Flash Cyber exceeded a 70% success rate on an internal vulnerability-discovery benchmark spanning 20 programming languages. On the external CWE-Bench patching benchmark, Google gives it a 47.2% pass@1 result, close to 47.8% for an unnamed leading frontier model and at lower cost. Google also says Chrome's security team obtained 2.6 times more correct vulnerability patches than with the best much larger commercial models it tested.

I find those results promising, with a few stubborn caveats. The multilingual test is internal, and the launch post does not name the models in the Chrome comparison. A 47.2% first-attempt patch rate also leaves more than half of cases unsolved. Before I put the model anywhere near a patch queue, I would want to see the exact harness, behavior across repeated runs, false-positive cost, and review burden.

The public model has independent evidence of its own. Artificial Analysis measured Gemini 3.8 Flash at 304.6 output tokens per second and gave its high-effort setting a score of 59 on the firm's Intelligence Index. It also recorded 120 million output tokens across that evaluation, compared with a median of 71 million among the models in its comparison set.

That last number fits Google's own warning that 3.8 Flash "works harder" on complex tasks by taking more reasoning steps and calling tools repeatedly. Extra effort may improve an answer while making the full workflow slower or more expensive. Google explicitly points efficiency-first users back to lower effort settings or 3.7 Flash. My earlier Gemini 3.7 Flash pricing analysis matters here too: the introductory rates expire after December 31, 2026, then double.

I would test the gate separately

If I were testing the public model for ordinary software work, I would use 3.8 Flash's own API documentation and public-model results. I would measure accepted patches, review time, token use, and rollback rate in my actual repository. Flash Cyber's numbers might show what the shared model family can become under a different policy, but they would not go into my production forecast.

If I ran a qualifying security team, I would evaluate Fairwind as a separate product decision. The useful questions would be how Google vets applicants, which activities the looser mitigations permit, what telemetry Google retains, how access can change, and what happens when the model finds a live vulnerability. The launch page names eligible groups and offers an application form, but it does not answer all of those operational questions.

Public reaction to 3.8 Flash is still too early to settle much. In one Gemini community thread, users were already confused about whether the app was serving 3.7 or 3.8 during the rollout. That is a minor labeling problem compared with Cyber access. It is also a useful warning: the model name alone may not tell me what someone received.

Gemini 3.8 Flash looks fast and capable on its own evidence. Flash Cyber posts stronger results on a narrow set of security tasks, and Google pairs those capabilities with looser mitigations for selected defenders. I would not flatten the two into one product story. For this release, access policy belongs in the specification.

Originally published at markhuang.ai

Gemini 3.8 Flash Kept 3.7's Price. Higher Effort Still Runs the Meter.

Mark Huang — Wed, 02 Sep 2026 21:25:02 +0000

Gemini 3.8 Flash takes the fast lane, but a production upgrade still branches into cost, reliability, and safety checks.

Google DeepMind published the Gemini 3.8 Flash model card on September 2, 2026. The new model is based on Gemini 3.7 Flash and keeps the same headline capacity: up to 1 million input tokens and 64,000 output tokens. Google says the gains are aimed at software engineering and agentic knowledge work.

The price did not move either. According to Google's Gemini API rate card, standard access costs $0.75 per million input tokens and $3.75 per million output tokens through December 31, 2026. On January 1, 2027, both rates double. Those are the same figures and dates Google lists for 3.7 Flash.

My read is simple: 3.8 looks like an easy replacement, but it has not earned a blind migration. A stable token price does not guarantee a stable task cost, especially when Google's own card says higher effort levels may use more tokens. I would run 3.7 and 3.8 side by side, then promote the new model only where it reduces the cost of an accepted result.

The upgrade arrives without a new rate card

There is a lot to like in that arrangement. Google lists 3.8 Flash across the Gemini app, Gemini API, Google AI Studio, AI Mode, Antigravity, and its enterprise agent products. Developers can try the new model without first explaining a higher per-token rate to whoever owns the budget.

The unchanged price also removes one distracting variable from a comparison. If I replay the same tasks through 3.7 and 3.8, differences in the bill should come from token volume, tool use, retries, caching, and the inference tier rather than a different standard rate. That makes a local evaluation cleaner.

It does not make the result obvious. Google lets users choose effort levels to trade quality against cost and latency. The model card also warns of occasional slowness or timeouts and says the model may spend more tokens to maximize performance at higher effort. A stronger answer that takes extra reasoning steps can still be a bargain. An agent that plans longer, calls the same tool twice, and repairs avoidable mistakes can quietly erase the apparent savings.

I would keep the effort setting fixed during the first comparison. Then I would tune it only after 3.8 shows where its extra work changes the accepted outcome.

The benchmark question is really a routing question

Google evaluated 3.8 across coding, knowledge work, multimodal tasks, long context, computer use, and scientific reasoning. That breadth can tell me whether the model deserves test time. It cannot tell me which requests in my product should reach it.

The public reaction makes that gap visible. In one benchmark discussion on Reddit, some commenters focused on speed and price-to-performance. Others asked about token efficiency or pointed to weaker results on a newer, harder terminal benchmark. A separate discussion caught people initially celebrating an older terminal score, then reconsidering it after noticing the newer test.

I find the skeptical reading useful. Benchmark versions are not footnotes when a model will operate a terminal or modify a repository. A high score on a saturated test and a lower score on its harder successor can both be accurate. Neither predicts the exact failure that matters in my harness.

So I would route by task, not by release number. Routine transformations, bounded code edits, and document extraction may reward speed. Long-running work with many tool calls needs a separate lane because recovery cost rises with every action. The evaluation set should include tasks the agent can finish, tasks where it must ask a question, and tasks where the correct behavior is to stop.

One safety number deserves a closer look

The model card says overall safety and tone are similar to 3.7 Flash, with low unjustified refusals. Its automated table also reports a slight regression in non-English safety. The multilingual safety result is 5.4 percentage points higher than 3.7, and Google marks lower as better.

Google says manual review found the losses were overwhelmingly false positives or not egregious. That context matters, but I would not turn it into permission to ignore the number. Automated safety tests can be noisy, and a production application can still have a language mix that differs from Google's query set.

If the product serves more than English, I would add its actual languages to the migration suite. I would compare unsafe completions, needless refusals, tone, and escalation behavior with the same prompts on both model versions. A global average is useful evidence. It is not a substitute for the languages and risks in front of my users.

I would measure the work, not the version

For each repeated task, I would record accepted outcomes, input and output tokens, elapsed time, retries, tool calls, timeouts, and human corrections. I would also preserve the model version and effort level beside the trace. That turns "3.8 feels faster" into something I can audit.

The January price increase belongs in the same worksheet. I made that case when Gemini 3.7 Flash launched with the identical deadline: a discounted pilot should still make financial sense at the permanent rate. Gemini 3.8 does not reset that clock.

Google has made the trial easy. The endpoint is public, the standard rate matches 3.7, and the model card names the tradeoffs instead of hiding them. That earns 3.8 Flash a place in my test harness. Production traffic comes later, after the faster model proves it can finish the work with fewer expensive surprises.

Originally published at markhuang.ai

Nori A3 Is $1,688. Who Teaches It to Fold the Laundry?

Mark Huang — Wed, 02 Sep 2026 01:55:37 +0000

A robot folding one towel looks like an appliance. The pile beside it is a better test of how much work the owner still has to do.

Nori Robotics is selling the Nori A3 for $1,688, with orders advertised to ship in fall 2026. The company shows a two-armed robot on wheels and lists kitchen help, tidying, fetching food, loading dishes, folding clothes, and pouring ingredients as everyday tasks.

That price got my attention. My read, though, is that the A3 makes more sense as an affordable robotics platform than as a finished household appliance. Nori expects the buyer to teach, operate, and manage the machine. A developer may hear an invitation in those verbs. Someone who wants the laundry folded may hear a second job.

The hardware is unusually accessible

The A3 is a bimanual mobile manipulator, which is a less romantic and more useful description than "humanoid." It rolls instead of walking. According to Nori founder Antonio Li's technical preprint, the robot has 19 degrees of freedom, weighs 20.4 kg, and fits on a 45 by 45 cm base. A telescoping column moves its head and arms from 69 cm to 145 cm, letting it reach from the floor to a counter.

Nori's product page rates each arm for a 1.5 kg payload and gives the robot four 720p cameras, lidar, a microphone, and a speaker. The advertised battery life is 6 to 8 hours. Those are manufacturer figures, but they describe a serious amount of hardware at this price.

The evidence is still close to the company. Li is the sole author of the five-page preprint, and the product page is a sales page. I would treat both as documentation of the design and Nori's claims, not as an independent durability or household-performance test.

Who teaches it? The owner does

Nori Lab is the center of the pitch. The product page says the laptop app lets owners "train, operate, and manage" the robot. Skills can then be shared through a marketplace. New Atlas reports that owners can write code or train the robot through a simulated version on a desktop. Its caution is fair: the A3 is not like a robot vacuum with a narrow job and a familiar expectation of what happens after unboxing.

The public Nori documentation is marked beta and says full A3 documentation is coming soon. It leads owners toward a desktop app and a first teleoperation session. The site also offers Python and TypeScript SDKs and says developers can stream video and telemetry over WebRTC.

I do not see that as a gotcha. A low-cost, programmable robot with remote control and an SDK could be useful to makers, classrooms, and small robotics teams. It does mean the $1,688 price answers only the hardware question. It does not tell me how long a skill takes to teach or how often it fails when a chair moves.

The Y Combinator company profile is direct about the larger plan. Nori wants affordable robots in many homes and businesses so their use can produce diverse training data for future generalist robot policies. The profile says, "Every user of Nori makes Nori smarter." If that works, early buyers help build the product's future capabilities. They are customers, but they are also part of the data strategy.

A chore demo is not a chore guarantee

The website shows the A3 doing recognizable work. That is better than selling a robot through specification sheets alone. What I still need is boring evidence: repeated task success rates, recovery after a failed grasp, setup time in a new room, breakage and spill rates, noise, charging behavior, and the hours an owner spends teaching one dependable routine.

Those numbers do not appear on the submitted product page. The page says the A3 can help with household tasks, but it does not give a benchmark or a service-level promise for them. New Atlas reaches a similar boundary, warning that buyers may not get much ready-to-use functionality or a guarantee that tasks will run without mistakes.

In a Reddit discussion about the launch, people asked about camera privacy, internet dependence, remote operators, stairs, clutter, and the difference between the 1.5 kg arm payload and the much larger vertical-lift figure repeated in coverage. The thread proves nothing about reliability. It does show what buyers want answered before a four-camera machine moves around private rooms.

Before ordering, I would ask Nori for an unedited repeated-task demo, the expected teaching time, a clear local-versus-cloud breakdown, data-retention terms, and the recovery path when the robot drops or misidentifies an object.

The cameras change the buying decision

A trainable home robot needs to see the home. That does not settle who receives the video, which recordings become training data, or what happens when a shared skill was learned in someone else's kitchen. Nori may have good answers. I would want them in writing before connecting the A3 to a home network.

A home contains people who did not buy the robot. Visitors, children, caregivers, and cleaners can all enter its cameras and microphones. I argued in my piece on smart-home AI and domestic workers that owner-only controls are a poor fit for devices that observe bystanders. A mobile robot carries that problem from room to room.

The skills marketplace adds another layer. A shared skill could save owners hours of training, which is the best argument for Nori's model. It also needs a clear account of what travels with that skill. The policy, demonstrations, video, room geometry, and logs do not all deserve the same treatment. I would want the marketplace to separate reusable behavior from household data by default.

I would buy the platform, not the promise

For a robotics developer, the rough edges may be the point. The A3 puts two arms, mobile sensing, teleoperation, and an SDK into a price range where experimenting on real hardware becomes easier to justify. A school or small lab could learn more from a limited machine it can modify than from a polished robot it cannot afford.

For a household buyer, I would wait for evidence from repeated use outside Nori's own demos. The decision changes when independent owners publish how long setup takes, how the robot behaves after small changes to a room, and whether downloaded skills transfer cleanly. Privacy terms and support matter just as much as another folding video.

So who teaches the $1,688 robot to fold the laundry? At first, you do, or someone in the Nori community does. That bargain is why the price can open robotics to more developers. It is also why I would not yet mistake the A3 for an appliance that gives household time back on day one.

Originally published at markhuang.ai

Claude Fable 5.1 Made Thinking History Append-Only for New API Accounts

Mark Huang — Wed, 02 Sep 2026 01:50:10 +0000

Fable 5.1 treats preserved thinking as part of an immutable conversation prefix. New turns can join the chain, but changing what came before can break it.

Anthropic's Claude Fable 5.1 launch leads with benchmarks and pricing. Deeper in the page is an API change that could break an agent loop. For new API accounts created on or after August 31, 2026 at 00:00 UTC, the Messages API checks that earlier messages, tools, and the system prompt have not changed around a preserved thinking block. A mismatch returns a 400 error by default.

I think the security control makes sense, and the migration risk deserves equal attention. Agent harnesses often compact history, rebuild system prompts, or change tools during a session. These are ordinary product behaviors. With Fable 5.1, developers need to express them without rewriting the prefix that produced the model's earlier thinking.

The rule starts narrowly with new accounts using Fable 5.1 through the API and supported cloud platforms. Anthropic says existing accounts are not affected by this model. Future model releases will enforce preserved thinking for everyone. That grace period is time to test, before an old integration fails during the next model migration.

The model launch also changed the conversation contract

Fable 5.1 has plenty of headline material. Anthropic reports 55.8% on Terminal-Bench 4.0, up from 42.0% for Fable 5, and says cache reads now cost $0.25 per million tokens. It also says the model is available across the Claude API, Amazon Web Services, Google Cloud, and Microsoft Azure under claude-fable-5-1.

But the anti-distillation change is the part I would put on an integration checklist. Anthropic's preserved-thinking notice says the API now verifies that a returned thinking block is paired with the same prior system prompt, tools, and messages that produced it. If the prefix differs, the developer has two choices: reject the request or opt into dropping the affected thinking blocks from what the model sees.

Dropping a block lets the request proceed with less context available to the model. The HTTP call may succeed even though the integration no longer behaves the same. Anthropic says the response reports which blocks were dropped, so that event belongs in logs and evaluation results.


flowchart LR
    A[Preserved thinking block] --> B{Earlier context unchanged?}
    B -->|Yes| C[Keep thinking and continue]
    B -->|No, strict mode| D[Return 400 error]
    B -->|No, drop mode| E[Remove affected thinking]
    E --> F[Continue with reduced context]

Why bind the thinking block at all?

Anthropic describes the restriction as an anti-distillation measure. The concern is that someone can move an encrypted reasoning block into a modified conversation and induce a model to reveal the hidden reasoning in plain text. That would make it easier to collect reasoning traces at scale for training another model.

An August 2026 research paper, "Stealing Reasoning Traces from Proprietary LLM APIs", describes attacks that move encrypted reasoning blocks across sessions, users, or models. The authors report demonstrations involving Anthropic, OpenAI, and Google. After responsible disclosure, they proposed cryptographic and system-level mitigations. It is a new paper, so I would not treat it as a settled measurement of every production system. It does support the threat model behind binding a block to its original context.

The defense looks reasonable to me. A reasoning block created under one tool set should not be replayed under a different, potentially hostile set of instructions without a check. The practical question is whether the API can block that attack without making ordinary context management needlessly brittle.

Ordinary harness behavior can now look like tampering

Anthropic's developer documentation gives the compatibility boundary in unusually practical terms. Direct API integrations should treat the message array as append-only. Trimming old turns, replacing them with a client-side summary, injecting and later removing a reminder, rebuilding a system prompt with the current time, or changing the tool list can invalidate later thinking blocks.

The platform offers supported alternatives. Server-side compaction can shorten the working context, while new message types can carry updated instructions and tool changes without rewriting earlier bytes. The resulting conversation looks more like an event log. New state gets appended instead of quietly revising the record that produced the current state.

If a harness calls the Messages API directly, I would assume it needs inspection whenever it compacts context, changes tools, injects reminders, rotates dynamic system text, or routes a conversation between models.

There is an awkward rollout detail here. Anthropic warns framework maintainers that users supplying newly created API keys may encounter enforcement before the maintainer does, because the maintainer's own account is likely older. That can produce the worst kind of compatibility bug: customer reports that the developer cannot reproduce with the same code and model.

I would make the future rule fail loudly in CI

Older accounts do not need to wait for automatic enforcement. The documentation says a request can opt into the binding check with the thinking-binding-controls-2026-08-01 beta header and a prefix_mismatch_behavior setting. I would run representative multi-turn sessions through that path and choose the error behavior in CI. A hard failure exposes the first mutated prefix.

The test set needs the transitions that agent demos tend to skip. Cross a compaction boundary, change a tool, refresh a dynamic system value, exercise a model fallback, and run long enough for reminders to repeat. Request bodies should be compared through the shared prefix. If production uses drop mode, the input_transformations field should be logged and counted because a successful HTTP response can still mean the model lost earlier thinking.

The test also needs to represent a fresh customer organization. The current rule covers new Claude Platform organizations and Amazon Bedrock accounts, plus new Google Cloud Vertex AI and Microsoft Azure Foundry projects created after the cutoff. Claude Code, Claude Cowork, Claude.ai, third-party products, and models other than Fable 5.1 are outside this specific enforcement scope.

The security fix should be treated as an API migration

Early public launch discussion is mostly about benchmark gains, cheaper cache reads, and whether subscription users share the savings. Those are fair questions. For teams that own an agent loop, preserved thinking is more likely to turn the upgrade into an integration failure.

Anthropic has a concrete reason to stop reasoning blocks from moving freely between rewritten conversations. The new API also offers migration mechanisms instead of banning long conversations. Even so, the control changes what counts as a valid conversation. Developers should treat it as an API contract.

Fable 5.1 gives new accounts the rule and old accounts the warning. My response is straightforward: keep history append-only where possible, put dynamic state in the supported message types, and test the rejection path while enforcement is still optional for older accounts. A future model will make the rule universal.

Originally published at markhuang.ai

Haiku Beta 6 Nearly Halved a Five-Hour Build. I'm Still Using a Spare Disk.

Mark Huang — Tue, 01 Sep 2026 08:20:21 +0000

Haiku Beta 6 makes old hardware inviting again. I would still give it a test disk, not the disk that holds my only copy of anything.

Haiku R1/beta6 arrived on August 26, 2026, almost two years after beta5. The short announcement understates the work behind it: the official release notes count more than 530 resolved bug and enhancement tickets, plus Firefox, hardware-accelerated QEMU on most x86_64 systems with Intel VT-x or AMD-V, and a broad sweep of performance and hardware fixes.

My read is simple. This is the first Haiku release in a while that makes me want to rescue an old PC just to see how pleasant it can be. It does not make me want to entrust that PC with my main working environment. The performance gains are concrete, but the project still calls the release beta-quality and says it cannot assure users against data loss. That warning sets the boundary.

The speed claim has receipts

The number that caught me was not the ticket count. On the same build host, a full HaikuWebKit rebuild fell from 4 hours 53 minutes on beta5 to 2 hours 33 minutes on beta6. A separate test of git status in a repository with more than 160,000 files dropped from about 15 seconds to 2.5 seconds with a hot disk cache.

Those are source-reported tests, not a promise that every workload is twice as fast. The notes say straightforward compiles may improve only a little and put Haiku at roughly 40% behind Linux for compile performance. A Haiku developer offered more context in the Lobsters discussion: the system can approach parity on some workloads, but its stronger distinction is low input latency and a responsive interface rather than raw computation.

I like that distinction because it gives Haiku a reason to exist beyond nostalgia. A personal computer can feel good even when it loses a throughput benchmark. Beta 6 also replaces its aging user-space memory allocator with one based largely on OpenBSD's, improves filesystem caching and locking, and cuts several fixed memory costs. The project says a stripped-down 32-bit installation can once again boot with 128 MB of RAM, though its published minimum remains 256 MB and modern Firefox is x86_64 only.

Firefox changes what the machine is for

A quick desktop is charming until the browser becomes the slow, incompatible room where most work happens. Beta 6 does not solve the modern web, but it changes the proposition. Firefox now appears under its official branding, alongside ports derived from the same codebase. The release notes are careful here too: missing features or incomplete behavior compared with Firefox elsewhere should be reported to HaikuPorts, not Mozilla.

Go is now in the package repositories as well, although the notes label the port as version 1.26.1, a bit out of date, and not yet submitted upstream. These additions benefit two groups immediately: curious users who need a viable browser, and developers who want to see whether Haiku can support real work instead of a guided tour around the desktop.

Public reaction reflects that shift. In the Haiku subreddit discussion, one commenter singled out Go as the feature that could make actual work possible. Another reported that beta6 felt less stable than beta5 on an older Dell laptop. Neither anecdote settles the release. Together, they are a useful reminder that a wider software catalog and dependable hardware support are different tests.

A virtual machine has its own trap

Trying Haiku in a VM sounds like the safest answer, and it mostly is. There is one sharp edge worth knowing before blaming the OS for an unbootable install. In the OSNews discussion, a Haiku developer confirmed that the virtio_block driver was disabled after intermittent disk corruption and kernel panics. virtio_scsi remains supported, but some virtualization interfaces choose virtio_block when the user selects a generic VirtIO disk.

That caveat did not make the official release notes. It should affect the test plan. I would use a backed-up spare machine or verify the VM's virtual disk controller before installation. I would also avoid treating the new NVMM support as finished infrastructure. Haiku calls its QEMU acceleration experimental even though it supports 32-bit and 64-bit guests, multiple virtual CPUs, and many guest operating systems.

My boundary is boring on purpose: use a disk you can erase, keep another copy of anything important, and confirm the VM is not using virtio_block. Beta 6 is interesting enough to install, but the release notes do not ask for production trust.

The beta label is doing honest work

Haiku has spent 25 years pursuing a specific kind of personal computer: fast, coherent, and built around its own ideas instead of assembling another Linux distribution. Beta 6 moves that project forward in ways a user can notice. Older package states can now be cleaned up, the screenshot tool can capture a selected area, newer x86 hardware gets more support, and a crashed app_server can usually restart while applications reconnect.

Still, this is not the release where the beta label becomes ceremonial. ARM64 can reach the desktop under virtualization but is not part of the supported release images. WebPositive's multiprocess WebKit work remains unfinished. The tracker lists more than 3,900 open tickets, and the project's own warning about known and unknown bugs is unusually direct.

That honesty makes the decision easier, not harder. I would install Haiku R1/beta6 for the same reason I would restore a well-designed old tool: not because it can replace every modern default, but because its priorities are still legible when I use it. Put it on hardware that has been waiting for a purpose. Give it a separate disk and a fair test. If it earns a larger role later, that should come from what survives daily use, not from turning an impressive release note into a guarantee the developers never made.

Originally published at markhuang.ai