DEV Community

Cover image for AI Weekly: Cheap Tokens, Tight Safeguards, and a Two Million GPU Order
Alex Merced
Alex Merced

Posted on

AI Weekly: Cheap Tokens, Tight Safeguards, and a Two Million GPU Order

Week of August 26 to September 2, 2026

By Alex Merced, Data Lakehouse and AI Evangelist

Three labs shipped models this week and every one of them led with price. Anthropic cut cache reads 75%, Z.ai put a natively multimodal 320B model on the table at $0.15 per million input tokens, and Alibaba previewed its next architecture with a model that computes six billion parameters per token. Underneath the model news, AWS and NVIDIA committed to two million more GPUs, AMD shipped a version-10 software stack built around agents, and MCP published a roadmap that puts agent identity at the center of the next spec.

Models: Anthropic ships Fable 5.1 and Mythos 5.1

Anthropic released Claude Fable 5.1 and Claude Mythos 5.1 on September 1. The two are the same underlying model with different safeguard levels. Fable 5.1 is generally available on the Claude API, Claude.ai, Claude Code, and Claude Cowork, and it runs on AWS, Google Cloud, and Microsoft Azure. Developers call it with the identifier claude-fable-5-1.

Mythos 5.1 goes only to vetted participants in two trusted access programs. The Cyber Verification Program covers defensive security work. The Life Sciences Verification Program, built with the US government, enrolled its first participants and plans to widen access. Anthropic also moved Claude Security, its codebase vulnerability scanner, onto Mythos 5.1.

Token pricing stays at $10 per million input and $50 per million output. The change is in cache reads, which drop 75% to $0.25 per million. Anthropic measured four weeks of real August usage and reports roughly 25% lower cost on typical workloads and up to 45% on context-heavy agentic work. For anyone running long agent loops where cached context dominates the bill, that second number is the one that matters.

On benchmarks, all figures below are vendor-reported and run with production safeguards enabled. Fable 5.1 scored 52.6% on Terminal-Bench-Science 0.1 against 24.7% for Fable 5, 29.0% for Opus 5, and 22.4% for GPT-5.6 Sol. Anthropic notes a standard error of 3.5 to 4.5 points on that benchmark, so read the gap as large rather than exact. Terminal-Bench 4.0 came in at 55.8%, with Mythos 5.1 at 60.9%. The spread between the two reflects tasks where cyber safeguards intervened on Fable.

Other numbers from the same table: 31.4% on AutomationBench against 17.1% for Fable 5, 73.4% on CursorBench 3.2.0, 60.9% on Humanity's Last Exam without tools, 1853 on GDPval-AA v2, and 77.9% partial credit on the August task release of OSWorld 2.0. The pattern is consistent. Long-horizon agentic work moved a lot, and short-horizon reasoning moved a few points.

The safeguard changes deserve as much attention as the scores. Anthropic reports that its updated biology safeguards fire 85% less often on benign elementary biology and medical questions. Cyber safeguards block 60% fewer false positives, in part because Fable 5.1 is now permitted to identify software vulnerabilities. Exploit development, penetration testing, and binary vulnerability scanning still route to Opus models. If you build security tooling on Claude, the routing map changed this week and your evals should account for it.

Two other changes affect anyone building on the API. Enterprise Frontier Safeguards store customer data on customer-controlled cloud infrastructure and give the privacy properties of a zero data retention agreement while keeping misuse detection in place. Rollout starts this fall, and eligible customers get zero data retention on Fable 5.1 until then. Separately, Anthropic added anti-distillation measures: new API accounts created from launch day forward cannot manually edit Claude's prior context in a multi-turn conversation while preserving the transcript of its earlier thinking. Existing accounts are unaffected for now. A small number of custom integrations will need adjustments.

Anthropic also confirmed it is watermarking outputs of models released after August 2, 2026, under the EU AI Act's Code of Practice on Transparency of AI-Generated Content, which it signed in July alongside 190 other organizations. The watermark is a statistical signal, invisible without the detection API, and carries no information about the user or the conversation. A detection API is in private preview for regulators, researchers, media, and enterprises with their own compliance obligations.

The science results are the part of this release that points somewhere new. Given open-source protein design and folding tools, Mythos 5.1 designed binders whose affinities on three targets ran ten times higher than the best entries in Adaptyv Bio's design competitions. Its hit rate reached nearly 50% across twelve targets, against a typical 10% to 15% in the field. Fable 5.1 trained a network on 30-year-old NASA Magellan radar data to build an elevation map of a third of Venus at two to three kilometer resolution, up from 10 to 20, with heights up to 25% more accurate. Anthropic released the map under a Creative Commons license ahead of the NASA VERITAS and ESA EnVision missions. Mythos 5.1 also wrote custom GPU kernels that sped up seven open-source genomics and protein models by as much as 2.5 times on an H100, cutting estimated GPU cost on genome-wide analyses by 30% to 60%.

That last result is worth pausing on. The work took days instead of the weeks a performance engineering team normally spends, and Anthropic plans to open-source the optimizations. Kernel optimization is exactly the kind of expensive, specialized work with outsized payoff most academic labs cannot afford. A model that does it cheaply changes who gets to run large-scale experiments.

Z.ai puts a multimodal 320B model at Flash prices

Z.ai released GLM-5.3-Flash on August 26. It is a mixture-of-experts model with 320 billion total parameters and 18 billion active per token, a 1,048,576-token context window, and native image and video input. Weights ship on Hugging Face under an open license, and the model runs natively in FP8.

The architecture is where the cost story comes from. Z.ai combined sparse and linear attention to hold down long-context serving cost, and the model starts from a newly trained base rather than a post-training pass on GLM-5.2. Self-reported numbers put it at 63.4 on DeepSWE against 46.2 for GLM-5.2, and 48.8 on AutomationBench against 26.2. Those are the lab's own figures and remain unverified by third-party evaluators.

List pricing runs $0.15 per million input, $0.03 cached, and $0.50 per million output, with a 50% launch promotion that expires September 9 at 16:00 UTC. Budget against the list rate, not the promo. Note that this is a different model from the text-only GLM-5.3 flagship, which lists at $1.40 and $4.40 and whose weights have not shipped.

The model spent twelve days on OpenRouter as an anonymous entry called Ox Alpha before the announcement. Reporting from MarkTechPost says it was served on domestically produced Chinese AI chips during that period. If that holds up, it is a signal about inference supply independent of the model itself.

Alibaba previews the Qwen4 architecture

Alibaba's Qwen team open-sourced Qwen3.8-Flash-Next on the same day, framing it as an architecture preview of the coming Qwen4 generation rather than a flagship. The team used the same pattern before, shipping Qwen3-Next ahead of the Qwen3.5 series so the community had time to build tooling.

The shape is unusual. A 125B backbone pairs with a 51B N-gram embedding table and a 4B multi-token prediction head, and only 6 billion parameters activate per token. The 48-layer stack mixes 36 Gated DeltaNet linear-attention layers with 12 full-attention layers using Qwen Sparse Attention, trained with the Muon optimizer. Native context is 262,144 tokens, extensible toward a million. It ships under a community license with weights on Hugging Face and ModelScope.

Reported scores include 62.5 on SWE-bench Pro and 91.7 on GPQA Diamond. The-decoder reports the model lands just below the Qwen3.8-Max flagship at roughly one twelfth the price on both input and output. The N-gram table is offloadable to host RAM, which changes the local-inference math for anyone running on a workstation rather than a rack.

Tencent also open-sourced a Hy4 preview on August 28 with 770 billion parameters and a one-million-token context, per TechNode. Details beyond that are thin, so treat it as an early signal rather than a deployable option.

Three open-weight releases in four days, two of them explicitly optimized for cost per token, is the shape of this market now. The frontier labs compete on capability at the top and the open-weight labs compete on the price floor underneath them. Anthropic cutting cache reads 75% in the same week is not a coincidence.

Tooling: coding agents get session management and vendor skills

Claude Code shipped a set of changes aimed at teams rather than individuals. Enterprise plans can now turn on skill and plugin security scanning, which checks third-party skills and plugins for malicious content when someone uploads or edits them. Given that skills are just instructions plus code that an agent will execute, scanning them at upload is a control that should have existed from the start.

Cloud sessions now sync plugins from claude.ai, showing them as name@synced and never overriding a same-named plugin installed locally. The release notes also record a fix worth reading if you run on Bedrock: streaming behind proxies that strip the response Content-Type header silently doubled billed API calls by re-running every turn non-streaming. That is a billing bug that produces no error, which is the worst kind. The usage-limit message now also reports when session and weekly limits reset, not only the monthly spend limit.

Fable 5.1 defaults to High effort in Claude Code and Medium in Claude Cowork and on Claude.ai. Effort level drives both quality and cost, so anyone moving to the new model should verify which default applies in each surface before comparing bills.

OpenAI's Codex spent the week on session and task management. The release notes list a new interactive codex agents dashboard for searching, starting, opening, renaming, and stopping tasks, plus a codex queue command for sending messages into existing local or remote sessions. New /cd, /pwd, and /cwd commands manage the working directory inside TUI sessions. The codex doctor command now diagnoses endpoint protection, network and proxy failures, desktop app state, and update connectivity.

SDK users can pass exact CLI config overrides and select max or ultra reasoning effort. A later drop added @ mentions across Codex tasks, letting agents read, create, and message other tasks from the terminal. Both tools are converging on the same realization: once agents run for hours, the interesting product surface is not the chat box, it is the queue.

The most interesting tooling item came from a chip vendor. AMD's ROCm 10 release includes AMD Skills, which packages validated AMD hardware knowledge and workflows into a form that Claude Code, Cursor, and Codex consume directly. A hardware company shipping its documentation as agent skills rather than as a PDF is a meaningful shift in how vendor knowledge reaches developers. Expect more of it, and expect skill provenance to become a security question fast, which is precisely what Anthropic's scanning feature anticipates.

On the adoption side, Cognition said it moved Devin's Opus 5 traffic to Fable 5.1 on launch day, starting with code review, and credited the cache read pricing for making a Fable-class model economical for workloads it had kept on cheaper tiers. That is the practical effect of a pricing change: it reshuffles which model sits in which part of the pipeline.

Standards: MCP puts agent identity at the center

The Model Context Protocol maintainers published a new roadmap on August 22, setting direction for the next spec release after the large 2026-07-28 revision. Five priority areas now govern which proposals get expedited review.

Agent identity is the one to watch. MCP authorization today assumes a person clicking approve in a browser. That model breaks when the caller is a cloud workload with its own identity, acting for a user who is not present, or delegating narrower authority to a sub-agent. The roadmap commits to finalizing Demonstrating Proof of Possession and driving its adoption, and to defining an opinionated path for agent identity and delegation through Workload Identity Federation, the ID-JAG grant behind Enterprise-Managed Authorization, and standard token exchange. The maintainers also plan to keep engaging the IETF OAuth and WIMSE working groups.

This is the right problem to solve next. Long-lived API keys pasted into agent configs are how most production MCP deployments authenticate today, and that pattern does not survive contact with an auditor. Building on OAuth machinery that enterprises already run is a better answer than inventing agent-specific credentials.

The second item practitioners will feel is progressive discovery. Connecting to a server with a hundred tools means the model pays for that entire surface before the user asks anything, and tool selection degrades as the list grows. The roadmap starts an effort to let a server expose a small entry point and reveal more of its catalog as the conversation narrows. Anyone who has watched an agent pick the wrong tool from a large catalog knows the cost of the current design.

The other three areas cover agentic messaging primitives, including server-initiated events through webhooks and channels so clients stop polling, transport unification so local servers speak Streamable HTTP over stdio, and result-type improvements so a server developer knows which form of a tool result a client will actually put in front of the model. That last one sounds small and is not. Ambiguous result contracts are why the same MCP server behaves differently across two hosts.

On the browser side, OpenAI introduced Site tools, its implementation of the proposed WebMCP standard. A website exposes actions directly to an agent alongside the interface people use, and in the ChatGPT desktop app's built-in browser, ChatGPT Work and Codex discover and call those tools against the same live page and signed-in session. WebMCP is the piece the agent stack has been missing. MCP connects agents to servers, A2A connects agents to each other, and WebMCP gives the existing web a way to expose actions without anyone building a separate API.

Two more standards-adjacent items from this week. Anthropic's anti-distillation change is a de facto API contract change, since editing prior assistant context while preserving thinking transcripts stops working for new accounts and will apply to all accounts on future model releases. And the EU AI Act watermarking requirement now has a working implementation with a detection API, which sets a template other signatories will follow.

Infrastructure: two million GPUs and a memory squeeze

AWS and NVIDIA announced a major expansion of their collaboration on August 26. AWS plans to deploy two million additional Blackwell Ultra, Rubin, and Rubin Ultra GPUs across its global infrastructure in 2027 and 2028. That comes on top of the one million GPUs AWS committed to at GTC 2026, which demand has already outrun.

The deal covers more than GPU count. NVIDIA Vera CPUs come to AWS for agentic workloads that need heavy CPU compute next to accelerators. NVLink Fusion extends with custom NVIDIA high-bandwidth memory inside Trainium racks. The two companies will build AI factories for the US government, including 100,000 GPUs on secure AWS infrastructure. EC2 G7 instances add RTX PRO 4500 Blackwell Server Edition GPUs.

NVIDIA followed on August 27 by confirming Vera CPU shipments at scale, with AWS receiving its first Vera CPU server and Vera Rubin GPU in Seattle. Earlier deliveries went to Oracle Cloud Infrastructure and to Anthropic, OpenAI, and SpaceXAI. NVIDIA's CFO said the company expects Vera deployment across every major hyperscaler, neocloud, AI lab, and system OEM. Vendor-reported figures put Vera at up to 1.8 times faster per core on selected agentic workloads with twice the energy efficiency of traditional infrastructure, and those comparisons are not independently verified.

NVIDIA also moved Groq 3 LPX into full production, positioning it as a decode-phase accelerator for latency-sensitive agentic work, with vendor figures of 3,400 output tokens per second on 100,000-token long-context use cases. Splitting prefill and decode across different silicon is the direction inference hardware has been heading, and a production part built specifically for token generation makes that split concrete.

AMD shipped ROCm 10 on August 27, ten years after ROCm 1.0. The headline is ROCm.AI, which bundles AMD Skills, the new ROCm CLI, and Hyperloom, an agentic system that profiles inference workloads, finds bottlenecks, modifies code, and benchmarks the result. AMD's internal testing reports an average 3.3 times inference improvement and 2.4 times training improvement against ROCm 7, measured on eight Instinct MI355X GPUs running GLM-5, Kimi-K2.5, and DeepSeek-R1-0528. Read that as a tuned configuration against an untuned baseline, not a blanket speedup.

The rest of the release addresses fragmentation, which has been AMD's real problem. Windows and Linux now share the ROCm Core SDK, and the separate Windows HIP SDK is retired. The TheRock build pipeline is production ready. RCCL advances to NCCL 2.30.4 with GPU-initiated networking, and vLLM v0.2x is supported. An open stack that an agent can drive is a more credible challenge to CUDA than another round of raw performance claims.

Memory is where the cost pressure sits. Kioxia and SanDisk committed more than $31 billion in Japan through 2032 to expand flash production, including a new building at Kitakami. SK hynix broke ground on its HBM production base in Indiana on August 28. TrendForce projects cloud provider capital expenditure rising 98% year over year in 2026 and another 50% in 2027, with DRAM and NAND accounting for 47% of that spending in 2026 and 68% in 2027.

That last figure is the one to sit with. When memory takes two thirds of cloud capital spending, the binding constraint on AI capacity stops being GPU allocation and becomes DRAM and HBM supply. Fabs take years. Every efficiency gain that reduces bytes moved per token, from linear attention in GLM-5.3-Flash to cache read pricing at Anthropic, is a response to the same physical limit.

What this means for the data layer

Three threads from this week land directly on anyone running data infrastructure.

Cache economics now shape architecture. When cache reads cost a quarter of what fresh input costs, the winning pattern is a stable, reusable context prefix with the variable part at the end. That favors agents that hold a fixed schema catalog, a fixed set of tool definitions, and a fixed instruction block, then append the query. Teams that rebuild context from scratch on every turn are paying full freight for work the provider will discount by 75%.

Progressive tool discovery in the MCP roadmap matters more for data platforms than for most MCP servers, because a data catalog is exactly the case where the tool surface is enormous. A server that exposes every table as a tool poisons model attention. A server that exposes search and drill-down, then reveals the specific tables the conversation needs, works. Design for that now rather than waiting for the spec.

Agent identity is the governance question. Column-level and row-level restrictions enforced at a catalog only mean something if the catalog knows which agent is asking, on whose behalf, with what delegated authority. The Iceberg REST catalog community voted on finer grained read restrictions this same week. MCP is working the credential side of the same problem. Those two lines of work need to meet, and today they do not.

What to watch

The GLM-5.3-Flash promotional price expires September 9 at 16:00 UTC, and independent evaluators have not yet verified its self-reported DeepSWE and AutomationBench numbers. Z.ai still owes the community the GLM-5.3 flagship weights it promised. Alibaba's full Qwen4 family follows the Flash-Next architecture preview, with no date announced.

Anthropic said it plans to bring Fable 5.1's improvements to the rest of the Claude model family, so watch for Opus and Sonnet updates. Enterprise Frontier Safeguards begin phased rollout this fall. The anti-distillation context restriction applies to all accounts on future model releases, not just new ones, so integrations that rely on editing prior assistant turns have a limited runway.

On the standards side, the MCP working groups are taking SEPs in the five roadmap areas, with agent identity and progressive discovery the two most likely to change how you build. And keep an eye on memory pricing. If DRAM and NAND really reach 68% of cloud capital spending next year, inference cost curves will bend for reasons that have nothing to do with model architecture.


If you want to go deeper on agentic AI, lakehouse architecture, and the data infrastructure underneath both, I keep a full catalog of my books at books.alexmerced.com.

Top comments (0)