Apple just announced the M5 Ultra Mac Studio — 36-core CPU, 80-core GPU, up to 512GB of unified memory, and 1.2 TB/s of memory bandwidth. Within the same 24-hour window, Alibaba's Qwen team dropped Qwen 3.8-Flash-Next, a 125-billion-parameter MoE model that activates only 6 billion parameters per token. These two announcements arrived independently, but together they represent the strongest signal yet that serious local AI inference has crossed a threshold most people didn't expect until 2027.
📖 Read the full version with charts and embedded sources on ComputeLeap →
The thesis is simple: Apple is quietly becoming an AI inference company. Not a model company. Not a training company. An inference company — one that sells the hardware that lets you run other people's models without sending a single token to the cloud. And Qwen 3.8-Flash-Next is the model that finally makes that hardware worth owning.
What the M5 Ultra Actually Delivers
Let's start with the numbers that matter for inference, not the marketing slides.
The M5 Ultra's 1.2 TB/s of unified memory bandwidth is the headline spec for anyone running LLMs. For comparison, that's 2.25x the bandwidth of NVIDIA's DGX Spark (273 GB/s) and nearly 5x AMD's Strix Halo (256 GB/s). Since LLM token generation is almost entirely memory-bandwidth-bound — you're reading model weights from memory for every token — this single number tells most of the inference story.
The capacity side is equally significant. At 512GB of unified memory, the M5 Ultra can hold models that simply don't fit on any other desktop hardware. The DGX Spark and Strix Halo both cap at 128GB. Apple just gave you 4x the headroom.
Hardware Comparison: Desktop AI Inference Rigs (August 2026)
Spec M5 Ultra Mac Studio NVIDIA DGX Spark AMD Strix Halo Memory Up to 512GB unified 128GB unified 128GB unified Bandwidth 1.2 TB/s 273 GB/s 256 GB/s Base Price $5,499 $4,699 $3,999 GPU Compute 80-core Apple GPU GB10 (Blackwell) Radeon iGPU AI Perf 4.3x vs M3 Ultra Native FP4, CUDA ROCm + Vulkan Software Core AI + MLX Full CUDA stack ROCm (usable)
But here's the detail that got buried in the press release: the 512GB configuration doesn't ship until late October. The September 22 launch gets you the M5 Ultra with up to 256GB. If you're buying for local inference today, the 192GB configuration at $5,499 is the one to evaluate — and as we'll see, that's actually the right amount for what matters right now.
View discussion on Hacker News →
Enter Qwen 3.8-Flash-Next: Why 6B Active Parameters Changes Everything
The timing of Alibaba's release couldn't be more perfect. Qwen 3.8-Flash-Next is a Mixture-of-Experts model built on the upcoming Qwen 4 architecture. The key numbers: 125 billion total parameters, but only 6 billion activated per token. That's a 20:1 ratio between model capacity and per-token compute cost.
This matters enormously for Apple Silicon inference. Here's why:
Memory footprint: At 4-bit quantization, 125B parameters need roughly 62GB of RAM. That fits comfortably on a 128GB M5 Max ($2,499), let alone a 192GB M5 Ultra. You're running a model with the knowledge capacity of 125 billion parameters on a desktop machine with room to spare for your OS, IDE, and a dozen browser tabs.
Inference speed: Because only 6B parameters are active per token, the compute work per token is equivalent to running a 6B dense model — but you get the quality of a model that learned from 125B parameters. Community benchmarks on earlier MoE models show M5 Max hitting 65+ tok/s on comparable architectures like Qwen3.5-122B-A10B. The M5 Ultra, with double the bandwidth, should push well above that.
The model also introduces two architectural innovations worth watching: GDN hybrid architecture and Qwen Sparse Attention (QSA). David Hendrickson caught the details before the official announcement page was updated — describing the architecture as "VERY Local AI friendly." That phrasing isn't accidental. The Qwen team is building models specifically optimized for the kind of hardware Apple just released.
View discussion on Hacker News →
The DGX Spark Comparison Nobody Wants to Make
NVIDIA's DGX Spark landed earlier in 2026 at $4,699. It's built around the GB10 Blackwell chip and comes with 128GB of unified memory. On paper, the comparison with M5 Ultra looks like a clear Apple win — more memory, more bandwidth, similar price.
But the real-world picture is more nuanced. The DGX Spark has three genuine advantages:
Prefill speed. On long-prompt processing, DGX Spark clocks 3-4x faster than comparable Mac hardware. This matters for agentic workflows where you're feeding large context windows. If your use case involves processing 100K+ token prompts before generating a response, the DGX Spark's prefill advantage is real.
Native FP4 quantization. Blackwell's native NVFP4 halves bytes-per-parameter compared to standard quantization. This effectively doubles the DGX Spark's usable model capacity — at FP4, a 70B model fits with headroom to spare, and 200B models become possible. Apple doesn't have a native FP4 path yet.
The CUDA ecosystem. Every ML tool, every framework, every optimization library works on NVIDIA first. ROCm on AMD is "genuinely usable now" but requires patience. Apple's MLX is excellent for inference but still a fraction of CUDA's ecosystem depth. If you've set up local inference with DGX Spark before, the software experience remains smoother.
So where does the M5 Ultra actually win? Token generation speed and model capacity at scale. Once you're past the prefill phase, the M5 Ultra's 1.2 TB/s bandwidth advantage means it generates tokens faster than the DGX Spark on the same model. And at 192-512GB, it can hold models that simply don't fit on 128GB hardware — no quantization tricks needed.
For Qwen 3.8-Flash-Next specifically, the comparison tilts hard toward Apple. The model's 62GB footprint (at Q4) fits both machines, but the M5 Ultra's bandwidth advantage means faster token generation on the same weights. And if you want to run it at higher precision (Q8, ~125GB), only the M5 Ultra has the headroom.
The Strix Halo Dark Horse
Don't sleep on AMD. The Ryzen AI Halo desktop — built around the Strix Halo Ryzen AI MAX+ 395 — launched at $3,999 with 128GB of unified memory and runs native Windows 11. It undercuts the DGX Spark by $700 and the M5 Ultra by $1,500.
Its bandwidth (256 GB/s) lags both competitors, and single-batch token generation roughly matches the DGX Spark. The x86 architecture means broader software compatibility than ARM-based DGX Spark, and ROCm support has improved to the point where inference workloads run without drama.
For running Qwen 3.8-Flash-Next at Q4 (~62GB), the Strix Halo is the budget pick. You'll get slower inference than the M5 Ultra, but it'll work — and you save $1,500.
What the Community Is Saying
The convergence of these announcements didn't go unnoticed. The Hacker News thread on the Mac Studio M5 Ultra drew heavy discussion about the DGX Spark comparison, with commenters noting that if the M5 Ultra outperforms NVIDIA's box by 2x, it reshapes who can run serious local models. One commenter reported 20 tok/s generation and ~200 tok/s prompt processing on an M1 Ultra with 128GB — suggesting the M5 Ultra's 4.3x AI performance boost could push generation well past 80 tok/s on dense models.
The Qwen 3.8-Flash-Next HN thread captured the moment perfectly: "Finally a reason to own 128GB unified memory hardware." The community immediately connected the dots between Apple's hardware drop and Alibaba's model release.
Clement Delangue, CEO of Hugging Face, teased the model's availability with 947 likes and 59K views — the open-weights community is paying attention. Meanwhile, FearBuck's spec summary noted that the M5 Ultra is "configurable to $18,299," a number that puts the ceiling firmly in workstation territory.
The broader Apple Silicon thread — 1,026 points and 948 comments — captured the meta-narrative: Apple is now marketing "agentic computing" as a consumer feature, not a developer niche. The company's press materials explicitly position local AI inference as a selling point for the Mac Mini and Mac Studio alike.
Apple's Core AI Framework: The Software Play
The hardware story isn't complete without the software announcement Apple buried in the same press release. Core AI is a new framework purpose-built for deploying AI models on Apple Silicon — optimized for unified memory, CPU, GPU, and Neural Engine simultaneously.
This sits alongside the existing open-source MLX framework, which has become the de facto standard for running LLMs on Macs. Core AI appears to be Apple's bet on a higher-level, more integrated inference stack — think of it as MLX for production apps, not just researcher experimentation. If you've followed the local AI trajectory all year, this is Apple formalizing what the community built informally.
The Thunderbolt 5 clustering support is also significant. Apple claims a cluster of four Mac Studios delivers 3x the performance of a single unit, with RDMA-based shared memory. That's a path to running frontier-class models (400B+ dense) on a desk-sized cluster without touching the cloud.
Contrarian Corner: The CUDA Moat Is Real
Here's the argument against getting excited: Apple Silicon will always be a second-class citizen for ML. Every major training framework, every optimization library, every deployment tool is CUDA-first. PyTorch's Metal backend works, but it's maintained by a fraction of the engineering effort that goes into CUDA support. When a new technique drops — speculative decoding, continuous batching, tensor parallelism — it ships on CUDA months before it ships anywhere else.
The DGX Spark's prefill speed advantage (3-4x) isn't a footnote — it's the whole story for agentic workloads. Modern coding agents send 50-100K token prompts. At 3-4x slower prefill, the M5 Ultra turns a 2-second wait into an 8-second wait, and that adds up across hundreds of agent calls per session.
And 512GB of memory sounds impressive until you ask: what model needs it? Qwen 3.8-Flash-Next fits in 62GB. DeepSeek V4 Flash fits in 128GB. The models that actually need 512GB are dense behemoths that run too slowly on Apple Silicon to be practical anyway. You're paying for headroom you might never use.
What This Means for You
Here's the practical buying guide, stripped of hype:
If you're spending $200+/month on API inference and your workloads can tolerate slightly slower prefill, the M5 Ultra 192GB at $5,499 running Qwen 3.8-Flash-Next pays for itself in under two years. You get unlimited local inference with zero per-token costs, full privacy, and no rate limits. Can it really replace your API bill? Forbes ran the numbers.
If you already have an older Mac Studio and want the fastest path to local AI, the M5 Max with 128GB at $2,499 is the value sweet spot — Qwen 3.8-Flash-Next fits comfortably, and you can always set up a Qwen model on LM Studio within minutes.
If you need CUDA compatibility for training, fine-tuning, or tools that don't run on Metal, the DGX Spark at $4,699 is still the right choice. Its prefill speed advantage matters for agentic coding workflows. We covered the full DGX Spark setup earlier this year.
If you're budget-constrained, the AMD Strix Halo at $3,999 runs the same models at the same memory capacity as the DGX Spark, with better software compatibility and a lower price. You lose prefill speed but gain x86 familiarity.
Don't buy the 512GB configuration yet. It ships in late October, costs significantly more, and the model ecosystem hasn't produced models that need that capacity for practical inference. Wait for Qwen 4 and DeepSeek V5 — if they ship 400B+ dense models with breakthrough quality, the 512GB config becomes relevant.
And if you're curious about what local inference looks like on mobile hardware, the same MoE architecture that makes Qwen 3.8-Flash-Next practical on desktop is trickling down to phones with 12GB of RAM.
The Convergence Signal
Apple Silicon Day and Qwen's 125B MoE dropping simultaneously isn't coordination — it's convergence. The hardware team at Apple built a machine optimized for the exact workload that Alibaba's model team independently optimized for. Both bets landed at the same time because both teams read the same demand signal: developers want to run competitive models locally, and they'll pay desktop-workstation prices to do it.
We've been tracking the open-weight counteroffensive from Chinese labs all year. The models keep closing the gap with proprietary frontier systems. The hardware keeps getting faster. The software keeps getting better.
This week, all three converged. The local-inference thesis just got its strongest evidence yet, and the rig that makes it real costs less than a year of API bills for a serious development team.
Originally published at ComputeLeap
Top comments (0)