TL;DR
Meta released Muse Spark Open on 2026-10-15: 816B total parameters, 42B active per token, a 1M-token context window, under a custom Muse Model License with limited commercial terms. Apple published decode benchmarks the same day on Mac Studio M5 Ultra machines linked by RDMA over Thunderbolt. At a 65,536-token context, the NVFP4 build decodes at 15.74 tokens/s on 2 machines with pipeline parallelism, 24.99 on 2 machines with tensor parallelism, and 36.73 on 4 machines with tensor parallelism.
I checked those numbers against two quantities that are easy to compute from public files: the bytes read per token and the memory bandwidth of the machine. Three findings:
- The model does not fit on one 512 GB machine. The NVFP4 repository is 533.0 GB, so two machines are a memory requirement, not a speed choice.
- The cache is small for a 1M-context model. From the public config, the cache for a full 1,048,576-token sequence is 69.1 GB in bf16, 3.85 times smaller than if every layer used full attention.
- Dense models in the same benchmark reach 82 to 89 percent of the bandwidth ceiling on one machine. Muse Spark Open reaches 42 percent on two machines with pipeline parallelism and 25 percent on four with tensor parallelism. The gap is the interesting part of this story.
Everything below is arithmetic on public files plus Apple's reported measurements. Where I am guessing, I say so.
What exactly was released, and what runs it?
From Meta's launch post and the Hugging Face repositories:
- 816B total parameters, 42B active, natively multimodal (text, image, audio inputs in the config), 1M-token maximum position embeddings.
- Weights on Hugging Face as
meta-models/Muse-Spark-Open-816B-A42B(bf16), an NVFP4 variant, and community GGUF and MLX builds already listed on the Hub. - Day-one serving support listed for SGLang, vLLM, TensorRT-LLM, Transformers, MLX, llama.cpp and Unsloth, plus hosted access through several clouds and inference providers.
The text config (readable from an open MLX mirror, since the original repository is gated) gives the shape: 62 layers, hidden size 8192, 128 attention heads with 16 key-value heads at head dimension 64, 256 routed experts with 8 active per token, and a sliding window of 2,048 tokens. The layer pattern is regular: every fourth layer, starting at index 1, uses full attention, and the other three use sliding-window attention. That is 16 full-attention layers and 46 sliding-window layers.
Why does it need two machines and not one?
Apple's report states the M5 Ultra Mac Studio offers up to 512 GB of unified memory. The repository sizes, summed from the Hugging Face file listings:
| Build | Repository size |
|---|---|
| bf16 original | 1,644.4 GB |
| NVFP4 (Meta) | 533.0 GB |
| MLX 4-bit (community) | 532.8 GB |
Both 4-bit builds exceed 512 GB before a single byte of cache, the operating system or activations is allocated. Apple's results list no single-machine run for this model. Two machines give 1,024 GB combined, which leaves about 491 GB after the weights for cache, runtime and the OS.
That headroom is generous for one user and thin for a fleet. An upper bound, ignoring everything but the cache: 491 GB divided by 69.1 GB per 1M-token sequence is about 7 sequences, or about 54 sequences at 128K tokens (8.98 GB each). Real systems lose a chunk of memory to the operating system, the GPU wired-memory limit and activation buffers, so treat those as ceilings.
How much cache does a 1M-token sequence need?
For a layer with h_kv key-value heads, head dimension d, and bf16 storage, the cache is 2 * h_kv * d * 2 bytes per token per layer. Here that is 2 * 16 * 64 * 2 = 4,096 bytes. Sliding-window layers stop growing at 2,048 tokens, so they cost a constant amount.
import json
t = json.load(open("config.json"))["text_config"]
layers = t["layer_types"]
full = layers.count("full_attention") # 16
slide = layers.count("sliding_attention") # 46
per_tok_layer = 2 * t["num_key_value_heads"] * t["head_dim"] * 2 # 4096 bytes
def kv_gb(ctx):
grows = full * per_tok_layer * ctx
fixed = slide * per_tok_layer * min(ctx, t["sliding_window"])
return (grows + fixed) / 1e9
print(kv_gb(1_048_576)) # 69.1 GB
The result: 64 KiB of cache growth per token (16 layers * 4,096 bytes), plus a constant 0.39 GB for the 46 sliding-window layers.
| Context | Actual layout | If all 62 layers were full attention | Ratio |
|---|---|---|---|
| 32K | 2.5 GB | 8.3 GB | 3.3x |
| 128K | 9.0 GB | 33.3 GB | 3.7x |
| 262K | 17.6 GB | 66.6 GB | 3.8x |
| 1M | 69.1 GB | 266.3 GB | 3.85x |
This is a cache-side design choice, and a conventional one. It trades the ability of 46 layers to look far back for a 3.85x smaller cache, and leaves 16 layers to carry long-range retrieval. Whether that costs quality on long-context tasks is something only long-context evaluations can answer, and I have not seen independent ones yet. For a different design that goes further, with linear-attention layers replacing most of the cache, see the Ling-3.1-flash post linked at the end.
What is the bandwidth ceiling for decoding?
At batch size 1, decoding one token means streaming the active weights and the relevant cache through the memory system once. Compute is almost idle. So the speed limit is:
tokens/s <= memory_bandwidth / (active_weight_bytes + cache_bytes_read_per_token)
Inputs, with the assumptions marked:
- Bandwidth: 1.2 TB/s per M5 Ultra, as reported in Apple's write-up (819 GB/s for the previous M3 Ultra).
- Active weight bytes: 42B active parameters times the average bytes per parameter of the NVFP4 repository, 533.0 GB / 816B = 0.653 bytes. That gives 27.4 GB per token. This is crude: the attention and embedding weights are stored in bf16 while the experts are 4-bit, so the true active read is somewhat higher and the true ceiling somewhat lower.
- Cache read per token at 65,536 tokens of context: 4.68 GB (16 full layers over 65,536 tokens plus 46 sliding layers over 2,048).
Sum: 32.1 GB per token. One machine's bandwidth gives 1.2e12 / 32.1e9 = 37.4 tokens/s.
How the ceiling scales with the setup depends on the parallelism. With pipeline parallelism, each token still passes through the machines one after another, so only one machine is reading at a time and the ceiling stays at 37.4 tokens/s however many machines you add. With tensor parallelism, every machine reads its slice of every layer at the same time, so the ceiling is roughly multiplied by the machine count, minus synchronization cost. That is why Apple's report shows pipeline parallelism preserving decode speed across nodes while tensor parallelism raises it, in exchange for a synchronization step in every layer.
How close do the measured numbers get?
| Setup (NVFP4, 65,536 context) | Measured | Ceiling (computed) | Share of ceiling |
|---|---|---|---|
| 2 machines, pipeline | 15.74 tok/s | 37.4 | 42% |
| 2 machines, tensor | 24.99 tok/s | 74.7 | 33% |
| 4 machines, tensor | 36.73 tok/s | 149.5 | 25% |
Doubling the machines under tensor parallelism did not double the speed: 24.99 to 36.73 is a 1.47x gain for 2x the hardware. Going from pipeline to tensor on two machines gained 1.59x. Each step adds communication cost that bandwidth alone does not predict.
Why do the dense models land closer to the roofline?
The same Apple report includes two dense-looking comparison models. I am assuming both are dense, which I have not verified from their model cards:
| Model (single M5 Ultra) | Weight bytes | Ceiling at 1.2 TB/s | Measured | Share |
|---|---|---|---|---|
| Qwen 3.8 27B, bf16 | 54 GB | 22.2 tok/s | 19.69 | 89% |
| Mistral Medium 3.5 128B, FP8 | 128 GB | 9.4 tok/s | 7.70 | 82% |
Both land at 82 to 89 percent, which says the 1.2 TB/s figure is a realistic target on this hardware and my ceiling method is not wildly off. The 816B mixture-of-experts model on two pipelined machines gets 42 percent. Possible reasons, none of which I can confirm from the published data:
- Expert gathering. At batch size 1, each layer reads 8 of 256 experts, which are separate memory regions. Many smaller reads can use bandwidth less efficiently than a few large contiguous ones.
- Mixed precision. As noted above, part of the active path is bf16, so my 27.4 GB estimate is probably low, which makes the true ceiling lower and the true share higher than 42 percent.
- Inter-machine transfer. Each token crosses the link between machines, and routing decisions add latency that dense layers do not have.
The honest summary is that the gap exists and has several plausible causes. A per-layer profile would separate them, and I would trust that over my estimate.
What changes at long context?
The weights cost the same at every context length. The cache does not. Using the same crude model with one machine's bandwidth:
| Context | Cache read per token | Weight + cache | Ceiling (1.2 TB/s) |
|---|---|---|---|
| 8K | 0.9 GB | 28.4 GB | 42.3 tok/s |
| 128K | 9.0 GB | 36.4 GB | 33.0 tok/s |
| 262K | 17.6 GB | 45.0 GB | 26.7 tok/s |
| 1M | 69.1 GB | 96.5 GB | 12.4 tok/s |
At 1M tokens the cache read (69.1 GB) is 2.5 times the active weight read (27.4 GB). Past roughly 420K tokens the cache outweighs the weights, so the fixed cost of the model stops being the main cost and the context length takes over. This model ceiling also ignores the compute cost of attention itself, which grows with context and which I did not model, so real long-context numbers should come in below these.
What should you take from this?
- Do the memory arithmetic first. A 4-bit build of an 816B model is 533 GB. Whether your hardware can hold it, with room for cache, is a two-line calculation, and it decides your topology before any benchmark does.
- Read the layer pattern, not just the context length. A 1M-token window with 16 full-attention layers has a very different cost from a 1M-token window with 62.
- Pick the parallelism for the goal. Pipeline parallelism pools memory and keeps per-token speed. Tensor parallelism buys speed at a synchronization cost. Apple's data shows both effects.
- Treat vendor tokens/s as one point on a curve. The 36.73 figure is at 65,536 tokens of context on four machines. At 1M tokens, my model says the same hardware has a much lower ceiling.
- Wait for long-context quality evaluations. A cheaper cache is only a win if retrieval quality survives it.
FAQ
Can Muse Spark Open run on a single Mac Studio?
Not the 4-bit builds. The NVFP4 repository is 533.0 GB and the largest single M5 Ultra configuration reported is 512 GB of unified memory. Apple's results list no single-machine run for it. Smaller quantizations may fit, but I have not seen measurements.
How fast is it in practice?
Apple reports 15.74 tokens/s with two machines and pipeline parallelism, 24.99 with two machines and tensor parallelism, and 36.73 with four machines and tensor parallelism, all at a 65,536-token context with the NVFP4 build. Longer contexts will be slower.
How much memory does a 1M-token context need?
From the public config, the cache for one 1,048,576-token sequence is about 69.1 GB in bf16, on top of the 533 GB of weights. A quantized cache would reduce that, and I did not model it.
Is memory bandwidth really the bottleneck for decoding?
At batch size 1, yes, to a first approximation. Two dense models in Apple's report reached 82 to 89 percent of the bandwidth ceiling on a single machine. The mixture-of-experts model reached less, so other overheads matter there.
What is the license?
Meta describes a Muse Model License with limited commercial terms that lets many developers use the model for free. The Hugging Face tag is license:other. Read the license text before building a product on it.
Method and caveats
- Cache arithmetic uses
text_configfrom an open MLX mirror of the model. The original repository is gated, so I could not confirm the config there. The shape numbers come from the mirror. - The ceiling uses 1.2 TB/s as reported and treats it as per machine. Real sustained bandwidth is lower than peak.
- Active weight bytes use one average bytes-per-parameter figure for the whole model, which is an approximation.
- The two dense comparison models are assumed to be dense. If either is not, its row in the table is wrong.
- Measured numbers are Apple's, not mine. I did not run these benchmarks.
- The script that produced every computed number is short enough to rerun from the config and the three repository sizes.
Sources
- Meta: Muse Spark Open announcement
- Apple Machine Learning Research: Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
- Hugging Face: meta-models/Muse-Spark-Open-816B-A42B-NVFP4
- Hugging Face: mlx-community/Muse-Spark-Open-816B-A42B-4bit
Top comments (0)