DSpark for Spark-X2.6-4B is now available. Faster inference. More magic. 🪄
The published benchmarks show more than double the average BF16 inference throughput: 2.14x on NVIDIA RTX 4090 with llama.cpp and 2.50x on Apple M5 Pro with MLX, across the six reported datasets.
Download on Hugging Face · Download on ModelScope
A small draft model, fewer sequential decoding steps
DSpark is a 624M-parameter BF16 draft model for speculative decoding with Spark-X2.6-4B. The draft proposes up to seven tokens per pass, and the target model verifies them in parallel. This reduces the number of sequential target decoding steps.
This release provides the draft in BF16 GGUF format, paired with Spark-X2.6-4B-GGUF. The target can use BF16 or quantized GGUF weights. The draft complements the target model and is not a standalone chat model.
What the benchmarks show
| Target / device / runtime | Average speedup | Average throughput, without → with DSpark |
|---|---|---|
| BF16 · RTX 4090 · llama.cpp | 2.14x | 95 → 203 tok/s |
| BF16 · M5 Pro · MLX | 2.50x | 31 → 77 tok/s |
| Q4_K_M · RTX 4090 · llama.cpp | 1.98x | 226 → 447 tok/s |
| INT4 · M5 Pro · MLX | 1.62x | 71 → 115 tok/s |
The peak reported speedup is 2.94x on MATH500 with BF16 on M5 Pro. The peak reported throughput is 532 tok/s on GSM8K with Q4_K_M on RTX 4090. These are different test configurations.
The six datasets are AIME25, GSM8K, MATH500, HumanEval, MBPP and Arena-Hard-v2. Results depend on hardware, precision and workload. These figures come from the published model card; they are not a new independent benchmark.
The acceptance-length evaluation uses seven draft tokens and a maximum output length of 4,096 tokens. Its average acceptance lengths range from 4.0236 to 4.5522 across the four reported configurations. Acceptance length includes one additional target token, so the maximum is eight.
Try it locally
Follow the model card's setup instructions for XHToken/llama.cpp. The documented commands target Linux, Bash and NVIDIA GPUs. Integration into upstream llama.cpp is in progress. The model card reports MLX benchmarks separately.
The Spark-X2.6 model series is released under Apache 2.0.
Like the model. Download the magic.
❤️ Give the model a Like, download the weights and try DSpark on your own workload:
Share your device, runtime, target precision and results with the community. We would love to see what you build.
Top comments (0)