Day 0 Deployment of GLM-5.2-FP8-DSpark on GPUStack

This article is curated from a real-world deployment experience shared by a GPUStack community member. GLM-5.2-FP8-DSpark is an enhanced version of GLM-5.2-FP8 that applies Speculative Decoding by loading an external draft model (Speculator) provided by Red Hat AI. It keeps the original model weights unchanged while adding an additional draft model to improve decoding throughput. On the day the model was released (Day 0), a community member deployed it on an 8× H20-141G environment through GPUStack and conducted a performance comparison against the original GLM-5.2-FP8 under the same hardware conditions.

Let's start with the conclusion. The results from this test are mixed and should be interpreted objectively:

• Single-concurrency scenario: In the GPUStack Playground, DSpark achieved approximately 171 TPS, while the original GLM-5.2-FP8 achieved approximately 72 TPS, delivering around a 2.2× improvement.
• High-concurrency scenario (64K input / 3K output): DSpark achieved 59.8 TPS output throughput, which was actually lower than the 99.2 TPS achieved by the original MTP approach. Due to the lower acceptance rate of the external draft model on the Random dataset, DSpark did not improve performance under high concurrency and even introduced regression.
• Key takeaway: With the specific external draft model + GLM-5.2-FP8 combination tested in this article, DSpark is currently better suited for single-concurrency / low-concurrency interactive scenarios where first-token latency and response throughput are critical, or for research and evaluation purposes. For high-concurrency production workloads, waiting for draft models with higher acceptance rates is recommended. (The conclusions above only apply to the draft model and dataset used in this test, and do not represent the upper limit of the DSpark speculative decoding approach.)

The following sections provide the complete deployment process and benchmark results.

---

1. Deploying GLM-5.2-FP8-DSpark on GPUStack

DSpark requires a newer vLLM image (v0.25.0) to support speculative decoding with an external draft model. With GPUStack's pluggable vLLM backend architecture, we only need to add a new vLLM backend version. The entire process can be completed through the GPUStack Web UI with just a few clicks.

① Add a New vLLM Backend Version

From the left navigation menu, go to Inference Backend, find the vLLM card, and click Edit.

Under Version Configuration, click Add Version and create a new version named 25. Enter the following image:

vllm/vllm-openai:v0.25.0、

`

Select CUDA as the framework and save the configuration. A new backend version named 25-custom will then become available.

② Create a Deployment and Select Model & Backend

Go back to the Deployments page and click Deploy Model in the upper-right corner. Configure the basic information as follows:

• Source: Select ModelScope
• Repository ID: ZhipuAI/GLM-5.2-FP8
• Backend: Select vLLM
• Backend Version: Select the newly added 25-custom

③ Configure Backend Parameters

In the Advanced section, configure the backend parameters as follows (using an 8× H20-141G environment as an example):

bash
--max-model-len 131072
--tensor-parallel-size 8
--tool-call-parser glm47
--enable-auto-tool-choice
--reasoning-parser glm45
--enable-expert-parallel
--trust-remote-code
--gpu-memory-utilization=0.9
--max-num-batched-tokens 32768
--max-num-seqs 16
--speculative-config '{"model":"/var/lib/gpustack/cache/model_scope/RedHatAI/GLM-5.2-speculator.dspark","num_speculative_tokens":7,"method":"dspark"}'


The key parameter is --speculative-config:

• Set method to dspark
• Set model to the downloaded external draft model path

If the engine takes longer to load, add the environment variable:

bash
VLLM_ENGINE_READY_TIMEOUT_S=3600


to extend the ready timeout.
https://

Draft model download:
https://modelscope.cn/models/RedHatAI/GLM-5.2-speculator.dspark

Submit the deployment. Once the instance status changes to Running, the deployment is complete.

---

2. Performance Benchmark: DSpark vs Original GLM-5.2-FP8

Under the same hardware environment (8× H20-141G), we compared the two models from two perspectives: single-concurrency interactive inference and high-concurrency batch inference.

Scenario 1: Single-Concurrency Interactive Inference

Run a conversation directly in the GPUStack Playground and monitor the real-time output throughput.

GLM-5.2-FP8-DSpark:
Output throughput reached 171.6 Tokens/s, remaining stable in the 170+ range.

Original GLM-5.2-FP8:
Output throughput reached 71.81 Tokens/s.

Under single-concurrency workloads, DSpark achieved approximately 171 TPS, around 2.2× faster than the original version (~72 TPS). This demonstrates the typical benefit of speculative decoding in low-concurrency interactive scenarios.

---

Scenario 2: High-Concurrency Batch Inference (64K Input / 3K Output, 10 Requests)

Using GPUStack's built-in benchmark tool, we tested the model with a Random dataset containing 64K input tokens / 3K output tokens and 10 requests (with an actual average concurrency of approximately 6).

The baseline for comparison is the original MTP approach with speculative decoding enabled.

Original MTP (method: mtp, num_speculative_tokens: 5):

• Total throughput: 2216.39 Tokens/s
• Output throughput: 99.22 Tokens/s
• TPOT: 63.92 ms

DSpark (method: dspark, num_speculative_tokens: 7):

• Total throughput: 1336.12 Tokens/s
• Output throughput: 59.82 Tokens/s
• TPOT: 103.08 ms

Under high-concurrency workloads, DSpark's output throughput (59.8 TPS) did not improve and instead regressed, performing significantly below the original MTP approach (99.2 TPS). TPOT was also higher.

The main reason is that the external draft model provided by Red Hat AI achieved a relatively low acceptance rate on the Random dataset. When speculative decoding has insufficient acceptance, the additional draft computation overhead can negatively impact overall throughput.

Performance Summary

Scenario	Metric	Original GLM-5.2-FP8	GLM-5.2-FP8-DSpark	Comparison
Single-concurrency (Playground)	Output TPS	≈ 71.8	≈ 171.6	≈ 2.2×
High-concurrency 64K/3K	Total Throughput (TPS)	2216.39 (MTP)	1336.12	≈ 0.60×
High-concurrency 64K/3K	Output Throughput (TPS)	99.22 (MTP)	59.82	≈ 0.60×
High-concurrency 64K/3K	TPOT (ms)	63.92 (MTP)	103.08	Higher

Summary

The performance characteristics of GLM-5.2-FP8-DSpark show a clear trade-off:

1. Single-concurrency / low-concurrency scenarios:
   
   Speculative decoding provides significant benefits. In this test, DSpark achieved 1.2–2.2× the throughput of the original GLM-5.2-FP8, making it suitable for interactive workloads that are sensitive to single-request throughput and response latency.

2. High-concurrency scenarios:
   
   With the external draft model used in this test, the acceptance rate on the Random dataset was relatively low. As a result, throughput did not improve significantly and was even lower than the original MTP approach. It is currently not recommended for similar high-concurrency production workloads.

3. Recommendation:
   
   Based on the model and workload tested in this article, DSpark is currently more suitable for research validation and low-concurrency scenarios. For high-concurrency production deployment, it is recommended to wait for draft model versions with higher acceptance rates on these workloads.
   
   (The conclusions above only apply to the draft model and dataset used in this test, and do not represent the upper limit of the DSpark speculative decoding approach.)

Thanks to GPUStack's pluggable vLLM backend architecture, adding a new image version, integrating an external draft model, or switching between different inference configurations can all be completed through the Web UI with just a few clicks. This enables teams to evaluate new models from Day 0 with minimal adaptation effort.

Test Environment: 8× H20-141G ｜ GPUStack v2 ｜ vLLM v0.25.0 (25-custom image)

Thanks to the GPUStack community member for sharing this real-world deployment experience.