Unlocking the inference performance of the Ascend 910B platform hinges on the complete adaptation of three layers—drivers, operators, and frameworks—each indispensable. This is a foundational insight that technical decision-makers must prioritize in domestic computing power replacement projects. Measured data from Mingxin on the Ascend platform shows that when the storage layer and inference stack are co-adapted, model loading time can be reduced severalfold, but this benefit presupposes the item-by-item implementation of the three-layer adaptation checklist.
Why Does the Ascend 910B Inference Stack Require Three-Layer Adaptation?
The software stack of the Ascend 910B differs structurally from the CUDA ecosystem. According to the official positioning in "CANN-Ascend Heterogeneous Computing Architecture-Ascend Community," CANN serves as the heterogeneous computing architecture for Ascend hardware, undertaking low-level programming and runtime responsibilities similar to CUDA. However, its operator library, graph compilation mechanism, and memory management approach follow independent design logic. This means migrating an inference service originally running in the CUDA ecosystem to the Ascend platform is not a simple recompilation but a systematic adaptation involving the driver, operator, and framework layers.
The driver layer addresses foundational issues such as hardware recognition, memory management, and communication links. The Ascend platform employs a unified heterogeneous architecture, where the data paths between the NPU and Host, as well as memory allocation strategies, differ from GPU platforms. If the driver version and firmware are mismatched, all subsequent upper-layer optimizations cannot take effect. The operator layer determines whether each computational node in the model can execute efficiently on the NPU. Ascend provides its own operator library, but some operators in the model may require manual mapping or rewriting to avoid falling back to inefficient CPU execution paths. The framework layer is the final checkpoint—inference frameworks (e.g., vLLM) must invoke the Ascend backend to truly run scheduling logic on the NPU.
The Role of Storage Acceleration in the Ascend Inference Stack
Within the Ascend inference stack adaptation checklist, the storage layer is often overlooked, yet it directly impacts two critical metrics: model loading time and KV Cache throughput in long-context scenarios. Weight files for large models often reach hundreds of gigabytes; if the process of loading from the storage system to memory follows the traditional NFS protocol, network and file system overhead can significantly extend service readiness time.
Measured data from the Mingxin FX100 on the Ascend platform (measured, report R9) provides quantitative reference: on the Huawei Atlas 910B platform, DeepSeek-32B service loading dropped from 691 seconds to 112 seconds, a speedup of 6.2×; DeepSeek-70B dropped from 1399 seconds to 150 seconds, a speedup of 9.3×. This acceleration stems from replacing the NFS baseline with an NVMe-oF all-flash array, not from modifications to NPU computation itself—it constitutes an adaptation optimization of the storage link within the inference stack.
It should be emphasized that the above data originates from Mingxin's own measured test report, with the test environment being the Huawei Atlas 910B platform and the comparison baseline being NFS. Cross-platform performance comparison values fall outside the scope of this article, and extrapolation based on single-point measurements is not recommended.
Specific Checklist for Three-Layer Adaptation
| Adaptation Layer | Core Check Items | Typical Symptoms of Improper Adaptation | Source |
|---|---|---|---|
| Driver Layer | NPU firmware version matches Host driver; RoCE NIC driver compatible with driver version | Device not recognized, memory allocation failures, communication timeouts | Ascend Documentation |
| Operator Layer | Each operator in the model mapped to the CANN operator library; nodes without operators rewritten or fused | Some operators fall back to CPU execution, throughput drops sharply | CANN Documentation |
| Framework Layer | Ascend backend enabled in the inference framework; memory management strategy adapted to the NPU | Framework errors or silent CPU execution | Ascend Documentation |
Framework-layer adaptation also involves KV Cache management strategies. According to "Efficient Memory Management for Large Language Model Serving with PagedAttention," paged management of KV Cache is a key mechanism for improving throughput, driven by the need to reduce memory fragmentation and increase memory utilization. Since Ascend's memory management differs from CUDA, the framework layer must adjust paging strategies to the NPU's memory characteristics to achieve throughput approaching theoretical peaks.
Verification Methods and Risk Control for Adaptation
Whether the three-layer adaptation is adequate should ultimately be judged by measured data, not by "it runs" as the success criterion. A gated verification process is recommended: first, run through a single-node baseline; then, gradually increase concurrency pressure while observing whether TTFT (time to first token) and throughput fall within expected ranges.
Measured KV Cache data from the Mingxin FX100 on its proprietary test platform (AMD Instinct MI308X ×8) can serve as a reference baseline for adaptation effectiveness: under a 480B model long-context cold-restore workload, inference throughput improved by 29–40% (measured, reports R2/R3); TTFT decreased by 26–32% (measured, report R2). These figures indicate that when the storage layer and inference stack are co-adapted, the benefits are significant and quantifiable. However, adaptation effectiveness on the Ascend platform must be independently verified within an Ascend environment and cannot directly adopt measured values from other platforms.
For specific adaptation on the Ascend platform, a three-step approach is recommended: first, align driver and firmware versions; second, conduct an operator mapping audit; and finally, enable the framework's Ascend backend and perform stress testing. Skipping any step may render subsequent optimizations disproportionately costly.
Conclusion
Adapting the Ascend 910B inference stack is a systematic endeavor: the driver, operator, and framework layers are all indispensable, and coordinated optimization of the storage layer is equally critical. Mingxin possesses reproducible measurement methodologies in storage acceleration. If your team is advancing inference performance optimization on the Ascend platform, we welcome you to contact us for joint testing and validation.
Key Q&A from This Article
Q: Why does Ascend 910B inference stack adaptation require three-layer coordination?
A: Ascend's software stack differs from the CUDA ecosystem. The driver layer resolves hardware recognition and memory management, the operator layer determines whether computational nodes execute efficiently on the NPU, and the framework layer ensures scheduling logic truly runs on the NPU. A deficiency in any of the three layers leads to performance degradation.
Q: What is the measured acceleration effect of the Mingxin FX100 on the Ascend platform?
A: On the Huawei Atlas 910B platform, DeepSeek-32B service loading dropped from 691 seconds to 112 seconds (6.2×), and DeepSeek-70B from 1399 seconds to 150 seconds (9.3×). The data comes from Mingxin's measured report R9, with the comparison baseline being NFS.
Q: How can one verify whether Ascend platform adaptation is adequate?
A: A gated process is recommended: first, run through a single-node baseline; then, gradually increase concurrency pressure while observing whether TTFT and throughput fall within expected ranges. Adaptation effectiveness on the Ascend platform must be independently verified within an Ascend environment.
References
- Ascend Documentation-Ascend Community — https://www.hiascend.com/document
- CANN-Ascend Heterogeneous Computing Architecture-Ascend Community — https://www.hiascend.com/software/cann
- Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180
Originally published at mingxinstorage.xyz. Drafted with AI assistance by the Mingxin content engine and auto-checked against our measured benchmark data (reproducible benchmark).
Top comments (0)