For domestic substitution of AI inference storage in the Xinchuang environment, the recommended approach is to start at the storage protocol and data path layer, rather than directly replacing the GPU compute layer or application framework layer. This assessment is based on measured results from Mingxin FX100 in long-context inference with a 480B model: KV Cache tiered acceleration delivers 29–40% throughput improvement and 26–32% TTFT reduction [measured, reports R2/R3]—performance bottlenecks and optimization headroom are concentrated in storage and data movement, which is precisely where technical risk is controllable and returns are quantifiable in Xinchuang substitution.
Why the Storage Layer Is the Optimal Entry Point for Xinchuang AI Inference Substitution
The challenge of Xinchuang substitution lies not in "whether it works," but in "whether the performance loss is acceptable." Replacing GPUs and compute frameworks often entails long-term engineering for operator adaptation and precision alignment, whereas storage-layer issues are more structured: AI inference memory access patterns are highly regular, and bottlenecks are clear and measurable. According to the FlashAttention paper (arXiv:2205.14135), attention computation is limited by HBM bandwidth rather than compute capacity—this conclusion applies equally to the KV Cache read/write path. Mingxin's measured results on the AMD MI308X platform show that for a 480B model with TP8 across three concurrency levels, TTFT p50 dropped from 10.17–35.73s to 7.53–26.35s [measured, report R2], demonstrating that storage-side optimization translates directly into user-perceivable latency improvements.
Another advantage of Xinchuang storage substitution is protocol maturity. NVMe-oF and RoCEv2 are open standards not tied to specific vendors; per SNIA's definition of storage networks, these protocol layers have high standardization and low interoperability risk. The Mingxin FX100 uses an all-flash NVMe-oF array with 100Gb per interface; in measured results, LMCache parallel read patching improved TTFT by 4.1x (37.97s→9.30s) [measured, report R1]—the gains from protocol-layer optimization are not strongly coupled to whether the underlying hardware is domestic, allowing storage-layer substitution to be completed ahead of the compute layer and reserving validation space for subsequent full-stack localization.
Measured Performance Benchmarks for Storage in Domestic Substitution
The greatest risk in Xinchuang substitution is "armchair theorizing." Mingxin's measured results on the Ascend 910B platform provide a reusable reference framework: DeepSeek-70B service load time dropped from 1399s to 150s (9.3x acceleration), and DeepSeek-32B from 691s to 112s (6.2x acceleration) [measured, report R9]. This data set shows that on domestic compute platforms, storage-side optimization yields significant load acceleration benefits with no strong dependency on GPU model—although the Ascend software stack differs, NVMe-oF and tiered caching mechanisms apply equally.
It should be clarified that these figures come from Mingxin's own measured reports and do not represent cross-platform performance comparisons. Per MLPerf Inference benchmark methodology, performance evaluations are comparable only when submitted under fixed precision and latency constraints—Mingxin has not conducted horizontal comparisons between Ascend and other platforms; the above data serves solely as evidence of feasibility for storage optimization on domestic platforms. For Xinchuang technology selectors, a more pragmatic approach is to reference Mingxin's gate-based joint testing model: using TTFT reduction ≥25% and throughput improvement of 29–40% as acceptance gates [measured, report R3], with stop-loss if targets are not met. This validation method front-loads substitution risk and is more reliable than relying on vendor whitepapers.
Architectural Trade-offs and Applicability Boundaries of Storage Substitution
Storage-layer substitution is not a panacea. In Mingxin's measured results, the benefit range for KV Cache tiered acceleration is 29–40%, with the upper bound appearing at the optimized operating point with 16-way concurrency [measured, reports R2/R3]—this means benefits are strongly correlated with workload patterns. For short-context, low-concurrency inference scenarios, storage optimization headroom is limited; for long-context, high-concurrency production workloads, benefits are substantial. According to the architectural analysis in the Mooncake paper (arXiv:2407.00079), the benefit premise of a KVCache-centric disaggregated compute-storage design is high prefix cache reuse—if business prefix reuse is low, tiered cache hit rates decline, and acceleration effects converge toward the lower bound of the range.
Another boundary is the training scenario. In Mingxin's measured results, Checkpoint save time for 8-card 32B LoRA training dropped from 178s to 94s (1.9x acceleration) [measured, report R1]—training-side benefits exist but are significantly lower than inference-side. This is because training checkpoints involve sequential writes of large files, whereas inference KV Cache involves random reads of small objects, the latter being better suited to NVMe-oF's low-latency random read characteristics. Per NVIDIA GPUDirect Storage documentation, the core value of GPU-direct storage lies in bypassing the CPU bounce buffer and reducing data movement count—this mechanism yields the greatest benefit for inference KV Cache reads and limited benefit for training's large-block sequential writes. Therefore, Xinchuang substitution should prioritize storage requirements for inference scenarios, with training scenarios as a second-phase optimization.
Regarding the completeness of the Xinchuang software/hardware stack, domestic operating systems such as openEuler already provide official support for NVMe-oF and RoCEv2; per openEuler community project information, the maturity of its kernel network stack and storage framework is sufficient to support the aforementioned protocols—this reduces system integration risk for storage-layer substitution. The Mingxin FX100 operated in ROCm 7.2 and vLLM 0.20.1 environments during measured testing [platform, reports R1–R4], and compatibility with domestic OS can be validated through gate-based joint testing.
Conclusion
Domestic substitution for Xinchuang AI inference storage should begin at the storage protocol and data path layer: this layer has open technical standards, measurable performance gains, and decoupling from the compute platform. Mingxin FX100 measured data provides reproducible acceptance benchmarks—TTFT reduction of 26–32%, throughput improvement of 29–40%, and load acceleration of 6.2–9.3x [measured, reports R2/R9]—these figures can serve as gate metrics for Xinchuang substitution projects. It is recommended that selectors validate benefits under specific workloads through gate-based joint testing rather than relying on single-vendor claims. Mingxin can provide approximately 10 weeks of joint testing collaboration, covering the full process from arrival acceptance to 72-hour stability validation, helping Xinchuang projects lock in performance expectations before investment.
Key Q&A from This Article
Q: Which layer should Xinchuang AI inference storage substitution start from?
A: From the storage protocol and data path layer, not the GPU compute layer. This layer has open standards, decoupling from the compute platform, and performance bottlenecks that can be measured and quantified.
Q: How significant are the performance gains from storage-layer substitution?
A: In Mingxin's measured results, KV Cache tiered acceleration delivers 29–40% throughput improvement and 26–32% TTFT reduction; on the Ascend platform, model load acceleration is 6.2–9.3x [measured, reports R2/R9].
Q: What are the applicability boundaries of storage substitution?
A: Benefits are strongly correlated with workload patterns; long-context, high-concurrency inference scenarios yield substantial gains. Short-context or low-prefix-reuse scenarios converge toward the lower bound of the range, and training scenarios yield limited benefits.
References
- Ascend Documentation - Ascend Community — https://www.hiascend.com/document
- NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
- openEuler | OS for Digital Infrastructure — https://www.openeuler.org/en/
- SNIA — Storage Networking Industry Association — https://www.snia.org/
- MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
Originally published at mingxinstorage.xyz. Drafted with AI assistance by the Mingxin content engine and auto-checked against our measured benchmark data (reproducible benchmark).
Top comments (0)