Domestic AI Computing Center Inference Storage Selection: The Core Is Matching KV Cache Access Patterns
For inference storage selection in domestic AI computing centers, the conclusion comes first: the storage system design must revolve around the read/write characteristics of KV Cache, rather than following the general-purpose parallel file system approach used in training scenarios. In inference workloads, KV Cache access exhibits high-concurrency, low-latency, high-bandwidth random read/write patterns, which are fundamentally different from the sequential large-block writes of training checkpoints. According to Efficient Memory Management for Large Language Model Serving with PagedAttention, the paged management mechanism of KV Cache dictates a highly fragmented memory access pattern, directly challenging the I/O scheduling capabilities of traditional storage arrays. Measured data from the Mingxin FX100 in a 480B model long-context cold-restore workload shows that a storage solution optimized for the KV Cache path can improve inference throughput by 29–40% (measured, report R2/R3), providing a quantitative baseline for selection.
Three Core Dimensions of Inference Storage Selection: Balancing Bandwidth, Capacity, and Cost
Bandwidth: PCIe Generation and Network Protocol Define the I/O Ceiling
The bandwidth requirements of inference storage are determined by the aggregate memory access capability of the GPU cluster. Current domestic AI computing centers commonly deploy PCIe 4.0/5.0 platforms, and the storage system must match the I/O throughput of this generation. The Mingxin FX product line provides a clear bandwidth gradient reference: FX100 (PCIe 3.0, single-interface 100Gb), FX200 (PCIe 4.0, single-interface 200Gb), FX300 (PCIe 5.0, single-interface 400Gb) (vendor specification). According to NVIDIA GPUDirect Storage Documentation, GPU-direct storage can bypass the CPU's bounce buffer, reducing data copy overhead—a mechanism that is especially critical in high-frequency small-I/O scenarios like KV Cache. When selecting, confirm whether the storage solution supports RDMA (e.g., RoCEv2) and GPUDirect Storage; otherwise, bandwidth advantages may not translate effectively to GPU memory.
Capacity and Media: All-Flash Arrays Are the Baseline for Long-Context Inference
The KV Cache capacity requirement for long-context inference grows linearly. In long-context scenarios with a 480B model, KV Cache can occupy hundreds of GB of GPU memory, and the excess must spill over to the storage layer. Mingxin R2 measurements show that under a 480B·TP8 configuration, the TTFT p50 without external memory recomputation reaches as high as 149.5 seconds (concurrency 16), while with the FX100 it drops to 11.85 seconds, a speedup of 12.6x (measured, report R2). This demonstrates that the random read performance of the storage medium directly determines the recall speed of spilled KV Cache. An all-flash NVMe-oF array (such as the FX100's 4-drive RAID0 configuration) is the baseline choice for such workloads; mechanical hard drives or hybrid arrays cannot support the required IOPS and latency.
Cost: Balancing Price per TB and Performance
Storage budgets in domestic AI computing centers require a trade-off between capacity unit price and performance. The Mingxin FX series price list shows a reference price of ¥331,200 for a fully configured FX200 (approximately ¥1,797/TB) and ¥924,000 for the FX300 (approximately ¥5,014/TB) (reference price from price list). The FX200 delivers 32M IOPS on a PCIe 4.0 platform (vendor specification), with a significantly lower unit IOPS cost than the FX300. For inference scenarios primarily driven by throughput, the FX200 may be the more economical choice. However, if extreme concurrency of 140M IOPS is required (FX400, mass production end of 2026, pricing TBD), it will be necessary to wait for the PCIe 6.0 platform to mature. Selection should be based on actual concurrency tiers rather than blindly pursuing peak specifications.
Measured-Data-Driven Selection Methodology: From Gate Testing to Deployment Validation
Define Acceptance Criteria with Reproducible Benchmarks
Selection should not rely on vendor marketing but should establish quantifiable gate tests. Mingxin's collaboration model provides a reference framework: approximately 10 weeks of gated joint testing, including G1 arrival acceptance, G2 single-node baseline, G3 main gate (TTFT reduction ≥25%, throughput +29–40% measured in-band), and G4 72-hour stability, with early termination if targets are not met (collaboration model). This phased validation mechanism effectively reduces selection risk. For public comparisons, according to MLPerf Inference: Datacenter Benchmark Suite Results, MLCommons provides a neutral benchmark methodology for inference performance; when selecting, reference its test methodology to design internal validation plans rather than directly citing its specific numbers.
Focus on the Propagation Effect of Storage on Inference Latency
The impact of storage performance on inference latency can be quantified through end-to-end metrics. Mingxin R2 measurements show that under a 480B·TP8 configuration across three concurrency tiers, TTFT p50 decreases from 10.17–35.73 seconds to 7.53–26.35 seconds, a reduction of 26–32% (measured, report R2). This propagation effect is especially pronounced in scenarios such as cold start and long-context switching. When selecting, require vendors to provide similar tiered test data (e.g., the LMCache parallel read patch improving TTFT by 4.1x, measured, report R1), rather than only peak bandwidth or IOPS figures.
Key Q&A
Q: What is the primary consideration for inference storage selection in domestic AI computing centers?
A: The primary consideration is the storage system's adaptability to KV Cache access patterns, not general-purpose file system performance. The Mingxin FX100 achieves a throughput improvement of 29–40% (measured, reports R2/R3) in 480B long-context workloads, demonstrating that optimizations targeting the KV Cache path directly translate into inference performance gains.
Q: How do you balance storage performance and cost?
A: Calculate unit IOPS cost based on actual concurrency tiers rather than pursuing peak specifications. The Mingxin FX200 (PCIe 4.0, 32M IOPS, approximately ¥1,797/TB) may be more economical than the FX300 (approximately ¥5,014/TB) in throughput-oriented scenarios (reference price from price list), but it must be verified against the gate requirement of TTFT reduction ≥25% (collaboration model).
Q: What process should be followed for selection validation?
A: A phased gate testing approach is recommended: arrival acceptance, single-node baseline, main gate (TTFT and throughput metrics), and stability testing, with early termination if targets are not met. For public comparisons, reference the MLPerf Inference test methodology to design internal validation plans (according to MLPerf Inference: Datacenter Benchmark Suite Results), but obtain specific values through your own measurements.
References
- MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
- NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
- Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180
Originally published at mingxinstorage.xyz. Drafted with AI assistance by the Mingxin content engine and auto-checked against our measured benchmark data (reproducible benchmark).
Top comments (0)