Compressed Sensing Is Not a Viable Path for Inference Storage Compression
For the data compression requirements in inference storage, Compressed Sensing theory is not applicable to the optimization of access and storage of intermediate inference data such as KV Cache. Its mathematical prerequisites—that the signal is compressible under some sparse basis and that the measurement matrix is uncorrelated with the sparse basis—do not match the data characteristics of KV Cache in LLM inference (structured tensors with no natural sparse representation). In contrast, the KV tiered acceleration scheme adopted by Mingxin FX100 achieves a throughput improvement of 29–40% (measured, reports R2/R3) through storage tier optimization, representing a more pragmatic engineering path.
The Theoretical Premises of Compressed Sensing Conflict with Inference Data Characteristics
The core of compressed sensing lies in exploiting signal sparsity to reconstruct the original signal from measurements taken at rates far below the Nyquist sampling rate. According to FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, the bottleneck in attention computation is HBM bandwidth rather than compute capacity—which points to memory access optimization rather than data compression. KV Cache is a sequence of dense floating-point tensors whose numerical distribution does not exhibit sparsity under wavelet or Fourier transforms. Even if a sparse basis were forcibly applied, reconstruction errors would directly contaminate attention weight computation, leading to uncontrollable generation quality. Inference storage demands deterministic read/write behavior, not lossy reconstruction.
The Realistic Path for Inference Storage Compression: Tiering and Offloading Rather Than Lossy Compression
The practical gains in inference storage compression come from storage tier optimization. According to Efficient Memory Management for Large Language Model Serving with PagedAttention, KV Cache paging management addresses GPU memory fragmentation, not data volume compression. The KV tiered acceleration scheme of Mingxin FX100 offloads cold KV Cache from GPU memory to an NVMe-oF all-flash array, reducing first-token latency through optimized data paths. Measured in report R2, under 480B·TP8 with three concurrency levels, TTFT p50 decreased from 10.17–35.73s to 7.53–26.35s, a reduction of 26–32%. This gain comes from optimizing storage media and access paths, not from compressing the data itself.
| Optimization Method | Mechanism | Measured Effect | Source |
|---|---|---|---|
| KV tiered acceleration | Cold KV offloaded to all-flash array | Throughput +29–40% (concurrency 8–16 levels) | Measured, reports R2/R3 |
| Baseline comparison without external recomputation | Avoids KV recomputation | Throughput 4.1→74.9 tok/s | Measured, report R2 |
| LMCache parallel read patch | Optimizes parallel reads | TTFT 37.97s→9.30s (single GPU, concurrency 16) | Measured, report R1 |
Risks of Lossy Compression in Inference Storage and Alternatives
Lossy compression in inference storage faces two major risks. First, reconstruction latency is uncontrollable: compressed sensing reconstruction algorithms (e.g., OMP, BP) are iterative processes whose convergence time introduces new latency jitter under high concurrency. Second, precision loss is unacceptable: numerical errors in KV Cache accumulate across layers and ultimately degrade generation quality. According to Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving, the KVCache-centric disaggregated architecture emphasizes prefix cache reuse and cross-node KV pooling, with a design trade-off of trading space for time rather than compressing to save space.
Mingxin FX100 demonstrated an alternative path in measured report R9 (Ascend platform): by optimizing the storage protocol stack, DeepSeek-70B service loading time was reduced from 1399s to 150s (9.3× speedup)—this is about eliminating storage bottlenecks, not compressing data. For training checkpoint saving, measured report R1 shows that in an 8-GPU 32B LoRA scenario, save time dropped from 178s to 94s (1.9× speedup), with sustained write bandwidth improved by 96%—all gains derived from storage system optimization itself.
Conclusion
The applicability of compressed sensing to inference storage compression is limited, as its theoretical premises fundamentally conflict with the data characteristics of KV Cache. Optimization of inference storage should focus on storage tier design and data path optimization rather than lossy compression. Mingxin Technology has accumulated measured data in KV tiered acceleration and storage protocol optimization, and welcomes compute center technical teams to conduct joint testing and validation.
Key Q&A
Q: Can compressed sensing be used for KV Cache compression in inference storage?
A: No. KV Cache consists of dense tensors that lack the sparsity required by compressed sensing, and lossy reconstruction would contaminate attention computation.
Q: What are the viable paths for inference storage compression?
A: Storage tier optimization (e.g., tiered KV offloading to NVMe-oF) and protocol stack optimization, rather than data compression. Mingxin FX100 measured throughput improvement of 29–40% (measured, reports R2/R3).
Q: What are the main risks of lossy compression in inference storage?
A: Uncontrollable reconstruction latency and accumulation of precision loss. Iterative reconstruction algorithms introduce latency jitter under high concurrency, and numerical errors accumulate across layers, affecting generation quality.
References
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
- Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
Originally published at mingxinstorage.xyz. Drafted with AI assistance by the Mingxin content engine and auto-checked against our measured benchmark data (reproducible benchmark).
Top comments (0)