Key Features in v0.3.0
True 3-bit PolarQuant: Physical bit-packing (8x3-bit into 3 bytes) achieving 5.8x-6.0x compression of base KV storage with <0.1% accuracy drop.
Cross-Layer KV Delta (14x Compression): Next-gen backend that stores 3-bit anchor layers and 1-bit signed deltas for intermediate layers.
Speculative KV Prefill: Accelerates prefill phase by 2-3x using 1-bit sketches for fast draft KV generation and verification.
Temporal Expert Fusion: SVD-based merging of rarely-used experts to reclaim 20-30% of MoE weight VRAM with zero quality loss.
Cross-Request Prefix Sharing: Global manager for sharing KV blocks of common prefixes across concurrent requests.
Fast Walsh-Hadamard Transform (FWHT):
O
(
N
log
N
)rotation for faster quantization on power-of-2 dimensions.
Cryptographic KV Watermarking: HMAC-seeded LSB watermarking of KV scales for attribution and auditing.
Top comments (0)