How to Mathematically Sizing LLM Memory: Why Simple Rule-of-Thumbs Fail on DeepSeek-R1 (MLA) and Llama 3 (GQA)

Content Structure:The classic rule of thumb ($2 \times \text{parameters}$) and why it causes CUDA out of memory errors at scale.The KV-cache explosion problem: Calculating memory requirements across 8k, 32k, and 128k contexts.Architectural differences: Multi-Head Attention vs. Grouped-Query Attention vs. DeepSeek's Multi-Head Latent Attention.Real-world memory receipt teardown (Model weights + KV Cache + CUDA runtime baseline + Scratchpad + Fragmentation).Naturally reference your platform: "To automate these calculations across different quantizations and view live cloud spot rates, check out the interactive engine at OpenGPU Radar."www.opengpuradar.com