DEV Community

LLM Inference Digest Series' Articles

Back to Felipe 0liveira's Series
The KV Cache Is the Bottleneck Now — 1-Bit Quantization, MoE Stragglers, and Per-Second GPU Billing
felipe0liveira profile

The KV Cache Is the Bottleneck Now — 1-Bit Quantization, MoE Stragglers, and Per-Second GPU Billing

#ai #cloud #llm #performance
Comments 4
5 min read
Multi-Turn Memory, Mixed Precision, and Safer Rollouts — Serving Catches Up to Agentic Workloads
felipe0liveira profile

Multi-Turn Memory, Mixed Precision, and Safer Rollouts — Serving Catches Up to Agentic Workloads

#agents #ai #llm #machinelearning
5
Comments 4
8 min read
Elastic MoE scaling, free batch-sorting wins, and DeepSeek's serving stack goes open
felipe0liveira profile

Elastic MoE scaling, free batch-sorting wins, and DeepSeek's serving stack goes open

#ai #llm #machinelearning #performance
1
Comments
12 min read
KV Cache Wars: How Compaction, Quantization, and a Rust Rewrite Are Redefining LLM Serving Economics
felipe0liveira profile

KV Cache Wars: How Compaction, Quantization, and a Rust Rewrite Are Redefining LLM Serving Economics

#llm #machinelearning #performance #rust
2
Comments 1
10 min read
Why KV-Cache Memory Tricks — Not New Math — Drove This Week's Biggest Inference Wins
felipe0liveira profile

Why KV-Cache Memory Tricks — Not New Math — Drove This Week's Biggest Inference Wins

#ai #llm #machinelearning #performance
1
Comments
8 min read