Skip to content

DEV Community

Serving at Scale Series' Articles

Back to AI Tech News's Series

Cover image for What Does It Actually Cost to Self-Host an LLM? The Batching Math Nobody Shows You

Oct 6

What Does It Actually Cost to Self-Host an LLM? The Batching Math Nobody Shows You

#llm #vllm #performance #devops

6 min read

Oct 6

KV Cache Quantization in LLM Serving: FP8 and INT8 Tradeoffs, the Silent config.json Trap, and How to Measure It Fairly

#llm #vllm #performance #devops

7 min read