LLM Serving Has Three Bills: Prefill, Decode, and KV Cache

LLM serving performance is not one number. A useful mental model has at least three separate bills:

1. Prefill: processing the prompt before the first generated token.
2. Decode: generating the next token, repeatedly.
3. KV cache: keeping active context available so decode does not recompute all of history.

That is why tiny-vllm is useful reading. It is not primarily an argument to replace a production serving stack. It is a C++/CUDA learning project that walks from loading model weights through prefill and decode to batching, online softmax, and PagedAttention.

I have not tested, run, built, benchmarked, or audited tiny-vllm. This is a source-based reading of its public README and linked materials, not a performance claim or deployment recommendation.