Skip to content

DEV Community

Designing LLM Inference Systems Series' Articles

Back to Pujun Lun's Series

Cover image for From a Rust Inference Engine to Heterogeneous Serving

Pujun Lun

Oct 9

From a Rust Inference Engine to Heterogeneous Serving

#showdev #ai #llm #performance

3 min read

Cover image for Why Continuous Batching Made My LLM Inference Engine Slower

Pujun Lun

Oct 11

Why Continuous Batching Made My LLM Inference Engine Slower

#showdev #ai #llm #performance

4 min read