Skip to content

DEV Community

Designing LLM Inference Systems Series' Articles

Back to Pujun Lun's Series

Cover image for From a Rust Inference Engine to Heterogeneous Serving

Pujun Lun

Oct 9

From a Rust Inference Engine to Heterogeneous Serving

#showdev #ai #llm #performance

3 min read

Cover image for Why Continuous Batching Made My LLM Inference Engine Slower

Pujun Lun

Oct 11

Why Continuous Batching Made My LLM Inference Engine Slower

#showdev #ai #llm #performance

4 min read

Cover image for Why My Prefix Cache Missed Identical Requests

Pujun Lun

Oct 16

Why My Prefix Cache Missed Identical Requests

#showdev #ai #llm #performance

6 min read

Cover image for When LLM Inference on CPU Beats Waiting for the GPU

Pujun Lun

Oct 24

When LLM Inference on CPU Beats Waiting for the GPU

#showdev #ai #llm #performance

5 min read

Cover image for What I Gave Up to Make Speculative Decoding Faster

Pujun Lun

Oct 28

What I Gave Up to Make Speculative Decoding Faster

#showdev #ai #llm #performance

5 min read

Cover image for When "Just Enable the CUDA Feature" Met Two Free GPUs

Pujun Lun

Oct 31

When "Just Enable the CUDA Feature" Met Two Free GPUs

#showdev #ai #llm #gpu

7 min read