Skip to content

DEV Community

Distributed LLM Series' Articles

Back to Lewis Won's Series

Cover image for Demystifying GPUs: From Core Architecture to Scalable Systems

Lewis Won

Jul 20 '25

Demystifying GPUs: From Core Architecture to Scalable Systems

#nvidia #gpu #architecture #cuda

12 min read

Cover image for From Scatter to All-Reduce: A Plain-English Guide to Collective Operations

Lewis Won

Jul 25 '25

From Scatter to All-Reduce: A Plain-English Guide to Collective Operations

#programming #distributedsystems

21 min read

Cover image for ZeRO by hand with a 4-parameter model

Lewis Won

Aug 1 '25

ZeRO by hand with a 4-parameter model

#distributedsystems #llm #machinelearning #ai

23 min read

Cover image for Tensor parallelism by hand

Lewis Won

Aug 23 '25

Tensor parallelism by hand

#machinelearning #pytorch #distributedsystems #llm

28 min read