Model pruning removes redundant weights from a neural network to reduce memory footprint and accelerate inference. For large language models, where parameter counts routinely exceed tens of billions, pruning is one of the most direct ways to shrink model size without training a smaller architecture from scratch. The technique is especially relevant for production pipelines where latency budgets are tight and GPU memory is a binding constraint.
What Is Model Pruning?
Pruning zeroes out or physically removes weights based on saliency criteria. The goal is to introduce sparsity, which compresses the model and can speed up matrix multiplications when hardware and kernels support sparse formats.
Unstructured pruning targets individual weights. It can achieve high compression rates but often yields irregular memory access patterns that standard GPU kernels do not accelerate efficiently.
Structured pruning removes entire components, such as heads, layers, or channels. It preserves dense matrix shapes and
Top comments (0)