The WebGPU matmul sweep started with a naive kernel, then added 16 by 16 workgroup tiling and a 4 by 4 output block per thread.
At a 2048 cubed matrix size, the measured time moved from 47.24 ms for the naive kernel to 17.23 ms for tiling and 9.12 ms for the 4 by 4 blocked kernel. The blocked version was 5.18 times faster than naive at that size.
The obvious next idea was an 8 by 8 block. It increased reuse on paper and lost at every measured size. At 2048 cubed, the 4 by 4 version took 10.15 ms while the 8 by 8 version took 11.52 ms. The likely cause was register pressure and lower workgroup occupancy.
Packed f16 storage was another apparent win. It beat the naive baseline but became slower when stacked on the already tiled kernel. Both optimizations were attacking the same bandwidth limit, so the speedups did not compound.
Keeping these rejected variants in the record prevents the next optimization pass from rediscovering them.
The complete sweep is at https://posttrainllm.com/devlog.
Top comments (0)