A drop-in replacement for the standard residual connection x = x + layer(x).
WHC replaces the single residual path with n parallel lanes and learnable mixing matrices.
Key Results:
6–15% faster training
Lower final loss on benchmark tasks
Proven stability (spectral norm ≤ 1)
82 parameters for the mixing matrix
Install:
text
pip install whc
GitHub: https://github.com/fardinsabid/wHC
License: MIT
Top comments (0)