HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces
Nasib Ullah, Jinbin Zhang, Jean Lucien Randrianantenaina, Erik Schultheis, and Rohit Babbar
Extreme multi-label classification requires learning over output spaces with millions of labels, making the output layer a memory and compute bottleneck. We introduce group-shared fixed fan-in sparsity with hardware-aware CUDA kernels, achieving up to 4.4x and 25x speedups in the forward and backward passes while matching dense-model accuracy.



