Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization

Aleksandra Nowak; Łukasz Gniecki; Filip Szatkowski; Jacek Tabor

2024 ICML ICML 2024

Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Aleksandra Nowak, Łukasz Gniecki, Filip Szatkowski, Jacek Tabor

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Neural Networks

Keywords

dynamical isometry givens rotation sparse initialization dense initialization arbitrary densities orthogonality

Download PDF