signal propagation

Fast Training of Sinusoidal Neural Fields via Scaling Initialization ICLR 2025 Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis ICLR 2025 Nonlinear spiked covariance matrices and signal propagation in deep neural networks COLT 2024 On Feature Learning in Structured State Space Models NeurIPS 2024 PANDA: Expanded Width-Aware Message Passing Beyond Rewiring ICML 2024 Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion ICLR 2024 Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models ICML 2024 Transformers need glasses! Information over-squashing in language tasks NeurIPS 2024 AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks AAAI 2023 Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation ICLR 2023 Principled Weight Initialisation for Input-Convex Neural Networks NeurIPS 2023 Feature Learning and Signal Propagation in Deep Neural Networks ICML 2022 Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse NeurIPS 2022 Characterizing signal propagation to close the performance gap in unnormalized ResNets ICLR 2021 Fractional moment-preserving initialization schemes for training deep neural networks AISTATS 2021 Regularization in ResNet with Stochastic Depth NeurIPS 2021 Sparse linear networks with a fixed butterfly structure: theory and practice UAI 2021 A Signal Propagation Perspective for Pruning Neural Networks at Initialization ICLR 2020 Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? ICML 2020 Lamina-specific neuronal properties promote robust, stable signal propagation in feedforward networks NeurIPS 2020 A Mean Field Theory of Batch Normalization ICLR 2019 A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off NeurIPS 2019 Characterizing Well-Behaved vs. Pathological Deep Neural Networks ICML 2019 Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function AISTATS 2019 Initialization of ReLUs for Dynamical Isometry NeurIPS 2019