weight initialization

Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks ICLR 2020 Train-by-Reconnect: Decoupling Locations of Weights from Their Values NeurIPS 2020 What's Hidden in a Randomly Weighted Neural Network? CVPR 2020 Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask NeurIPS 2019 Initialization of ReLUs for Dynamical Isometry NeurIPS 2019 MetaInit: Initializing learning by learning to initialize NeurIPS 2019 On the Impact of the Activation function on Deep Neural Networks Training ICML 2019 The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks ICLR 2019 Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation ACL 2019 How to Start Training: The Effect of Initialization and Architecture NeurIPS 2018 Invariance of Weight Distributions in Rectified MLPs ICML 2018 Learning Steerable Filters for Rotation Equivariant CNNs CVPR 2018 Multi-Task Learning of Speech Recognition and Speech Synthesis Parameters for Ultrasound-based Silent Speech Interfaces Interspeech 2018 Initializing Convolutional Filters with Semantic Features for Text Classification EMNLP 2017 Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice NeurIPS 2017 Bank of Weight Filters for Deep CNNs ACML 2016 Learning by Stretching Deep Networks ICML 2014 Pre-training of Recurrent Neural Networks via Linear Autoencoders NeurIPS 2014 Understanding the difficulty of training deep feedforward neural networks AISTATS 2010