weight initialization

How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023 Mimetic Initialization of Self-Attention Layers ICML 2023 On the Convergence of Gradient Flow on Multi-layer Linear Models ICML 2023 On the Initialization of Graph Neural Networks ICML 2023 Principled Weight Initialisation for Input-Convex Neural Networks NeurIPS 2023 Sparse tree-based Initialization for Neural Networks ICLR 2023 Understanding the Covariance Structure of Convolutional Filters ICLR 2023 A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks ICML 2022 Is High Variance Unavoidable in RL? A Case Study in Continuous Control ICLR 2022 Measuring Representational Robustness of Neural Networks Through Shared Invariances ICML 2022 Towards Understanding the Condensation of Neural Networks at Initial Training NeurIPS 2022 Distance-Based Regularisation of Deep Networks for Fine-Tuning ICLR 2021 Fast Certified Robust Training with Short Warmup NeurIPS 2021 Learned Initializations for Optimizing Coordinate-Based Neural Representations CVPR 2021 Learning where to learn: Gradient sparsity in meta and continual learning NeurIPS 2021 Learning with Retrospection AAAI 2021 Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not? ICML 2021 On the mapping between Hopfield networks and Restricted Boltzmann Machines ICLR 2021 Revisiting Weight Initialization of Deep Neural Networks ACML 2021 Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot? NeurIPS 2021 What’s Hidden in a One-layer Randomly Weighted Transformer? EMNLP 2021 Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? ICML 2020 Cooperative Initialization based Deep Neural Network Training WACV 2020 Improving Transformer Optimization Through Better Initialization ICML 2020 Principled Weight Initialization for Hypernetworks ICLR 2020