Neural Network Optimization

Triple descent and the two kinds of overfitting: where & why do they appear? NeurIPS 2020 TRP: Trained Rank Pruning for Efficient Deep Neural Networks IJCAI 2020 TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog EMNLP 2020 Two Routes to Scalable Credit Assignment without Weight Symmetry ICML 2020 Ultra-Low Precision 4-bit Training of Deep Neural Networks NeurIPS 2020 Uncertainty-Aware Lookahead Factor Models for Quantitative Investing ICML 2020 Understanding and Exploring the Network with Stochastic Architectures NeurIPS 2020 Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks NeurIPS 2020 Understanding Attention for Text Classification ACL 2020 Understanding the Difficulty of Training Transformers EMNLP 2020 Understanding the Disharmony between Weight Normalization Family and Weight Decay AAAI 2020 Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning EMNLP 2020 Understanding the Role of Training Regimes in Continual Learning NeurIPS 2020 Understanding Why Neural Networks Generalize Well Through GSNR of Parameters ICLR 2020 Unfolding recurrence by Green’s functions for optimized reservoir computing NeurIPS 2020 Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks NeurIPS 2020 Universal Average-Case Optimality of Polyak Momentum ICML 2020 VECA: A Method for Detecting Overfitting in Neural Networks (Student Abstract) AAAI 2020 Visual Speech In Real Noisy Environments (VISION): A Novel Benchmark Dataset and Deep Learning-Based Baseline System Interspeech 2020 What Makes Training Multi-Modal Classification Networks Hard? CVPR 2020 When BERT Plays the Lottery, All Tickets Are Winning EMNLP 2020 Why and when should you pool? Analyzing Pooling in Recurrent Architectures EMNLP 2020 Why Gradient Clipping Accelerates Training: A Theoretical Justification for Adaptivity ICLR 2020 Wide Hidden Expansion Layer for Deep Convolutional Neural Networks WACV 2020 Winnowing with Gradient Descent COLT 2020