Neural Network Optimization

BackPACK: Packing more into Backprop ICLR 2020 Bad Global Minima Exist and SGD Can Reach Them NeurIPS 2020 Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks NeurIPS 2020 Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods NeurIPS 2020 Better depth-width trade-offs for neural networks through the lens of dynamical systems ICML 2020 Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks AAAI 2020 Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness AISTATS 2020 Beyond Lazy Training for Over-parameterized Tensor Decomposition NeurIPS 2020 Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? ICML 2020 Bidirectional LSTM Network with Ordered Neurons for Speech Enhancement Interspeech 2020 BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations ICLR 2020 Biologically Inspired Sleep Algorithm for Reducing Catastrophic Forgetting in Neural Networks AAAI 2020 Biologically Plausible Sequence Learning with Spiking Neural Networks AAAI 2020 Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation CVPR 2020 Boosting First-Order Methods by Shifting Objective: New Schemes with Faster Worst-Case Rates NeurIPS 2020 Bounds on Over-Parameterization for Guaranteed Existence of Descent Paths in Shallow ReLU Networks ICLR 2020 Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness ICLR 2020 Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints ICLR 2020 Cascading Convolutional Color Constancy AAAI 2020 Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks JMLR 2020 Checkpoint Reranking: An Approach to Select Better Hypothesis for Neural Machine Translation Systems ACL 2020 Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks IJCAI 2020 Cloth in the Wind: A Case Study of Physical Measurement Through Simulation CVPR 2020 Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization ICLR 2020 Collegial Ensembles NeurIPS 2020