Neural Network Optimization

Neural Contextual Bandits with UCB-based Exploration ICML 2020 Neural Inheritance Relation Guided One-Shot Layer Assignment Search AAAI 2020 NeuralScale: Efficient Scaling of Neurons for Resource-Constrained Deep Neural Networks CVPR 2020 Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks Using PAC-Bayesian Analysis ICML 2020 Norm-Based Curriculum Learning for Neural Machine Translation ACL 2020 On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning AAAI 2020 On the Generalization Benefit of Noise in Stochastic Gradient Descent ICML 2020 On the Variance of the Adaptive Learning Rate and Beyond ICLR 2020 On Warm-Starting Neural Network Training NeurIPS 2020 Optimizing Millions of Hyperparameters by Implicit Differentiation AISTATS 2020 Orthogonal Convolutional Neural Networks CVPR 2020 Orthogonal Gradient Descent for Continual Learning AISTATS 2020 Orthogonalized SGD and Nested Architectures for Anytime Neural Networks ICML 2020 Overcoming Multi-Model Forgetting in One-Shot NAS With Diversity Maximization CVPR 2020 PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices AAAI 2020 Quantization Aware Training with Absolute-Cosine Regularization for Automatic Speech Recognition Interspeech 2020 Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning AAAI 2020 Query-Key Normalization for Transformers EMNLP 2020 Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate NeurIPS 2020 Reducing Underflow in Mixed Precision Training by Gradient Scaling IJCAI 2020 Regularization on Spatio-Temporally Smoothed Feature for Action Recognition CVPR 2020 Regularizing Neural Networks via Minimizing Hyperspherical Energy CVPR 2020 Reinforcing Neural Network Stability with Attractor Dynamics AAAI 2020 ReSprop: Reuse Sparsified Backpropagation CVPR 2020 Rethinking Bias-Variance Trade-off for Generalization of Neural Networks ICML 2020