← Optimization & Theory
Machine Learning › Optimization & Theory ›

Neural Network Optimization

3648 directly classified papers

Papers per year

Papers

Random Reshuffling: Simple Analysis with Vast Improvements NIPS 2020
DisARM: An Antithetic Gradient Estimator for Binary Latent Variables NIPS 2020
Position-based Scaled Gradient for Model Quantization and Pruning NIPS 2020
Practical Quasi-Newton Methods for Training Deep Neural Networks NIPS 2020
ShiftAddNet: A Hardware-Inspired Deep Network NIPS 2020
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval NIPS 2020
Neural Power Units NIPS 2020
On Correctness of Automatic Differentiation for Non-Differentiable Functions NIPS 2020
Understanding the Role of Training Regimes in Continual Learning NIPS 2020
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning NIPS 2020
Neural Networks Learning and Memorization with (almost) no Over-Parameterization NIPS 2020
Sharp Representation Theorems for ReLU Networks with Precise Dependence on Depth NIPS 2020
A shooting formulation of deep learning NIPS 2020
The phase diagram of approximation rates for deep neural networks NIPS 2020
Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping NIPS 2020
Rational neural networks NIPS 2020
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate NIPS 2020
Optimizing Mode Connectivity via Neuron Alignment NIPS 2020
Explicit Regularisation in Gaussian Noise Injections NIPS 2020
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks NIPS 2020
Learning to Mutate with Hypergradient Guided Population NIPS 2020
MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles NIPS 2020
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks NIPS 2020
A Theoretical Framework for Target Propagation NIPS 2020
Learning Parities with Neural Networks NIPS 2020