← Optimization & Theory
Deep Learning › Optimization & Theory ›

Neural Network Optimization

902 directly classified papers

Papers per year

Papers

Generalization Performance of Multi-pass Stochastic Gradient Descent with Convex Loss Functions JMLR 2021
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent JMLR 2021
On ADMM in Deep Learning: Convergence and Saturation-Avoidance JMLR 2021
Multiplicative Noise and Heavy Tails in Stochastic Optimization ICML 2021
Orthogonal Over-Parameterized Training CVPR 2021
Rethinking BiSeNet for Real-Time Semantic Segmentation CVPR 2021
Network Quantization With Element-Wise Gradient Scaling CVPR 2021
Train simultaneously, generalize better: Stability of gradient-based minimax learners ICML 2021
Attention is not all you need: pure attention loses rank doubly exponentially with depth ICML 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks ICML 2021
Adversarial Robustness Guarantees for Random Deep Neural Networks ICML 2021
Bayesian Deep Learning via Subnetwork Inference ICML 2021
On the Adequacy of Untuned Warmup for Adaptive Optimization AAAI 2021
ReconfigISP: Reconfigurable Camera Image Processing Pipeline ICCV 2021
ReZero is all you need: fast convergence at large depth UAI 2021
Sparse linear networks with a fixed butterfly structure: theory and practice UAI 2021
Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimization L4DC 2021
Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective ICML 2021
Zero Time Waste: Recycling Predictions in Early Exit Neural Networks NIPS 2021
The Heavy-Tail Phenomenon in SGD ICML 2021
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks ICML 2021
Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks NIPS 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling ICML 2021
Better Training using Weight-Constrained Stochastic Dynamics ICML 2021
On Monotonic Linear Interpolation of Neural Network Parameters ICML 2021