← Optimization & Theory
Deep Learning › Optimization & Theory ›

Neural Network Optimization

902 directly classified papers

Papers per year

Papers

Differentiable hierarchical and surrogate gradient search for spiking neural networks NIPS 2022
Automatic differentiation of nonsmooth iterative algorithms NIPS 2022
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent NIPS 2022
The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NIPS 2022
Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution NIPS 2022
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study NIPS 2022
Local Identifiability of Deep ReLU Neural Networks: the Theory NIPS 2022
Adam Can Converge Without Any Modification On Update Rules NIPS 2022
EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring NIPS 2022
BLOX: Macro Neural Architecture Search Benchmark and Algorithms NIPS 2022
Interpreting Operation Selection in Differentiable Architecture Search: A Perspective from Influence-Directed Explanations NIPS 2022
Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search NIPS 2022
On the Parameterization and Initialization of Diagonal State Space Models NIPS 2022
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) NIPS 2022
Explain My Surprise: Learning Efficient Long-Term Memory by predicting uncertain outcomes NIPS 2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NIPS 2022
Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis NIPS 2022
ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks CVPR 2021
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent EMNLP 2021
Second-Order Neural ODE Optimizer NIPS 2021
When Are Solutions Connected in Deep Networks? NIPS 2021
Reverse engineering learned optimizers reveals known and novel mechanisms NIPS 2021
Particle Dual Averaging: Optimization of Mean Field Neural Network with Global Convergence Rate Analysis NIPS 2021
Heavy Ball Neural Ordinary Differential Equations NIPS 2021
The Implicit Bias of Minima Stability: A View from Function Space NIPS 2021