conftrace_
← Optimization & Theory
Machine Learning › Optimization & Theory ›

Neural Network Optimization

3,648 papers

Papers per year

Papers

Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules NIPS 2022
PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies NIPS 2022
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks NIPS 2022
Sharpness-Aware Training for Free NIPS 2022
Generalization Properties of NAS under Activation and Skip Connection Search NIPS 2022
A Fast Post-Training Pruning Framework for Transformers NIPS 2022
Exact Solutions of a Deep Linear Network NIPS 2022
Deep Active Learning by Leveraging Training Dynamics NIPS 2022
Benign Overfitting in Two-layer Convolutional Neural Networks NIPS 2022
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling NIPS 2022
Efficient Training of Low-Curvature Neural Networks NIPS 2022
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks NIPS 2022
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent NIPS 2022
The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NIPS 2022
Does Momentum Change the Implicit Regularization on Separable Data? NIPS 2022
Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution NIPS 2022
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study NIPS 2022
Is Integer Arithmetic Enough for Deep Learning Training? NIPS 2022
A Reparametrization-Invariant Sharpness Measure Based on Information Geometry NIPS 2022
Improving Transformer with an Admixture of Attention Heads NIPS 2022
LTMD: Learning Improvement of Spiking Neural Networks with Learnable Thresholding Neurons and Moderate Dropout NIPS 2022
Adam Can Converge Without Any Modification On Update Rules NIPS 2022
M³ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design NIPS 2022
Proppo: a Message Passing Framework for Customizable and Composable Learning Algorithms NIPS 2022
Training stochastic stabilized supralinear networks by dynamics-neutral growth NIPS 2022