← Optimization & Theory
Deep Learning › Optimization & Theory ›

Neural Network Optimization

902 directly classified papers

Papers per year

Papers

Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022
What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation ACL 2022
Controllable Dynamic Multi-Task Architectures CVPR 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs CVPR 2022
Revisiting Random Channel Pruning for Neural Network Compression CVPR 2022
Smooth Maximum Unit: Smooth Activation Function for Deep Networks Using Smoothing Maximum Technique CVPR 2022
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models EMNLP 2022
The Devil in Linear Transformer EMNLP 2022
VectorAdam for Rotation Equivariant Geometry Optimization NIPS 2022
Chaotic Dynamics are Intrinsic to Neural Network Training with SGD NIPS 2022
Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again NIPS 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization NIPS 2022
Gradient Descent: The Ultimate Optimizer NIPS 2022
A Closer Look at the Adversarial Robustness of Deep Equilibrium Models NIPS 2022
Towards Practical Control of Singular Values of Convolutional Layers NIPS 2022
ZARTS: On Zero-order Optimization for Neural Architecture Search NIPS 2022
Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations NIPS 2022
Stochastic Adaptive Activation Function NIPS 2022
Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes NIPS 2022
On global convergence of ResNets: From finite to infinite width using linear parameterization NIPS 2022
Towards Theoretically Inspired Neural Initialization Optimization NIPS 2022
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations NIPS 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs NIPS 2022
An In-depth Study of Stochastic Backpropagation NIPS 2022
To update or not to update? Neurons at equilibrium in deep models NIPS 2022