Neural Network Optimization

Generalization Guarantees for Neural Architecture Search with Train-Validation Split ICML 2021 Generalized Spatio-Temporal RNN Beamformer for Target Speech Separation Interspeech 2021 Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances ICML 2021 Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs ICML 2021 Going Deeper With Directly-Trained Larger Spiking Neural Networks AAAI 2021 Going Deeper With Image Transformers ICCV 2021 Goten: GPU-Outsourcing Trusted Execution of Neural Network Training AAAI 2021 Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability ICLR 2021 Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias NeurIPS 2021 Gradient Methods Never Overfit On Separable Data JMLR 2021 Gradient Projection Memory for Continual Learning ICLR 2021 Gradient Regularization for Noise-Robust Speaker Verification Interspeech 2021 Gradient Starvation: A Learning Proclivity in Neural Networks NeurIPS 2021 GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training NeurIPS 2021 GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training ICML 2021 Group Whitening: Balancing Learning Efficiency and Representational Capacity CVPR 2021 G-Transformer for Document-Level Machine Translation ACL 2021 Guarantees for Tuning the Step Size using a Learning-to-Learn Approach ICML 2021 HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search ICML 2021 Heavy Ball Neural Ordinary Differential Equations NeurIPS 2021 Hessian Eigenspectra of More Realistic Nonlinear Models NeurIPS 2021 "Hey, that’s not an ODE": Faster ODE Adjoints via Seminorms ICML 2021 Hierarchical Context-Aware Transformers for Non-Autoregressive Text to Speech Interspeech 2021 High-Capacity Expert Binary Networks ICLR 2021 High-Performance Large-Scale Image Recognition Without Normalization ICML 2021