Neural Network Optimization

Generalization Properties of NAS under Activation and Skip Connection Search NeurIPS 2022 Generalized Federated Learning via Sharpness Aware Minimization ICML 2022 Global Convergence and Stability of Stochastic Gradient Descent NeurIPS 2022 Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning CVPR 2022 GraB: Finding Provably Better Data Permutations than Random Reshuffling NeurIPS 2022 Gradient Descent on Neurons and its Link to Approximate Second-order Optimization ICML 2022 Gradient Descent: The Ultimate Optimizer NeurIPS 2022 Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs NeurIPS 2022 Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win AAAI 2022 GradMax: Growing Neural Networks using Gradient Information ICLR 2022 GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks WACV 2022 Group-based Interleaved Pipeline Parallelism for Large-scale DNN Training ICLR 2022 Half-Inverse Gradients for Physical Deep Learning ICLR 2022 Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit NeurIPS 2022 Hierarchical Transformers Are More Efficient Language Models NAACL 2022 High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation NeurIPS 2022 High-dimensional limit theorems for SGD: Effective dynamics and critical scaling NeurIPS 2022 High Probability Guarantees for Nonconvex Stochastic Gradient Descent with Heavy Tails ICML 2022 Hindsight Network Credit Assignment: Efficient Credit Assignment in Networks of Discrete Stochastic Units AAAI 2022 HNO: High-Order Numerical Architecture for ODE-Inspired Deep Unfolding Networks AAAI 2022 HoD-Net: High-Order Differentiable Deep Neural Networks and Applications AAAI 2022 Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations NeurIPS 2022 How many degrees of freedom do we need to train deep networks: a loss landscape perspective ICLR 2022 How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective ICML 2022 Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials NeurIPS 2022