Neural Network Optimization

Neural Network Pruning Denoises the Features and Makes Local Connectivity Emerge in Visual Tasks ICML 2022 Neural Networks can Learn Representations with Gradient Descent COLT 2022 Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective ICML 2022 Neural Piecewise-Constant Delay Differential Equations AAAI 2022 Neural Tangent Kernel Analysis of Deep Narrow Neural Networks ICML 2022 NODEO: A Neural Ordinary Differential Equation Based Optimization Framework for Deformable Image Registration CVPR 2022 (Non-)Convergence Results for Predictive Coding Networks ICML 2022 Non-Uniform Adversarially Robust Pruning AutoML 2022 Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again NeurIPS 2022 On Biased Stochastic Gradient Estimation JMLR 2022 On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems JMLR 2022 On Convergence of Gradient Descent Ascent: A Tight Local Analysis ICML 2022 On-Device Training Under 256KB Memory NeurIPS 2022 On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptation NeurIPS 2022 On feature learning in neural networks with global convergence guarantees ICLR 2022 On global convergence of ResNets: From finite to infinite width using linear parameterization NeurIPS 2022 Online Convolutional Re-Parameterization CVPR 2022 On Non-local Convergence Analysis of Deep Linear Networks ICML 2022 On the Benefits of Large Learning Rates for Kernel Methods COLT 2022 On the Convergence of Decentralized Adaptive Gradient Methods ACML 2022 On the Double Descent of Random Features Models Trained with SGD NeurIPS 2022 On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias NeurIPS 2022 On the Finite-Time Complexity and Practical Computation of Approximate Stationarity Concepts of Lipschitz Functions ICML 2022 On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications ICLR 2022 On the Implicit Bias of Gradient Descent for Temporal Extrapolation AISTATS 2022