Neural Network Optimization

General Cutting Planes for Bound-Propagation-Based Neural Network Verification NeurIPS 2022 Gradient and Mangitude Based Pruning for Sparse Deep Neural Networks AAAI 2022 Gradient Descent: The Ultimate Optimizer NeurIPS 2022 Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs NeurIPS 2022 Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win AAAI 2022 GreedyNASv2: Greedier Search With a Greedy Path Filter CVPR 2022 HNO: High-Order Numerical Architecture for ODE-Inspired Deep Unfolding Networks AAAI 2022 HoD-Net: High-Order Differentiable Deep Neural Networks and Applications AAAI 2022 Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations NeurIPS 2022 Hyperverlet: A Symplectic Hypersolver for Hamiltonian Systems AAAI 2022 I AM A.I. Gradient Descent – an Open-Source Digital Game for Inquiry-Based CLIL Learning AAAI 2022 Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models EMNLP 2022 Interlocking Backpropagation: Improving depthwise model-parallelism JMLR 2022 Interpreting Operation Selection in Differentiable Architecture Search: A Perspective from Influence-Directed Explanations NeurIPS 2022 ISNAS-DIP: Image-Specific Neural Architecture Search for Deep Image Prior CVPR 2022 JFB: Jacobian-Free Backpropagation for Implicit Networks AAAI 2022 KOALA: A Kalman Optimization Algorithm with Loss Adaptivity AAAI 2022 Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect ICLR 2022 Large-Scale Graph Neural Architecture Search ICML 2022 LayerConnect: Hypernetwork-Assisted Inter-Layer Connector to Enhance Parameter Efficiency COLING 2022 Learning by Directional Gradient Descent ICLR 2022 Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training JMLR 2022 Local Identifiability of Deep ReLU Neural Networks: the Theory NeurIPS 2022 Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations NeurIPS 2022 Masked Training of Neural Networks with Partial Gradients AISTATS 2022