Neural Network Optimization

Why Random Pruning Is All We Need to Start Sparse ICML 2023 Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model NeurIPS 2023 WLD-Reg: A Data-Dependent Within-Layer Diversity Regularizer AAAI 2023 YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors CVPR 2023 SO(2)-Equivariant Reinforcement Learning ICLR 2022 Accelerated SGD for Non-Strongly-Convex Least Squares COLT 2022 Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization JMLR 2022 Accelerating Adaptive Cubic Regularization of Newton's Method via Random Sampling JMLR 2022 Accelerating DETR Convergence via Semantic-Aligned Matching CVPR 2022 Accelerating training of batch normalization: A manifold perspective UAI 2022 A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases NeurIPS 2022 A Closer Look at Parameter Contributions When Training Neural Language and Translation Models COLING 2022 A Combinatorial Perspective on the Optimization of Shallow ReLU Networks NeurIPS 2022 A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks NeurIPS 2022 A Computable Definition of the Spectral Bias AAAI 2022 A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows AISTATS 2022 AdaBlock: SGD with Practical Block Diagonal Matrix Adaptation for Deep Learning AISTATS 2022 AdaLoss: A Computationally-Efficient and Provably Convergent Adaptive Gradient Method AAAI 2022 Adam Can Converge Without Any Modification On Update Rules NeurIPS 2022 Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum ICML 2022 Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification COLING 2022 A Differential Entropy Estimator for Training Neural Networks ICML 2022 Adversarial Attack for Asynchronous Event-Based Data AAAI 2022 AdwU-Net: Adaptive Depth and Width U-Net for Medical Image Segmentation by Differentiable Neural Architecture Search MIDL 2022 A Fast Post-Training Pruning Framework for Transformers NeurIPS 2022