Optimization

The Devil is in the Details: On the Pitfalls of Vocabulary Selection in Neural Machine Translation NAACL 2022 The Fast Kernel Transform AISTATS 2022 The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022 The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection AAAI 2022 The role of optimization geometry in single neuron learning AISTATS 2022 Three-stage Evolution and Fast Equilibrium for SGD with Non-degerate Critical Points ICML 2022 Tight Neural Network Verification via Semidefinite Relaxations and Linear Reformulations AAAI 2022 To Fold or Not to Fold: a Necessary and Sufficient Condition on Batch-Normalization Layers Folding IJCAI 2022 Towards an Effective Orthogonal Dictionary Convolution Strategy AAAI 2022 Towards Better Plasticity-Stability Trade-Off in Incremental Learning: A Simple Linear Connector CVPR 2022 Towards Efficient and Scalable Sharpness-Aware Minimization CVPR 2022 Towards understanding how momentum improves generalization in deep learning ICML 2022 Tricks for Training Sparse Translation Models NAACL 2022 Understanding and Increasing Efficiency of Frank-Wolfe Adversarial Training CVPR 2022 Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search NeurIPS 2022 Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization JMLR 2022 What Language Model to Train if You Have One Million GPU Hours? EMNLP 2022 When Does Differentially Private Learning Not Suffer in High Dimensions? NeurIPS 2022 When Do Flat Minima Optimizers Work? NeurIPS 2022 Zonotope Domains for Lagrangian Neural Network Verification NeurIPS 2022 1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed ICML 2021 3D CNNs With Adaptive Temporal Feature Resolutions CVPR 2021 Accelerating Continuous Normalizing Flow with Trajectory Polynomial Regularization AAAI 2021 Acceleration via Fractal Learning Rate Schedules ICML 2021 Achieving Robustness in Classification Using Optimal Transport With Hinge Regularization CVPR 2021