flat minima

Where Do Large Learning Rates Lead Us? NeurIPS 2024 Why is parameter averaging beneficial in SGD? An objective smoothing perspective AISTATS 2024 Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks AAAI 2023 An Adaptive Policy to Employ Sharpness-Aware Minimization ICLR 2023 An SDE for Modeling SAM: Theory and Insights ICML 2023 Boosting Adversarial Transferability by Achieving Flat Local Maxima NeurIPS 2023 Class-Conditional Sharpness-Aware Minimization for Deep Long-Tailed Recognition CVPR 2023 Differentially Private Sharpness-Aware Training ICML 2023 Enhancing Sharpness-Aware Optimization Through Variance Suppression NeurIPS 2023 Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation CVPR 2023 Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization CVPR 2023 Lookaround Optimizer: k steps around, 1 step average NeurIPS 2023 Randomized Adversarial Training via Taylor Expansion CVPR 2023 Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models ICML 2023 Sharpness-Aware Gradient Matching for Domain Generalization CVPR 2023 Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias ICML 2023 Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum ICML 2022 Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data NeurIPS 2022 Fisher SAM: Information Geometry and Sharpness Aware Minimisation ICML 2022 Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging EMNLP 2022 Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape AISTATS 2022 Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning ICML 2022 ROSE: Robust Selective Fine-tuning for Pre-trained Language Models EMNLP 2022 Surrogate Gap Minimization Improves Sharpness-Aware Training ICLR 2022 The alignment property of SGD noise and how it helps select flat minima: A stability analysis NeurIPS 2022