adaptive gradient

AdaGrad under Anisotropic Smoothness ICLR 2025 AdaRankGrad: Adaptive Gradient Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning ICLR 2025 On the O(sqrt(d)/T^(1/4)) Convergence Rate of RMSProp and Its Momentum Extension Measured by l_1 Norm JMLR 2025 Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction NeurIPS 2024 High Probability Convergence of Stochastic Gradient Methods ICML 2023 Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization ICLR 2023 AdaGrad Avoids Saddle Points ICML 2022 A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022 Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization AISTATS 2022 On the Convergence of mSGD and AdaGrad for Stochastic Optimization ICLR 2022 On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NeurIPS 2022 The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance COLT 2022 ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization AAAI 2021 Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities AAAI 2021 Momentum Centering and Asynchronous Update for Adaptive Gradient Methods NeurIPS 2021 On the distributional properties of adaptive gradients UAI 2021 SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients NeurIPS 2021 AdaGrad stepsizes: Sharp convergence over nonconvex landscapes JMLR 2020 A new regret analysis for Adam-type algorithms ICML 2020 Better Full-Matrix Regret via Parameter-Free Online Learning NeurIPS 2020 AdaGrad Stepsizes: Sharp Convergence Over Nonconvex Landscapes ICML 2019 Decoupled Weight Decay Regularization ICLR 2019 Necessary and Sufficient Geometries for Gradient Methods NeurIPS 2019 Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization ICML 2019 The Implicit Bias of AdaGrad on Separable Data NeurIPS 2019