conftrace_

adaptive gradient

24 papers

Explore in graph

Also known as

ADAGRAD

Co-occurring keywords

stochastic gradient descent (1091) adaptive gradient method (28) convergence rate (607) stochastic optimization (1060) gradient descent (1144) nonconvex optimization (316) regret bound (1926) online convex optimization (165) convex optimization (1321) non-convex optimization (547)

Papers

On the O(sqrt(d)/T^(1/4)) Convergence Rate of RMSProp and Its Momentum Extension Measured by l_1 Norm JMLR 2025
Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction NIPS 2024
High Probability Convergence of Stochastic Gradient Methods ICML 2023
The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance COLT 2022
AdaGrad Avoids Saddle Points ICML 2022
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NIPS 2022
A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022
Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization AISTATS 2022
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization AAAI 2021
SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients NIPS 2021
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods NIPS 2021
On the distributional properties of adaptive gradients UAI 2021
Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities AAAI 2021
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes JMLR 2020
A new regret analysis for Adam-type algorithms ICML 2020
Better Full-Matrix Regret via Parameter-Free Online Learning NIPS 2020
Necessary and Sufficient Geometries for Gradient Methods NIPS 2019
AdaGrad Stepsizes: Sharp Convergence Over Nonconvex Landscapes ICML 2019
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization ICML 2019
The Implicit Bias of AdaGrad on Separable Data NIPS 2019
FLAG n’ FLARE: Fast Linearly-Coupled Adaptive Gradient Methods AISTATS 2018
Adaptive Methods for Nonconvex Optimization NIPS 2018
Efficient Adaptive Online Learning via Frequent Directions IJCAI 2018
Delay-Tolerant Algorithms for Asynchronous Distributed Online Learning NIPS 2014