conftrace_

adaptive gradient method

28 papers

Explore in graph

Co-occurring keywords

stochastic gradient descent (1091) adaptive gradient (24) neural network optimization (1293) stochastic optimization (1060) nonconvex optimization (316) adam optimizer (54) convergence guarantee (169) learning rate (192) gradient descent (1144) convergence analysis (395)

Papers

On the O(sqrt(d)/T^(1/4)) Convergence Rate of RMSProp and Its Momentum Extension Measured by l_1 Norm JMLR 2025
Faster Double Adaptive Gradient Methods AAAI 2025
AdaR: An Adaptive Gradient Method with Cyclical Restarting of Moment Estimations IJCAI 2025
ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate NIPS 2024
Iterate Averaging in the Quest for Best Test Error JMLR 2024
Open Problem: Black-Box Reductions and Adaptive Gradient Methods for Nonconvex Optimization COLT 2024
Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction NIPS 2024
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad NIPS 2024
Adaptive Proximal Gradient Method for Convex Optimization NIPS 2024
Fed-LAMB: Layer-wise and Dimension-wise Locally Adaptive Federated Learning UAI 2023
CAME: Confidence-guided Adaptive Memory Efficient Optimization ACL 2023
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NIPS 2022
Robust Training of Neural Networks Using Scale Invariant Architectures ICML 2022
A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022
AdaBlock: SGD with Practical Block Diagonal Matrix Adaptation for Deep Learning AISTATS 2022
On the Convergence of Decentralized Adaptive Gradient Methods ACML 2022
Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities AAAI 2021
Adaptive Proximal Gradient Methods for Structured Neural Networks NIPS 2021
An Optimistic Acceleration of AMSGrad for Nonconvex Optimization ACML 2021
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization AAAI 2021
Domain-Independent Dominance of Adaptive Methods CVPR 2021
Towards Better Generalization of Adaptive Gradient Methods NIPS 2020
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks IJCAI 2020
Escaping Saddle Points with Adaptive Gradient Methods ICML 2019
Adaptive Methods for Nonconvex Optimization NIPS 2018