momentum method

Provable Acceleration of Nesterov’s Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks IJCAI 2024 Revisiting Convergence of AdaGrad with Relaxed Assumptions UAI 2024 Tight Convergence Rate Bounds for Optimization Under Power Law Spectral Conditions JMLR 2024 A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta. ICLR 2023 Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity AISTATS 2023 Communication-Efficient Stochastic Gradient Descent Ascent with Momentum Algorithms IJCAI 2023 Does Momentum Help in Stochastic Optimization? A Sample Complexity Analysis. UAI 2023 Generalized Polyak Step Size for First Order Optimization with Momentum ICML 2023 Proximal Stochastic Recursive Momentum Methods for Nonconvex Composite Decentralized Optimization AAAI 2023 Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning ICML 2023 Unifying Nesterov’s Accelerated Gradient Methods for Convex and Strongly Convex Objective Functions ICML 2023 Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization JMLR 2022 A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract) AAAI 2022 Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum L4DC 2022 Does Momentum Change the Implicit Regularization on Separable Data? NeurIPS 2022 Gradient Temporal Difference with Momentum: Stability and Convergence AAAI 2022 Improve Single-Point Zeroth-Order Optimization Using High-Pass and Low-Pass Filters ICML 2022 KOALA: A Kalman Optimization Algorithm with Loss Adaptivity AAAI 2022 MDPGT: Momentum-Based Decentralized Policy Gradient Tracking AAAI 2022 Momentum Accelerates the Convergence of Stochastic AUPRC Maximization AISTATS 2022 Momentum Aggregation for Private Non-convex ERM NeurIPS 2022 Nesterov Accelerated Shuffling Gradient Method for Convex Optimization ICML 2022 On the Last Iterate Convergence of Momentum Methods ALT 2022 Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Lojasiewicz Functions when the Non-Convexity is Averaged-Out ICML 2022 Super-Acceleration with Cyclical Step-sizes AISTATS 2022