conftrace_

momentum optimization

10 papers

Explore in graph

Co-occurring keywords

convergence rate (607) gradient descent (1144) stochastic gradient descent (1091) neural network (6616) momentum method (75) neural network optimization (1293) deep learning (2118) state vector (1) sparse mixture of expert (6) expert routing (44)

Papers

Explicit and Implicit Graduated Optimization in Deep Neural Networks AAAI 2025
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts NIPS 2024
In-Context Learning State Vector with Inner and Momentum Optimization NIPS 2024
On the Hyperparameters in Stochastic Gradient Descent with Momentum JMLR 2024
Quantitative Convergences of Lie Group Momentum Optimizers NIPS 2024
Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning ICML 2023
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers ICML 2023
Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions ICML 2023
On the Global Optimum Convergence of Momentum-based Policy Gradient AISTATS 2022
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Lojasiewicz Functions when the Non-Convexity is Averaged-Out ICML 2022