conftrace_

momentum optimization

10 papers

Explore in graph

Co-occurring keywords · papers mentioning both

convergence rate (3) gradient descent (3) stochastic gradient descent (3) neural network (2) momentum method (2) neural network optimization (2) deep learning (2) state vector (1) sparse mixture of expert (1) expert routing (1)

Papers

Explicit and Implicit Graduated Optimization in Deep Neural Networks AAAI 2025 In-Context Learning State Vector with Inner and Momentum Optimization NeurIPS 2024 MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts NeurIPS 2024 On the Hyperparameters in Stochastic Gradient Descent with Momentum JMLR 2024 Quantitative Convergences of Lie Group Momentum Optimizers NeurIPS 2024 Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers ICML 2023 Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions ICML 2023 Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning ICML 2023 On the Global Optimum Convergence of Momentum-based Policy Gradient AISTATS 2022 Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Lojasiewicz Functions when the Non-Convexity is Averaged-Out ICML 2022