conftrace_

momentum optimizer

7 papers

Explore in graph

Co-occurring keywords · papers mentioning both

stochastic gradient descent (4) deep learning (3) iterate averaging (1) exponential moving average (1) gradient accumulation (1) adaptive optimizer (1) scale invariance (1) large language model (1) moving average (1) stochastic gradient noise (1)

Papers

Dynamic Momentum Recalibration in Online Gradient Learning CVPR 2026 Periodical Moving Average Accelerates Gradient Accumulation for Post-Training UAI 2025 The AdEMAMix Optimizer: Better, Faster, Older ICLR 2025 The Road Less Scheduled NeurIPS 2024 AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights ICLR 2021 Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization ICML 2021 L4: Practical loss-based stepsize adaptation for deep learning NeurIPS 2018