conftrace_

momentum optimizer

3 papers

Explore in graph

Co-occurring keywords

deep learning (2118) stochastic gradient descent (1091) iterate averaging (14) adaptive optimizer (14) stepsize adaptation (3) stochastic gradient noise (3) learning rate schedule (13) loss function (462) implicit regularization (93) adam optimizer (54)

Papers

The Road Less Scheduled NIPS 2024
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization ICML 2021
L4: Practical loss-based stepsize adaptation for deep learning NIPS 2018