conftrace_

learning rate schedule

13 papers

Explore in graph

Also known as

LR SCHEDULE

Co-occurring keywords

stochastic gradient descent (1091) stochastic differential equation (181) gradient descent (1144) neural network optimization (1293) convex optimization (1321) step decay (2) geometric decay (2) final iterate (1) iterate averaging (14) momentum optimizer (3)

Papers

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations NIPS 2024
The Road Less Scheduled NIPS 2024
Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule JMLR 2023
Accelerating Self-Supervised Learning via Efficient Training Strategies WACV 2023
Learning Rate Schedules in the Presence of Distribution Shift ICML 2023
Reverse engineering learned optimizers reveals known and novel mechanisms NIPS 2021
Acceleration via Fractal Learning Rate Schedules ICML 2021
On the Generalization Benefit of Noise in Stochastic Gradient Descent ICML 2020
Marthe: Scheduling the Learning Rate Via Online Hypergradients IJCAI 2020
New Convergence Aspects of Stochastic Gradient Algorithms JMLR 2019
Competence-based Curriculum Learning for Neural Machine Translation NAACL 2019
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares NIPS 2019
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD AISTATS 2018