conftrace_

learning rate schedule

27 papers

Explore in graph

Also known as

LR SCHEDULE

Co-occurring keywords · papers mentioning both

stochastic gradient descent (8) gradient descent (4) neural network optimization (4) compute optimization (2) pretraining loss (2) weight rewinding (2) stochastic differential equation (2) gradient clipping (2) convex optimization (2) neural network pruning (2)

Papers

The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares NeurIPS 2019 Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD AISTATS 2018