conftrace_

learning rate decay

8 papers

Explore in graph

Co-occurring keywords

stochastic gradient descent (1091) language model (4599) polyak-ruppert averaging (14) non-asymptotic analysis (14) learning rate scheduling (9) learning rate (192) convergence rate (607) convolutional neural network (4226) risk bound (71) stochastic differential equation (181)

Papers

Understanding the Generalization Benefits of Late Learning Rate Decay AISTATS 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models NIPS 2024
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models NIPS 2023
On Learning Rates and Schrödinger Operators JMLR 2023
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression ICML 2022
Phase-Wise Parameter Aggregation for Improving SGD Optimization WACV 2021
Stagewise Training Accelerates Convergence of Testing Error Over SGD NIPS 2019
Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning NIPS 2011