conftrace_

learning rate decay

10 papers

Explore in graph

Co-occurring keywords · papers mentioning both

stochastic gradient descent (5) language model (2) convergence analysis (2) polyak-ruppert averaging (1) non-asymptotic analysis (1) learning rate scheduling (1) generalization (1) learning rate (1) convergence rate (1) convolutional neural network (1)

Papers

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks ICML 2025 When, Where and Why to Average Weights? ICML 2025 Resolving Discrepancies in Compute-Optimal Scaling of Language Models NeurIPS 2024 Understanding the Generalization Benefits of Late Learning Rate Decay AISTATS 2024 No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models NeurIPS 2023 On Learning Rates and Schrödinger Operators JMLR 2023 Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression ICML 2022 Phase-Wise Parameter Aggregation for Improving SGD Optimization WACV 2021 Stagewise Training Accelerates Convergence of Testing Error Over SGD NeurIPS 2019 Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning NeurIPS 2011