conftrace_

learning rate

192 papers

Explore in graph

Also known as

LR

Co-occurring keywords

stochastic gradient descent (1091) neural network optimization (1293) gradient descent (1144) convergence rate (607) convergence analysis (395) stochastic optimization (1060) online learning (1777) kernel methods (1097) neural network (6616) regret bound (1926)

Papers

Universal Rates for Regression: Separations between Cut-Off and Absolute Loss COLT 2024
Stepping on the Edge: Curvature Aware Learning Rate Tuners NIPS 2024
On the Hyperparameters in Stochastic Gradient Descent with Momentum JMLR 2024
Adaptive Learning Rate for Follow-the-Regularized-Leader: Competitive Analysis and Best-of-Both-Worlds COLT 2024
Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms NIPS 2024
Where Do Large Learning Rates Lead Us? NIPS 2024
Exact Mean Square Linear Stability Analysis for SGD COLT 2024
ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate NIPS 2024
On the Unstable Convergence Regime of Gradient Descent AAAI 2024
Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate NIPS 2024
Mechanic: A Learning Rate Tuner NIPS 2023
Generalized Polyak Step Size for First Order Optimization with Momentum ICML 2023
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent ICML 2023
Learning-Rate-Free Learning by D-Adaptation ICML 2023
Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective NIPS 2023
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training NIPS 2023
On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm ICML 2023
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width NIPS 2023
On the Dynamics Under the Unhinged Loss and Beyond JMLR 2023
Maximal Initial Learning Rates in Deep ReLU Networks ICML 2023
Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates ICML 2023
How to Scale Your EMA NIPS 2023
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization NIPS 2023
Special Properties of Gradient Descent with Large Learning Rates ICML 2023
Learning threshold neurons via edge of stability NIPS 2023