learning rate

Generalized Polyak Step Size for First Order Optimization with Momentum ICML 2023 How to Scale Your EMA NeurIPS 2023 Improving Multi-Fidelity Optimization With a Recurring Learning Rate for Hyperparameter Tuning WACV 2023 Learning-Rate-Free Learning by D-Adaptation ICML 2023 Learning threshold neurons via edge of stability NeurIPS 2023 Maximal Initial Learning Rates in Deep ReLU Networks ICML 2023 Mechanic: A Learning Rate Tuner NeurIPS 2023 Meta-AdaM: An Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning NeurIPS 2023 Minimax Optimal Kernel Operator Learning via Multilevel Training ICLR 2023 On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm ICML 2023 On the Dynamics Under the Unhinged Loss and Beyond JMLR 2023 Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width NeurIPS 2023 Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability ICLR 2023 Special Properties of Gradient Descent with Large Learning Rates ICML 2023 Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training NeurIPS 2023 The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent ICML 2023 Universal Rates for Multiclass Learning COLT 2023 Why (and When) does Local SGD Generalize Better than SGD? ICLR 2023 A Loss Curvature Perspective on Training Instabilities of Deep Learning Models ICLR 2022 Amortized Proximal Optimization NeurIPS 2022 Better Parameter-Free Stochastic Optimization with ODE Updates for Coin-Betting AAAI 2022 Beyond spectral gap: the role of the topology in decentralized learning NeurIPS 2022 Convergence Rates of Non-Convex Stochastic Gradient Descent Under a Generic Lojasiewicz Condition and Local Smoothness ICML 2022 Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information ICLR 2022 Exploiting Explainable Metrics for Augmented SGD CVPR 2022