adaptive learning rate

AdaLoss: A Computationally-Efficient and Provably Convergent Adaptive Gradient Method AAAI 2022 Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction ICML 2022 Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum ICML 2022 Adversarially Robust Multi-Armed Bandit Algorithm with Variance-Dependent Regret Bounds COLT 2022 A Stochastic Bundle Method for Interpolation JMLR 2022 Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits ICLR 2022 MetaER-TTE: An Adaptive Meta-learning Model for En Route Travel Time Estimation IJCAI 2022 Meta-Learning for Online Update of Recommender Systems AAAI 2022 MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification COLING 2022 No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models ICLR 2022 Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration JMLR 2022 Adaptive Approximate Policy Iteration AISTATS 2021 A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance ICML 2021 Large Batch Optimization for Deep Learning Using New Complete Layer-Wise Adaptive Rate Scaling AAAI 2021 MetaGrad: Adaptation using Multiple Learning Rates in Online Learning JMLR 2021 MTAdam: Automatic Balancing of Multiple Training Loss Terms EMNLP 2021 Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence AISTATS 2021 STORM+: Fully Adaptive SGD with Recursive Momentum for Nonconvex Optimization NeurIPS 2021 SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients NeurIPS 2021 Three Operator Splitting with Subgradients, Stochastic Gradients, and Adaptive Learning Rates NeurIPS 2021 Adam with Bandit Sampling for Deep Learning NeurIPS 2020 Adaptive Gradient Descent without Descent ICML 2020 AdaScale SGD: A User-Friendly Algorithm for Distributed Training ICML 2020 Better Full-Matrix Regret via Parameter-Free Online Learning NeurIPS 2020 Meta-Learning with Adaptive Hyperparameters NeurIPS 2020