adaptive learning rate

Momentum Improves Normalized SGD ICML 2020 On the Variance of the Adaptive Learning Rate and Beyond ICLR 2020 Training Neural Networks for and by Interpolation ICML 2020 Understanding the Difficulty of Training Transformers EMNLP 2020 AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods ICLR 2019 A Sufficient Condition for Convergences of Adam and RMSProp CVPR 2019 Cogra: Concept-Drift-Aware Stochastic Gradient Descent for Time-Series Forecasting AAAI 2019 Efficient Learning of Restricted Boltzmann Machines Using Covariance Estimates ACML 2019 Finite-Time Performance Bounds and Adaptive Learning Rate Selection for Two Time-Scale Reinforcement Learning NeurIPS 2019 HyperAdam: A Learnable Task-Adaptive Adam for Network Training AAAI 2019 Lipschitz Adaptivity with Multiple Learning Rates in Online Learning COLT 2019 Lookahead Optimizer: k steps forward, 1 step back NeurIPS 2019 Momentum-Based Variance Reduction in Non-Convex SGD NeurIPS 2019 Nostalgic Adam: Weighting More of the Past Gradients When Designing the Adaptive Learning Rate IJCAI 2019 Adafactor: Adaptive Learning Rates with Sublinear Memory Cost ICML 2018 On the Convergence of Adam and Beyond ICLR 2018 SADAGRAD: Strongly Adaptive Stochastic Gradient Methods ICML 2018 Adaptive Learning Rate via Covariance Matrix Based Preconditioning for Deep Neural Networks IJCAI 2017 A survey of Algorithms and Analysis for Adaptive Online Learning JMLR 2017 Automated Inference with Adaptive Batches AISTATS 2017 Scalable Adaptive Stochastic Optimization Using Random Projections NeurIPS 2016 Equilibrated adaptive learning rates for non-convex optimization NeurIPS 2015 Adaptive Hedge NeurIPS 2011 Spatially-Adaptive Learning Rates for Online Incremental SLAM RSS 2007