step size

The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance COLT 2022 Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent ICML 2022 Understanding the unstable convergence of gradient descent ICML 2022 Acceleration via Fractal Learning Rate Schedules ICML 2021 Guarantees for Tuning the Step Size using a Learning-to-Learn Approach ICML 2021 The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization ACML 2021 Parabolic Approximation Line Search for DNNs NeurIPS 2020 SAdam: A Variant of Adam for Strongly Convex Functions ICLR 2020 Sparse Coding with Gated Learned ISTA ICLR 2020 Communication trade-offs for Local-SGD with large step size NeurIPS 2019 Optimal Mini-Batch and Step Sizes for SAGA ICML 2019 Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD NeurIPS 2019 Adaptive Three Operator Splitting ICML 2018 Step Size Matters in Deep Learning NeurIPS 2018 Adaptive Batch Size for Safe Policy Gradients NeurIPS 2017 A Study on Trust Region Update Rules in Newton Methods for Large-scale Linear Classification ACML 2017 Barzilai-Borwein Step Size for Stochastic Gradient Descent NeurIPS 2016 Exploration of the (Non-)Asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics JMLR 2016 On the Iteration Complexity of Oblivious First-Order Optimization Algorithms ICML 2016 Approximation Analysis of Stochastic Gradient Langevin Dynamics by using Fokker-Planck Equation and Ito Process ICML 2014 Adaptive Step-Size for Policy Gradient Methods NeurIPS 2013