stochastic gradient descent

The Benefits of Implicit Regularization from SGD in Least Squares Problems NeurIPS 2021 The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers ICLR 2021 The ensmallen library for flexible numerical optimization JMLR 2021 The Heavy-Tail Phenomenon in SGD ICML 2021 The Implicit Bias of Minima Stability: A View from Function Space NeurIPS 2021 THOR, Trace-based Hardware-driven Layer-Oriented Natural Gradient Descent Computation AAAI 2021 Toward Better Generalization Bounds with Locally Elastic Stability ICML 2021 Towards Tight Communication Lower Bounds for Distributed Optimisation NeurIPS 2021 Towards Understanding Learning in Neural Networks with Linear Teachers ICML 2021 Towards Understanding Why Lookahead Generalizes Better Than SGD and Beyond NeurIPS 2021 Usable Information and Evolution of Optimal Representations During Training ICLR 2021 Variance Reduced Training with Stratified Sampling for Forecasting Models ICML 2021 vqSGD: Vector Quantized Stochastic Gradient Descent AISTATS 2021 When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC ICML 2021 When Are Solutions Connected in Deep Networks? NeurIPS 2021 Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks NeurIPS 2021 XOR-SGD: provable convex stochastic optimization for decision-making under uncertainty UAI 2021 A Bayesian Perspective on Training Speed and Model Selection NeurIPS 2020 Accelerating SGD with momentum for over-parameterized learning ICLR 2020 Accelerating Stratified Sampling SGD by Reconstructing Strata IJCAI 2020 A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning JMLR 2020 AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients NeurIPS 2020 AdaGrad stepsizes: Sharp convergence over nonconvex landscapes JMLR 2020 Adaptive Gradient Quantization for Data-Parallel SGD NeurIPS 2020 Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizes NeurIPS 2020