stochastic gradient descent

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021 On ADMM in Deep Learning: Convergence and Saturation-Avoidance JMLR 2021 One-pass Stochastic Gradient Descent in overparametrized two-layer neural networks AISTATS 2021 On Linear Stability of SGD and Input-Smoothness of Neural Networks NeurIPS 2021 Online stochastic gradient descent on non-convex losses from high-dimensional inference JMLR 2021 On Monotonic Linear Interpolation of Neural Network Parameters ICML 2021 On the Convergence of Step Decay Step-Size for Stochastic Optimization NeurIPS 2021 On the Convergence of Stochastic Compositional Gradient Descent Ascent Method IJCAI 2021 On the distributional properties of adaptive gradients UAI 2021 On the Origin of Implicit Regularization in Stochastic Gradient Descent ICLR 2021 On the Power of Differentiable Learning versus PAC and SQ Learning NeurIPS 2021 On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) NeurIPS 2021 Optimal Rates for Random Order Online Optimization NeurIPS 2021 PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization ICML 2021 Phase-Wise Parameter Aggregation for Improving SGD Optimization WACV 2021 Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization ICML 2021 Practical and Private (Deep) Learning Without Sampling or Shuffling ICML 2021 Private Adaptive Gradient Methods for Convex Optimization ICML 2021 Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise ICML 2021 Provably Faster Algorithms for Bilevel Optimization NeurIPS 2021 Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent NeurIPS 2021 Randomized Automatic Differentiation ICLR 2021 Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems NeurIPS 2021 Regularizing Neural Networks via Adversarial Model Perturbation CVPR 2021 Revisiting Model Stitching to Compare Neural Representations NeurIPS 2021