stochastic gradient descent

Towards Efficient and Scalable Sharpness-Aware Minimization CVPR 2022 Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent ICML 2022 Towards understanding how momentum improves generalization in deep learning ICML 2022 Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NeurIPS 2022 Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022 Understanding the unstable convergence of gradient descent ICML 2022 Vanishing Curvature in Randomly Initialized Deep ReLU Networks AISTATS 2022 Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization JMLR 2022 What Happens after SGD Reaches Zero Loss? --A Mathematical Framework ICLR 2022 What You See is What You Get: Principled Deep Learning via Distributional Generalization NeurIPS 2022 Accelerating Distributed SGD for Linear Regression using Iterative Pre-Conditioning L4DC 2021 Accelerating Gossip SGD with Periodic Global Averaging ICML 2021 AC-GC: Lossy Activation Compression with Guaranteed Convergence NeurIPS 2021 ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization AAAI 2021 Adaptive Knowledge Driven Regularization for Deep Neural Networks AAAI 2021 Adaptive Proximal Gradient Methods for Structured Neural Networks NeurIPS 2021 A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization AAAI 2021 A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima ICLR 2021 A Faster Decentralized Algorithm for Nonconvex Minimax Problems NeurIPS 2021 A Gradient Based Strategy for Hamiltonian Monte Carlo Hyperparameter Optimization ICML 2021 Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball COLT 2021 An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias NeurIPS 2021 An Efficient Algorithm for Deep Stochastic Contextual Bandits AAAI 2021 A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance ICML 2021 Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections ICML 2021