stochastic gradient descent

AdaNorm: Adaptive Gradient Norm Correction Based Optimizer for CNNs WACV 2023 Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction AAAI 2023 Adaptive Optimization in the ∞-Width Limit ICLR 2023 Adaptive Riemannian stochastic gradient descent and reparameterization for Gaussian mixture model fitting ACML 2023 Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction NeurIPS 2023 AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix NeurIPS 2023 A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates JMLR 2023 A Guide Through the Zoo of Biased SGD NeurIPS 2023 Aiming towards the minimizers: fast convergence of SGD for overparametrized problems NeurIPS 2023 Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions ICML 2023 Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares ALT 2023 Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics JMLR 2023 Alternating Projected SGD for Equality-constrained Bilevel Optimization AISTATS 2023 A Mathematical Model for Curriculum Learning for Parities ICML 2023 A new characterization of the edge of stability based on a sharpness measure aware of batch gradient distribution ICLR 2023 A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces AISTATS 2023 An SDE for Modeling SAM: Theory and Insights ICML 2023 Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent NeurIPS 2023 ASkewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks AISTATS 2023 A Smooth Binary Mechanism for Efficient Private Continual Observation NeurIPS 2023 Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees JMLR 2023 A Theoretical Analysis of the Learning Dynamics under Class Imbalance ICML 2023 A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity ICLR 2023 A Unified Fast Gradient Clipping Framework for DP-SGD NeurIPS 2023 Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger NeurIPS 2023