stochastic gradient descent

The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions ICML 2025 The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning JMLR 2025 The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations ICML 2025 The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD ICLR 2025 The Optimization Landscape of SGD Across the Feature Learning Strength ICLR 2025 Theoretical Convergence Guarantees for Variational Autoencoders AISTATS 2025 To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions ICLR 2025 Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory CVPR 2025 Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs AISTATS 2025 Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets NAACL 2025 A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems AAAI 2024 A Comparison of Continuous-Time Approximations to Stochastic Gradient Descent JMLR 2024 Adaptive Federated Learning with Auto-Tuned Clients ICLR 2024 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 A General Algorithm for Solving Rank-one Matrix Sensing AISTATS 2024 A large deviations perspective on policy gradient algorithms L4DC 2024 Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods JMLR 2024 An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness NeurIPS 2024 A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness ICML 2024 A Near-optimal Algorithm for Learning Margin Halfspaces with Massart Noise NeurIPS 2024 A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks JMLR 2024 Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization ICLR 2024 AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms AISTATS 2024 AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop UAI 2024 Banded Square Root Matrix Factorization for Differentially Private Model Training NeurIPS 2024