← Keywords

stochastic gradient descent

1088 papers

Explore in graph

Also known as

SGD ASGD SAGA SGM SGDA PSGD SKGD

Co-occurring keywords

non-convex optimization (546) distributed learning (563) neural network optimization (1293) convergence rate (606) convex optimization (1320) neural network (6616) variance reduction (520) stochastic optimization (1060) convergence analysis (394) differential privacy (1010)

Papers

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks AISTATS 2024
The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank AAAI 2024
Stochastic Fine-Tuning of Language Models Using Masked Gradients EMNLP 2024
Fast Forwarding Low-Rank Training EMNLP 2024
A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems AAAI 2024
Scaling Laws in Linear Regression: Compute, Parameters, and Data NIPS 2024
Benign Overfitting of Constant-Stepsize SGD for Linear Regression JMLR 2023
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD JMLR 2023
On Biased Compression for Distributed Learning JMLR 2023
Improved Powered Stochastic Optimization Algorithms for Large-Scale Machine Learning JMLR 2023
HiGrad: Uncertainty Quantification for Online Learning and Stochastic Approximation JMLR 2023
Mixtures of All Trees AISTATS 2023
A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates JMLR 2023
Decentralized Learning: Theoretical Optimality and Practical Improvements JMLR 2023
HeteRSGD: Tackling Heterogeneous Sampling Costs via Optimal Reweighted Stochastic Gradient Descent AISTATS 2023
Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval JMLR 2023
On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size JMLR 2023
Robust Linear Regression for General Feature Distribution AISTATS 2023
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees ICML 2023
Implicit Bias of (Stochastic) Gradient Descent for Rank-1 Linear Neural Network NIPS 2023
“Plus/minus the learning rate”: Easy and Scalable Statistical Inference with SGD AISTATS 2023
Learning Trajectories are Generalization Indicators NIPS 2023
Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity AISTATS 2023
Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond ICML 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods NIPS 2023