← Keywords

stochastic gradient descent

1088 papers

Explore in graph

Also known as

SGD ASGD SAGA SGM SGDA PSGD SKGD

Co-occurring keywords

non-convex optimization (546) distributed learning (563) neural network optimization (1293) convergence rate (606) convex optimization (1320) neural network (6616) variance reduction (520) stochastic optimization (1060) convergence analysis (394) differential privacy (1010)

Papers

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients NIPS 2020
Quantitative Propagation of Chaos for SGD in Wide Neural Networks NIPS 2020
Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian NIPS 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems NIPS 2020
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks NIPS 2020
Bad Global Minima Exist and SGD Can Reach Them NIPS 2020
PEP: Parameter Ensembling by Perturbation NIPS 2020
The Pitfalls of Simplicity Bias in Neural Networks NIPS 2020
Improving Neural Network Training in Low Dimensional Random Bases NIPS 2020
CSER: Communication-efficient SGD with Error Reset NIPS 2020
Directional Pruning of Deep Neural Networks NIPS 2020
When Do Neural Networks Outperform Kernel Methods? NIPS 2020
Why are Adaptive Methods Good for Attention Models? NIPS 2020
Improved Analysis of Clipping Algorithms for Non-convex Optimization NIPS 2020
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods NIPS 2020
Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling NIPS 2020
Breaking Reversibility Accelerates Langevin Dynamics for Non-Convex Optimization NIPS 2020
A Lyapunov analysis for accelerated gradient methods: from deterministic to stochastic case AISTATS 2020
A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent AISTATS 2020
Student Specialization in Deep Rectified Networks With Finite Width and Input Dimension ICML 2020
Learning Halfspaces with Massart Noise Under Structured Distributions COLT 2020
Efficient Parameter Estimation of Truncated Boolean Product Distributions COLT 2020
Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process COLT 2020
How Good is SGD with Random Shuffling? COLT 2020
Open Problem: Tight Convergence of SGD in Constant Dimension COLT 2020