stochastic gradient descent

1088 papers

Explore in graph

Also known as

SGD ASGD SAGA SGM SGDA PSGD SKGD

Co-occurring keywords

non-convex optimization (546) distributed learning (563) neural network optimization (1293) convergence rate (606) convex optimization (1320) neural network (6616) variance reduction (520) stochastic optimization (1060) convergence analysis (394) differential privacy (1010)

Papers

Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimization IJCAI 2024

Exploring the Inefficiency of Heavy Ball as Momentum Parameter Approaches 1 IJCAI 2024

Scaling Laws in Linear Regression: Compute, Parameters, and Data NIPS 2024

Online Sensitivity Optimization in Differentially Private Learning AAAI 2024

BFCI at AraFinNLP2024: Support Vector Machines for Arabic Financial Text Classification ACL 2024

Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models EMNLP 2024

Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD NIPS 2024

SAMPa: Sharpness-aware Minimization Parallelized NIPS 2024

Communication Efficient Distributed Training with Distributed Lion NIPS 2024

Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement NIPS 2024

A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks JMLR 2024

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems AAAI 2024

SGD vs GD: Rank Deficiency in Linear Networks NIPS 2024

An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness NIPS 2024

Wasserstein Differential Privacy AAAI 2024

The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms NIPS 2024

Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning JMLR 2024

SLowcalSGD : Slow Query Points Improve Local-SGD for Stochastic Convex Optimization NIPS 2024

DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction NIPS 2024

Why Do We Need Weight Decay in Modern Deep Learning? NIPS 2024

Iterate Averaging in the Quest for Best Test Error JMLR 2024

Structured Dynamic Pricing: Optimal Regret in a Global Shrinkage Model JMLR 2024

On the Hyperparameters in Stochastic Gradient Descent with Momentum JMLR 2024

Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods JMLR 2024

Why Warmup the Learning Rate? Underlying Mechanisms and Improvements NIPS 2024