stochastic gradient descent

Asynchronous Decentralized SGD with Quantized and Local Updates NeurIPS 2021 Asynchronous Stochastic Gradient Descent for Extreme-Scale Recommender Systems AAAI 2021 Automatic variational inference with cascading flows ICML 2021 Batch Normalization Orthogonalizes Representations in Deep Random Networks NeurIPS 2021 Benign Overfitting of Constant-Stepsize SGD for Linear Regression COLT 2021 Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification ICLR 2021 Byzantine-Resilient High-Dimensional SGD with Local Iterations on Heterogeneous Data ICML 2021 Byzantine-Resilient Non-Convex Stochastic Gradient Descent ICLR 2021 CADA: Communication-Adaptive Distributed Adam AISTATS 2021 Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization ICML 2021 Closing the Gap: Tighter Analysis of Alternating Stochastic Gradient Methods for Bilevel Problems NeurIPS 2021 Communication-efficient SGD: From Local SGD to One-Shot Averaging NeurIPS 2021 Communication Efficient SGD via Gradient Sampling With Bayes Prior CVPR 2021 Continuized Accelerations of Deterministic and Stochastic Gradient Descents, and of Gossip Algorithms NeurIPS 2021 Convergence rates and approximation results for SGD and its continuous-time counterpart COLT 2021 Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance NeurIPS 2021 Cooperative SGD: A Unified Framework for the Design and Analysis of Local-Update SGD Algorithms JMLR 2021 Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates AISTATS 2021 CWY Parametrization: a Solution for Parallelized Optimization of Orthogonal and Stiefel Matrices AISTATS 2021 Debiasing a First-order Heuristic for Approximate Bi-level Optimization ICML 2021 Deep Frequency Principle Towards Understanding Why Deeper Learning Is Faster AAAI 2021 Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGD NeurIPS 2021 Differentially Private and Communication Efficient Collaborative Learning AAAI 2021 Differentially Private Stochastic Coordinate Descent AAAI 2021 Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate ICLR 2021