Stochastic Methods

Assessing Generalization of SGD via Disagreement ICLR 2022 Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent NeurIPS 2022 Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay NeurIPS 2022 Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization NeurIPS 2022 Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit NeurIPS 2022 Hindsight Network Credit Assignment: Efficient Credit Assignment in Networks of Discrete Stochastic Units AAAI 2022 Implicit Gradient Alignment in Distributed and Federated Learning AAAI 2022 Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence ICML 2022 Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization NeurIPS 2022 On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NeurIPS 2022 On the Theoretical Properties of Noise Correlation in Stochastic Optimization NeurIPS 2022 The alignment property of SGD noise and how it helps select flat minima: A stability analysis NeurIPS 2022 Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NeurIPS 2022 Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022 What Happens after SGD Reaches Zero Loss? --A Mathematical Framework ICLR 2022 AC-GC: Lossy Activation Compression with Guaranteed Convergence NeurIPS 2021 ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization AAAI 2021 A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima ICLR 2021 Communication Efficient SGD via Gradient Sampling With Bayes Prior CVPR 2021 Conditional Poisson Stochastic Beams EMNLP 2021 Convergence of adaptive algorithms for constrained weakly convex optimization NeurIPS 2021 Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGD NeurIPS 2021 Deterministic Mini-batch Sequencing for Training Deep Neural Networks AAAI 2021 Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent NeurIPS 2021 Efficient and Accurate Gradients for Neural SDEs NeurIPS 2021