Optimization

SAM operates far from home: eigenvalue regularization as a dynamical phenomenon ICML 2023 Scaling Law for Document Neural Machine Translation EMNLP 2023 Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation ICLR 2023 Search-Map-Search: A Frame Selection Paradigm for Action Recognition CVPR 2023 Self-Supervised Primal-Dual Learning for Constrained Optimization AAAI 2023 Sequence Parallelism: Long Sequence Training from System Perspective ACL 2023 (S)GD over Diagonal Linear Networks: Implicit bias, Large Stepsizes and Edge of Stability NeurIPS 2023 SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance ICML 2023 SGD with Large Step Sizes Learns Sparse Features ICML 2023 Sharpness-Aware Minimization Leads to Low-Rank Features NeurIPS 2023 Simplex Random Features ICML 2023 Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning ICML 2023 Sliced Optimal Partial Transport CVPR 2023 Small batch deep reinforcement learning NeurIPS 2023 SOL: Sampling-based Optimal Linear bounding of arbitrary scalar functions NeurIPS 2023 Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective CVPR 2023 SP2 : A Second Order Stochastic Polyak Method ICLR 2023 Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation EMNLP 2023 Spider GAN: Leveraging Friendly Neighbors To Accelerate GAN Training CVPR 2023 spred: Solving L1 Penalty with SGD ICML 2023 Stabilizing Transformer Training by Preventing Attention Entropy Collapse ICML 2023 Stable Nonconvex-Nonconcave Training via Linear Interpolation NeurIPS 2023 Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks ICML 2023 Symbolic Discovery of Optimization Algorithms NeurIPS 2023 TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches NSDI 2023