optimization theory

Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent AAAI 2025 Error Analysis of Spherically Constrained Least Squares Reformulation in Solving the Stackelberg Prediction Game NeurIPS 2024 Faster Rates of Differentially Private Stochastic Convex Optimization JMLR 2024 From Activation to Initialization: Scaling Insights for Optimizing Neural Fields CVPR 2024 Global Convergence in Training Large-Scale Transformers NeurIPS 2024 Gradient Guidance for Diffusion Models: An Optimization Perspective NeurIPS 2024 Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? NeurIPS 2024 On the Generalization of Stochastic Gradient Descent with Momentum JMLR 2024 Provable Accelerated Convergence of Nesterov’s Momentum for Deep ReLU Neural Networks ALT 2024 The Price of Adaptivity in Stochastic Convex Optimization COLT 2024 Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent AAAI 2024 Unraveling the Gradient Descent Dynamics of Transformers NeurIPS 2024 Does a sparse ReLU network training problem always admit an optimum ? NeurIPS 2023 Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data NeurIPS 2023 Large deviations rates for stochastic gradient descent with strongly convex functions AISTATS 2023 Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions ICML 2023 Plateau in Monotonic Linear Interpolation --- A "Biased" View of Loss Landscape for Deep Networks ICLR 2023 TD Convergence: An Optimization Perspective NeurIPS 2023 On Margin Maximization in Linear and ReLU Networks NeurIPS 2022 On Numerical Integration in Neural Ordinary Differential Equations ICML 2022 Oracle Complexity in Nonsmooth Nonconvex Optimization JMLR 2022 Power-Law Escape Rate of SGD ICML 2022 Embedding Principle of Loss Landscape of Deep Neural Networks NeurIPS 2021 Exponentially Many Local Minima in Quantum Neural Networks ICML 2021 Hessian Eigenspectra of More Realistic Nonlinear Models NeurIPS 2021