Neural Network Optimization

SPT: Learning to Selectively Insert Prompts for Better Prompt Tuning EMNLP 2023 Stability and Generalization of lp-Regularized Stochastic Learning for GCN IJCAI 2023 Stability-Based Generalization Analysis of the Asynchronous Decentralized SGD AAAI 2023 Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints NeurIPS 2023 Stable Nonconvex-Nonconcave Training via Linear Interpolation NeurIPS 2023 STay-ON-the-Ridge: Guaranteed Convergence to Local Minimax Equilibrium in Nonconvex-Nonconcave Games COLT 2023 StEik: Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation NeurIPS 2023 Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks NeurIPS 2023 Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network ICML 2023 Strong Lottery Ticket Hypothesis with ε–perturbation AISTATS 2023 Structured Pruning for Efficient Generative Pre-trained Language Models ACL 2023 Super-Resolution Neural Operator CVPR 2023 Symbolic Discovery of Optimization Algorithms NeurIPS 2023 Task-aware Adaptive Learning for Cross-domain Few-shot Learning ICCV 2023 Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training NeurIPS 2023 Test Time Adaptation With Regularized Loss for Weakly Supervised Salient Object Detection CVPR 2023 The Crucial Role of Normalization in Sharpness-Aware Minimization NeurIPS 2023 The Dormant Neuron Phenomenon in Deep Reinforcement Learning ICML 2023 The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks NeurIPS 2023 The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima JMLR 2023 The Geometry of Neural Nets' Parameter Spaces Under Reparametrization NeurIPS 2023 The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent ICML 2023 The Implicit Regularization of Momentum Gradient Descent in Overparametrized Models AAAI 2023 The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing ICML 2023 The Transient Nature of Emergent In-Context Learning in Transformers NeurIPS 2023