Neural Network Optimization

SGD vs GD: Rank Deficiency in Linear Networks NeurIPS 2024 ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models EMNLP 2024 Sharpened Lazy Incremental Quasi-Newton Method AISTATS 2024 Sharpness-Aware Minimization and the Edge of Stability JMLR 2024 Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning ICLR 2024 Sheared Backpropagation for Fine-tuning Foundation Models CVPR 2024 Shears: Unstructured Sparsity with Neural Low-rank Adapter Search NAACL 2024 Should I try multiple optimizers when fine-tuning a pre-trained Transformer for NLP tasks? Should I tune their hyperparameters? EACL 2024 SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning ACL 2024 Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network ICML 2024 Simplicity Bias via Global Convergence of Sharpness Minimization ICML 2024 Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation ICLR 2024 Slicing Vision Transformer for Flexible Inference NeurIPS 2024 Sliding Down the Stairs: How Correlated Latent Variables Accelerate Learning with Neural Networks ICML 2024 Small-scale proxies for large-scale Transformer training instabilities ICLR 2024 Some Fundamental Aspects about Lipschitz Continuity of Neural Networks ICLR 2024 Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training ICLR 2024 SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers EMNLP 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024 Sparser, Better, Deeper, Stronger: Improving Static Sparse Training with Exact Orthogonal Initialization ICML 2024 Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention AAAI 2024 Speaker Personalization for Automatic Speech Recognition using Weight-Decomposed Low-Rank Adaptation Interspeech 2024 Spectrum Extraction and Clipping for Implicitly Linear Layers AISTATS 2024 Speculative Contrastive Decoding ACL 2024 Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism ACL 2024