Neural Network Optimization

Speech Prefix-Tuning with RNNT Loss for Improving LLM Predictions Interspeech 2024 SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification ACL 2024 SRF: Enhancing Document-Level Relation Extraction with a Novel Secondary Reasoning Framework EMNLP 2024 ST$_k$: A Scalable Module for Solving Top-k Problems NeurIPS 2024 Stability and Generalization of Asynchronous SGD: Sharper Bounds Beyond Lipschitz and Smoothness NeurIPS 2024 Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms ICML 2024 Stabilizing Backpropagation Through Time to Learn Complex Physics ICLR 2024 Stable Language Model Pre-training by Reducing Embedding Variability EMNLP 2024 StableMask: Refining Causal Masking in Decoder-only Transformer ICML 2024 Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training NeurIPS 2024 Stepping on the Edge: Curvature Aware Learning Rate Tuners NeurIPS 2024 Stochastic Bandits with ReLU Neural Networks ICML 2024 Stochastic Fine-Tuning of Language Models Using Masked Gradients EMNLP 2024 Stochastic Modified Equations and Dynamics of Dropout Algorithm ICLR 2024 Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent JMLR 2024 Stochastic Newton Proximal Extragradient Method NeurIPS 2024 Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators NeurIPS 2024 Straight-Through Meets Sparse Recovery: the Support Exploration Algorithm ICML 2024 Strengthening Layer Interaction via Dynamic Layer Attention IJCAI 2024 Structured Gradient-based Interpretations via Norm-Regularized Adversarial Training CVPR 2024 Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC ICML 2024 Super Consistency of Neural Network Landscapes and Learning Rate Transfer NeurIPS 2024 Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling NeurIPS 2024 SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning ICLR 2024 Symmetric Dot-Product Attention for Efficient Training of BERT Language Models ACL 2024