training dynamics

Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models EMNLP 2024 How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics EMNLP 2024 Identifying Equivalent Training Dynamics NeurIPS 2024 In-Context Learning with Representations: Contextual Generalization of Trained Transformers NeurIPS 2024 Investigation into Training Dynamics of Learned Optimizers (Student Abstract) AAAI 2024 JoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention ICLR 2024 Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection CVPR 2024 Learning, Forgetting, Remembering: Insights From Tracking LLM Memorization During Training EMNLP 2024 Linear attention is (maybe) all you need (to understand Transformer optimization) ICLR 2024 Masked Structural Growth for 2x Faster Language Model Pre-training ICLR 2024 Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes NeurIPS 2024 Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks NeurIPS 2024 Non-asymptotic Convergence of Training Transformers for Next-token Prediction NeurIPS 2024 On Training Data Influence of GPT Models EMNLP 2024 Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization ICLR 2024 Post-Hoc Reversal: Are We Selecting Models Prematurely? NeurIPS 2024 SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics NAACL 2024 Soft ascent-descent as a stable and flexible alternative to flooding NeurIPS 2024 Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning CVPR 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024 Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs ICLR 2024 Tending Towards Stability: Convergence Challenges in Small Language Models EMNLP 2024 The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks NeurIPS 2024 To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets ICLR 2024 Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics NeurIPS 2024