transformer training

Residual Koopman Spectral Profiling for Predicting and Preventing Transformer Training Instability UAI 2026 Taming Transformer Without Using Learning Rate Warmup ICLR 2025 Global Convergence in Training Large-Scale Transformers NeurIPS 2024 Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024 Understanding and Minimising Outlier Features in Transformer Training NeurIPS 2024 Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers NeurIPS 2024 Dynamic Stashing Quantization for Efficient Transformer Training EMNLP 2023 H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training NeurIPS 2023 Learning to Grow Pretrained Models for Efficient Transformer Training ICLR 2023 Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks NeurIPS 2023 Tempo: Accelerating Transformer-Based Model Training through Memory Footprint Reduction NeurIPS 2022 Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent EMNLP 2021 Speeding Up Transformer Training By Using Dataset Subsampling - An Exploratory Analysis EMNLP 2021 Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change ACL 2020 Understanding the Difficulty of Training Transformers EMNLP 2020 Making Asynchronous Stochastic Gradient Descent Work for Transformers EMNLP 2019