sequence length

Beyond Fixed-Length Calibration for Post-Training Compression of LLMs EMNLP 2025 Extending LLM Context Window with Adaptive Grouped Positional Encoding: A Training-Free Method ACL 2025 How to Train Long-Context Language Models (Effectively) ACL 2025 MrT5: Dynamic Token Merging for Efficient Byte-level Language Models ICLR 2025 Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference ACL 2025 Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding ACL 2025 Understanding and Improving Length Generalization in Recurrent Models ICML 2025 What is the Best Sequence Length for BabyLM? EMNLP 2025 Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum NeurIPS 2024 Extending Input Contexts of Language Models through Training on Segmented Sequences NAACL 2024 How Smooth Is Attention? ICML 2024 In-Context Learning Dynamics with Random Binary Sequences ICLR 2024 Memorization Capacity of Multi-Head Attention in Transformers ICLR 2024 Parallelizing Model-based Reinforcement Learning Over the Sequence Length NeurIPS 2024 Position: On the Possibilities of AI-Generated Text Detection ICML 2024 Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention ICML 2024 Hyena Hierarchy: Towards Larger Convolutional Language Models ICML 2023 Improving Joint Speech-Text Representations Without Alignment Interspeech 2023 On the Expressive Flexibility of Self-Attention Matrices AAAI 2023 Scaling Law for Document Neural Machine Translation EMNLP 2023 Treeformer: Dense Gradient Trees for Efficient Attention Computation ICLR 2023 Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation NAACL 2022 Multi Resolution Analysis (MRA) for Approximate Self-Attention ICML 2022 The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NeurIPS 2022 Nyströmformer: A Nyström-based Algorithm for Approximating Self-Attention AAAI 2021