long sequence

Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models ACL 2026 Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models ACL 2025 Iterative Sparse Attention for Long-sequence Recommendation AAAI 2025 ReMamba: Equip Mamba with Effective Long-Sequence Modeling EMNLP 2025 AutoChunk: Automated Activation Chunk for Memory-Efficient Deep Learning Inference ICLR 2024 Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers ACL 2024 CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling EMNLP 2024 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length NeurIPS 2024 Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors ICLR 2024 RingAttention with Blockwise Transformers for Near-Infinite Context ICLR 2024 BASS: Block-wise Adaptation for Speech Summarization Interspeech 2023 Blockwise Parallel Transformers for Large Context Models NeurIPS 2023 ChordMixer: A Scalable Neural Attention Model for Sequences with Different Length ICLR 2023 Contrastive Disentangled Learning for Memory-Augmented Transformer Interspeech 2023 Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention NeurIPS 2023 Investigating Efficiently Extending Transformers for Long Input Summarization EMNLP 2023 mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences EMNLP 2023 Parallel Context Windows for Large Language Models ACL 2023 Prefix Propagation: Parameter-Efficient Tuning for Long Sequences ACL 2023 Recasting Self-Attention with Holographic Reduced Representations ICML 2023 Sequence Parallelism: Long Sequence Training from System Perspective ACL 2023 The NLP Task Effectiveness of Long-Range Transformers EACL 2023 Flowformer: Linearizing Transformers with Conservation Flows ICML 2022 FNet: Mixing Tokens with Fourier Transforms NAACL 2022 Improving Transformers with Probabilistic Attention Keys ICML 2022