context length

PEPE: Long-context Extension for Large Language Models via Periodic Extrapolation Positional Encodings EMNLP 2025 ReAttention: Training-Free Infinite Context with Finite Attention Scope ICLR 2025 Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling ICLR 2025 Selective Attention Improves Transformer ICLR 2025 Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts EMNLP 2025 UniICL: An Efficient ICL Framework Unifying Compression, Selection, and Generation ACL 2025 What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs ACL 2025 When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models EMNLP 2025 Why Does the Effective Context Length of LLMs Fall Short? ICLR 2025 A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts ICML 2024 An Empirical Study on the Characteristics of Bias upon Context Length Variation for Bangla ACL 2024 BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack NeurIPS 2024 Balancing Context Length and Mixing Times for Reinforcement Learning at Scale NeurIPS 2024 Base of RoPE Bounds Context Length NeurIPS 2024 Beyond Attention: Breaking the Limits of Transformer Context Length with Recurrent Memory AAAI 2024 Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models IJCAI 2024 CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending ACL 2024 Data Engineering for Scaling Language Models to 128K Context ICML 2024 E2-LLM: Efficient and Extreme Length Extension of Large Language Models ACL 2024 HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position ACL 2024 Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts ACL 2024 KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization NeurIPS 2024 LongAlign: A Recipe for Long Context Alignment of Large Language Models EMNLP 2024 Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning EMNLP 2024 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length NeurIPS 2024