long-context modeling

Long-Short Alignment for Effective Long-Context Modeling in LLMs ICML 2025 M-LongDoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework EMNLP 2025 Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention ACL 2025 PeerQA: A Scientific Question Answering Dataset from Peer Reviews NAACL 2025 ProLongVid: A Simple but Strong Baseline for Long-context Video Instruction Tuning EMNLP 2025 R3Mem: Bridging Memory Retention and Retrieval via Reversible Compression ACL 2025 Re3Syn: A Dependency-Based Data Synthesis Framework for Long-Context Post-training ACL 2025 ResFormer: All-Time Reservoir Memory for Long Sequence Classification EMNLP 2025 SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling EMNLP 2025 The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval ACL 2025 Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective COLING 2025 ZigZagKV: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty COLING 2025 Insights into LLM Long-Context Failures: When Transformers Know but Don’t Tell EMNLP 2024 Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA EMNLP 2024 Length Generalization of Causal Transformers without Position Encoding ACL 2024 Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models ACL 2024 LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding NeurIPS 2024 Making Long-Context Language Models Better Multi-Hop Reasoners ACL 2024 MMLONGBENCH-DOC: Benchmarking Long-context Document Understanding with Visualizations NeurIPS 2024 NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time ACL 2024 General-purpose, long-context autoregressive modeling with Perceiver AR ICML 2022 Simple Local Attentions Remain Competitive for Long-Context Tasks NAACL 2022 Luna: Linear Unified Nested Attention NeurIPS 2021