conftrace_

length extrapolation

31 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) positional encoding (182) large language model (13587) transformer architecture (1560) language model (4599) context window (73) rotary position embedding (32) position encoding (54) transformer model (1996) context length (48)

Papers

SCOPE: Boosting LLM Efficiency with Scoped Position Encoding ACL 2026
Disentangle to Decay: Linear Attention with Trainable Decay Factor COLING 2025
Context-aware Biases for Length Extrapolation EMNLP 2025
A Training-Free Length Extrapolation Approach for LLMs: Greedy Attention Logit Interpolation EMNLP 2025
SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling EMNLP 2025
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation ACL 2025
Hansel: Output Length Controlling Framework for Large Language Models AAAI 2025
Teaching by Failure: Counter-Example–Driven Curricula for Transformer Self-Improvement AACL 2025
LongSafety: Enhance Safety for Long-Context LLMs ACL 2025
DCIS: Efficient Length Extrapolation of LLMs via Divide-and-Conquer Scaling Factor Search EMNLP 2025
Teaching by Failure: Counter-Example–Driven Curricula for Transformer Self-Improvement IJCNLP 2025
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection EMNLP 2025
HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position ACL 2024
Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation NAACL 2024
Extending Context Window of Large Language Models from a Distributional Perspective EMNLP 2024
Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge EMNLP 2024
Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs NIPS 2024
Exploring Context Window of Large Language Models via Decomposed Positional Vectors NIPS 2024
On the token distance modeling ability of higher RoPE attention dimension EMNLP 2024
Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding EMNLP 2024
Exploring Transformer Extrapolation AAAI 2024
E2-LLM: Efficient and Extreme Length Extension of Large Language Models ACL 2024
DAPE: Data-Adaptive Positional Encoding for Length Extrapolation NIPS 2024
Advancing Regular Language Reasoning in Linear Recurrent Neural Networks NAACL 2024
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis ACL 2023