conftrace_

length extrapolation

36 papers

Explore in graph

Co-occurring keywords · papers mentioning both

attention mechanism (11) large language model (10) positional encoding (8) language model (7) transformer architecture (7) context window (6) rotary position embedding (5) context length (5) position encoding (3) long-context modeling (3)

Papers

HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position ACL 2024 Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding EMNLP 2024 Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs NeurIPS 2024 On the token distance modeling ability of higher RoPE attention dimension EMNLP 2024 A Length-Extrapolatable Transformer ACL 2023 Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis ACL 2023 Empirical Investigation of Neural Symbolic Reasoning Strategies EACL 2023 LISTER: Neighbor Decoding for Length-Insensitive Scene Text Recognition ICCV 2023 Transformer Working Memory Enables Regular Language Reasoning And Natural Language Length Extrapolation EMNLP 2023 KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation NeurIPS 2022 The EOS Decision and Length Extrapolation EMNLP 2020