conftrace_

rotary position embedding

32 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) large language model (13587) position encoding (54) positional encoding (182) context length (48) context window (73) length extrapolation (31) video understanding (1658) long context (121) context window extension (7)

Papers

Resonance RoPE: Improving Context Length Generalization of Large Language Models ACL 2024
E2-LLM: Efficient and Extreme Length Extension of Large Language Models ACL 2024
On the token distance modeling ability of higher RoPE attention dimension EMNLP 2024
What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information NIPS 2024
CCIIPLab at SIGHAN-2024 dimABSA Task: Contrastive Learning-Enhanced Span-based Framework for Chinese Dimensional Aspect-Based Sentiment Analysis ACL 2024
Mixture of In-Context Experts Enhance LLMs' Long Context Awareness NIPS 2024
Resource-Efficient Hybrid X-Formers for Vision WACV 2022