rotary position embedding

TC-LLaVA: Rethinking the Transfer of LLava from Image to Video Understanding with Temporal Considerations AAAI 2025 The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval ACL 2025 Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective COLING 2025 UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generation CVPR 2025 VideoRoPE: What Makes for Good Video Rotary Position Embedding? ICML 2025 VRoPE: Rotary Position Embedding for Video Large Language Models EMNLP 2025 Why Does the Effective Context Length of LLMs Fall Short? ICLR 2025 Base of RoPE Bounds Context Length NeurIPS 2024 CCIIPLab at SIGHAN-2024 dimABSA Task: Contrastive Learning-Enhanced Span-based Framework for Chinese Dimensional Aspect-Based Sentiment Analysis ACL 2024 CLEX: Continuous Length Extrapolation for Large Language Models ICLR 2024 CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending ACL 2024 E2-LLM: Efficient and Extreme Length Extension of Large Language Models ACL 2024 Extending Context Window of Large Language Models from a Distributional Perspective EMNLP 2024 Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use ACL 2024 HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position ACL 2024 LongEmbed: Extending Embedding Models for Long Context Retrieval EMNLP 2024 Mixture of In-Context Experts Enhance LLMs' Long Context Awareness NeurIPS 2024 On the token distance modeling ability of higher RoPE attention dimension EMNLP 2024 PSC: Extending Context Window of Large Language Models via Phase Shift Calibration EMNLP 2024 Resonance RoPE: Improving Context Length Generalization of Large Language Models ACL 2024 Scaling Laws of RoPE-based Extrapolation ICLR 2024 What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information NeurIPS 2024 YaRN: Efficient Context Window Extension of Large Language Models ICLR 2024 Resource-Efficient Hybrid X-Formers for Vision WACV 2022