conftrace_

positional encoding

182 papers

Explore in graph

Also known as

PE

Co-occurring keywords

attention mechanism (4001) transformer architecture (1560) graph neural network (3962) large language model (13587) context window (73) neural network (6616) language model (4599) self-attention mechanism (352) vision transformer (1106) length extrapolation (31)

Papers

Understanding How Positional Encodings Work in Transformer Model COLING 2024
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding NIPS 2024
GIN-SD: Source Detection in Graphs with Incomplete Nodes via Positional Encoding and Attentive Fusion AAAI 2024
LongHeads: Multi-Head Attention is Secretly a Long Context Processor EMNLP 2024
Improved MLP Point Cloud Processing with High-Dimensional Positional Encoding AAAI 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models IJCAI 2024
Long-Context Language Modeling with Parallel Context Encoding ACL 2024
Exploring Context Window of Large Language Models via Decomposed Positional Vectors NIPS 2024
DAPE: Data-Adaptive Positional Encoding for Length Extrapolation NIPS 2024
Polyhedral Complex Derivation from Piecewise Trilinear Networks NIPS 2024
Advancing Spiking Neural Networks for Sequential Modeling with Central Pattern Generators NIPS 2024
Spatio-Spectral Graph Neural Networks NIPS 2024
Base of RoPE Bounds Context Length NIPS 2024
Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling NIPS 2024
An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding NIPS 2024
Seamless Human Motion Composition with Blended Positional Encodings CVPR 2024
Improved Implicit Neural Representation with Fourier Reparameterized Training CVPR 2024
How Much Context Does My Attention-Based ASR System Need? INTERSPEECH 2024
Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure NIPS 2024
Activating Self-Attention for Multi-Scene Absolute Pose Regression NIPS 2024
A Morphology-Based Investigation of Positional Encodings EMNLP 2024
Algebraic Positional Encodings NIPS 2024
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models NAACL 2024
Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding EMNLP 2024
Making Vision Transformers Truly Shift-Equivariant CVPR 2024