conftrace_

positional encoding

233 papers

Explore in graph

Also known as

PE

Co-occurring keywords · papers mentioning both

attention mechanism (38) transformer architecture (32) graph neural network (27) large language model (17) graph transformer (13) vision transformer (12) context window (11) language model (10) self-attention mechanism (9) length extrapolation (8)

Papers

How do Transformers Perform In-Context Autoregressive Learning ? ICML 2024 How Much Context Does My Attention-Based ASR System Need? Interspeech 2024 Improved Implicit Neural Representation with Fourier Reparameterized Training CVPR 2024 Improved MLP Point Cloud Processing with High-Dimensional Positional Encoding AAAI 2024 Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding EMNLP 2024 LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models NAACL 2024 Long-Context Language Modeling with Parallel Context Encoding ACL 2024 LongHeads: Multi-Head Attention is Secretly a Long Context Processor EMNLP 2024 Making Vision Transformers Truly Shift-Equivariant CVPR 2024 Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective ICML 2024 MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo ICLR 2024 Neurons in Large Language Models: Dead, N-gram, Positional ACL 2024 On the Stability of Expressive Positional Encodings for Graphs ICLR 2024 Polyhedral Complex Derivation from Piecewise Trilinear Networks NeurIPS 2024 PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training ICLR 2024 Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure NeurIPS 2024 Seamless Human Motion Composition with Blended Positional Encodings CVPR 2024 Solving Masked Jigsaw Puzzles with Diffusion Vision Transformers CVPR 2024 Spatio-Spectral Graph Neural Networks NeurIPS 2024 Spline-based Transformers ECCV 2024 Subgraphormer: Unifying Subgraph GNNs and Graph Transformers via Graph Products ICML 2024 Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation ICML 2024 Understanding How Positional Encodings Work in Transformer Model COLING 2024 Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling NeurIPS 2024 What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding ICML 2024