conftrace_

linear attention

52 papers

Explore in graph

Co-occurring keywords

language modeling (660) transformer architecture (1560) efficient computing (781) vision transformer (1106) state space model (390) attention mechanism (4001) language model (4599) image classification (1944) sequence modeling (288) long sequence (30)

Papers

QT-ViT: Improving Linear Attention in ViT with Quadratic Taylor Expansion NIPS 2024
Scaling Laws for Linear Complexity Language Models EMNLP 2024
Birdie: Advancing State Space Language Modeling with Dynamic Mixtures of Training Objectives EMNLP 2024
Small-E: Small Language Model with Linear Attention for Efficient Speech Synthesis INTERSPEECH 2024
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers AISTATS 2024
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map NIPS 2024
Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond NIPS 2024
LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory IJCAI 2024
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization NIPS 2024
Bridging the Divide: Reconsidering Softmax and Linear Attention NIPS 2024
Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers NIPS 2024
Gated Slot Attention for Efficient Linear-Time Sequence Modeling NIPS 2024
Span-Selective Linear Attention Transformers for Effective and Robust Schema-Guided Dialogue State Tracking ACL 2023
FLatten Transformer: Vision Transformer using Focused Linear Attention ICCV 2023
RWKV: Reinventing RNNs for the Transformer Era EMNLP 2023
Flowformer: Linearizing Transformers with Conservation Flows ICML 2022
Meta-Learning Fast Weight Language Models EMNLP 2022
Linearizing Transformer with Key-Value Memory EMNLP 2022
Resource-Efficient Hybrid X-Formers for Vision WACV 2022
Transformer Quality in Linear Time ICML 2022
Modeling Context With Linear Attention for Scalable Document-Level Translation EMNLP 2022
Improving Transformers with Probabilistic Attention Keys ICML 2022
LinearSpeech: Parallel Text-to-Speech with Linear Complexity INTERSPEECH 2021
Sub-Linear Memory: How to Make Performers SLiM NIPS 2021
Linear Transformers Are Secretly Fast Weight Programmers ICML 2021