conftrace_

softmax attention

10 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) vision transformer (1106) in-context learning (1405) linear attention (52) linear complexity (62) gradient descent (1144) computational complexity (399) multi-head attention (172) reward mechanism (10) function lipschitzness (1)

Papers

Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling ACL 2026
Cumulant Attention in Vision Transformers (Student Abstract) AAAI 2026
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map NIPS 2024
Bridging the Divide: Reconsidering Softmax and Linear Attention NIPS 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context NIPS 2024
A Theoretical Understanding of Self-Correction through In-context Alignment NIPS 2024
In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness NIPS 2024
Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization NIPS 2024
BiViT: Extremely Compressed Binary Vision Transformers ICCV 2023
Linear Complexity Randomized Self-attention Mechanism ICML 2022