efficient transformer

Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers NeurIPS 2024 CoLT5: Faster Long-Range Transformers with Conditional Computation EMNLP 2023 Composite Slice Transformer: An Efficient Transformer with Composition of Multi-Scale Multi-Range Attentions ICLR 2023 Diffuser: Efficient Transformers with Multi-Hop Attention Diffusion for Long Sequences AAAI 2023 Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding IJCAI 2023 Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model EMNLP 2023 When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants ACL 2023 LittleBird: Efficient Faster & Longer Transformer for Question Answering EMNLP 2022 Modify Self-Attention via Skeleton Decomposition for Effective Point Cloud Transformer AAAI 2022 Token and Head Adaptive Transformers for Efficient Natural Language Processing COLING 2022 FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention NeurIPS 2021 Long Range Arena : A Benchmark for Efficient Transformers ICLR 2021 Long-Short Transformer: Efficient Transformers for Language and Vision NeurIPS 2021 Scatterbrain: Unifying Sparse and Low-rank Attention NeurIPS 2021