Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences

Zicheng Liu; Siyuan Li; Li Wang; Zedong Wang; Yunfan Liu; Stan Z. Li

2024 ICML ICML 2024

Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Zicheng Liu, Siyuan Li, Li Wang, Zedong Wang, Yunfan Liu, Stan Z. Li

Topics

Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Transformers Deep Learning > Models > Large Language Models

Keywords

divide and conquer state space model linear attention long-range arena hardware-efficient short-long convolutions

Download PDF