LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning

Hongye Jin; Xiaotian Han; Jingfeng Yang; Zhimeng Jiang; Zirui Liu; Chia-Yuan Chang; Huiyuan Chen; Xia Hu

2024 ICML ICML 2024

LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, Xia Hu

Topics

Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Transformers Artificial Intelligence > Core AI > Large Language Models

Keywords

context window grouped attention bi-level attention neighbor attention inference self-extend

Download PDF