Curse of High Dimensionality Issue in Transformer for Long Context Modeling

Shuhai Zhang; Zeng You; Yaofo Chen; Zhiquan Wen; Qianyue Wang; Zhijie Qiu; Yuanqing Li; Mingkui Tan

2025 ICML ICML 2025

Curse of High Dimensionality Issue in Transformer for Long Context Modeling

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Shuhai Zhang, Zeng You, Yaofo Chen, Zhiquan Wen, Qianyue Wang, Zhijie Qiu, Yuanqing Li, Mingkui Tan

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Natural Language Processing > Generation > Language Modeling Artificial Intelligence > Core AI > Large Language Models

Keywords

transformer self attention long-context modeling dynamic group attention redundancy group coding

Download PDF