NExtLong: Toward Effective Long-Context Training without Long Documents

Chaochen Gao; Xing W; Zijia Lin; Debing Zhang; Songlin Hu

2025 ICML ICML 2025

NExtLong: Toward Effective Long-Context Training without Long Documents

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Chaochen Gao, Xing W, Zijia Lin, Debing Zhang, Songlin Hu

Topics

Machine Learning > Learning Types > Unsupervised Learning Natural Language Processing > Resources & Methods > Large Language Models Artificial Intelligence > Core AI > Large Language Models

Keywords

long-context training long-range dependency modeling pretraining corpora hard negative distractors meta-chunks negative document extension

Download PDF