Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Chronos: Chronology-Grounded Continued Pre-training for Language Model Adaptation to Materials Science

AbstractContinued pre-training on domain-specific corpora has become the prevailing approach for adapting pre-trained language models (PLMs) to materials science. However, existing strategies typically treat scientific literature as independent samples, thereby overlooking the cumulative nature of scientific knowledge where novel discoveries systematically build upon prior research. To address this limitation, we propose Chronology-grounded continued pre-training for materials science (Chronos), which explicitly considers the chronological knowledge in terms of data construction and training schemes. Specifically, our framework traces the relevant prior articles by using citation-based information and constructs a chronological curriculum to progressively learn the materials knowledge. The comprehensive experiments demonstrate that Chronos outperforms state-of-the-art (SOTA) baselines on various materials science benchmarks and backbone models, achieving improvements of 4.7% on the Macro-F1 score. In-depth analysis shows that considering the cumulative nature of knowledge enhances both the quality of the corpus and the effectiveness of training schemes.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →