Language Modeling Across Script Variants: The Case of Japanese
Abstract
AbstractWe investigate the effect of script variants on language modeling in Japanese, which uniquely combines logographic (kanji) and phonetic (kana) writing systems. Motivated by educational curricula in which learners are gradually exposed to increasing numbers of logographic characters, we simulate this staged exposure during training to study its effect on model behavior. We construct multiple versions of a corpus with controlled script transformations, including fully transliterated kana text and intermediate variants corresponding to curriculum-based kanji levels, and train language models under different schedules that vary the timing and progression of script exposure during pre-training. In addition, we propose a cross-script evaluation framework using multiple versions of a minimal-pair benchmark with systematically varied kanji usage to assess the robustness of language models to script variation. Our results show that training on a mixture of two extreme script variants improves robustness across script conditions, whereas curriculum-based progression does not consistently improve performance.