Skip to content
conftrace_
2026 EMNLP EMNLP 2026

On-Policy Delta Distillation for Multilingual Math Reasoning

AbstractOn-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexplored. We study OPD and its advanced variant, OnPolicy Delta Distillation (OPD2 ), for mathematical reasoning in English, Korean, and Japanese. OPD2 improves OPD by using the probability gap between a post-trained teacher and its base model as the learning signal. Experiments with Qwen3 show that OPD2 consistently outperforms the original OPD, with particularly strong improvements in Korean and Japanese. We further compare multilingual OPD with English-only OPD to analyze the effect of multilingual training on OPD.

More from EMNLP 2026

All EMNLP 2026 papers →