Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Mitigating Forgetting Between Supervised and Reinforcement Learning Yields Stronger Reasoners

AbstractState-of-the-art (SOTA) reasoning post-training pipelines increasingly combine supervised fine-tuning (SFT) with reinforcement learning (RL) to balance external knowledge injection and self-exploration. In this work, we identify a critical but underexplored failure mode of these SFT+RL pipelines: catastrophic forgetting. To analyze the cause, we reveal a fundamental asymmetry between SFT and RL. SFT typically induces much larger, redundant parameter updates than RL, while RL’s updates are smaller and more parsimonious. Therefore, SFT updates can easily overwrite RL’s small non-robust updates, causing the forgetting of the knowledge learned by RL. To address this issue, we propose MIFO (MItigating FOrgetting between SFT and RL), a plug-and-play framework that integrates SFT into RL while explicitly constraining SFT-induced updates. MIFO mitigates forgetting through two complementary designs: (i) Data selection, which interleaves RL and SFT by selecting challenging samples based on rollout accuracy and optimizing only high-entropy tokens for SFT; and (ii) parameter freezing, which identifies RL-critical parameters and temporarily freezes them during SFT to prevent overwrite. Beyond stability, MIFO is data- and response-efficient and remains agnostic to the underlying SFT or RL algorithm. Empirically, MIFO achieves SOTA reasoning performance using only 1.5% of the SFT data and 20.4% of the RL data required by prior SOTA methods, while producing reasoning traces up to 10× shorter than the standard SFT→RL pipeline.

More from EMNLP 2026

All EMNLP 2026 papers →