Routing-Aware Safety Alignment for Mixture-of-Experts Models
Abstract
AbstractMixture-of-Experts (MoE) language models introduce unique challenges for safety alignment due to their sparse routing mechanisms, which can enable degenerate optimization behaviors under standard full-parameter fine-tuning. In our preliminary experiments, we observe that naively applying full-parameter safety fine-tuning to MoE models can appear to improve safety through routing or expert dominance effects, rather than by directly repairing Safety-Critical Experts. To address this challenge, we propose RASA, a routing-aware expert-level alignment framework that explicitly repairs Safety-Critical Experts while preventing routing-based bypasses. RASA identifies experts disproportionately activated by successful jailbreaks, selectively fine-tunes only these experts under fixed routing, and subsequently enforces routing consistency with safety-aligned contexts. Across three representative MoE architectures spanning 1B to 47B total parameters, RASA achieves near-perfect single-turn robustness and cross-attack generalization, maintains low attack success rate under a RASA-aware adaptive attack, and extends effectively to held-out multi-turn jailbreaks, while preserving general capabilities and controlling over-refusal. Causal necessity and sufficiency tests confirm that the identified experts drive the safety repair. Our results suggest that robust MoE safety alignment benefits from targeted expert repair rather than global parameter updates, offering a practical and architecture-preserving alternative to prior approaches.