Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Boundary-Aware Expert Ordering Preservation with Next-Router Supervision for MoE Quantization

AbstractMixture-of-Experts (MoE) language models scale model capacity through sparse expert activation, but their large parameter footprint remains costly to deploy. Post-training quantization (PTQ) offers a practical compression strategy; however, quantization errors can alter router decisions and cause the quantized model to activate experts different from those selected by its full-precision counterpart. Prior router-aware PTQ objectives mitigate such expert-shift by aligning router outputs or preserving top-k expert orderings and margins at a given router. Yet sequential block-wise PTQ introduces an additional failure mode: quantizing a decoder block changes the representations received by the next MoE router, potentially causing downstream expert-shift that is not directly optimized by existing objectives. To address this issue, we propose MENDS-MoE (Margin-Preserving Expert Orderings with Next-Router Decision Supervision), an auxiliary calibration objective that supervises current-block quantization using routing decisions at the next MoE router. MENDS-MoE further introduces a boundary-aware margin loss that preserves orderings among selected experts while maintaining their separation from unselected experts near the top-k boundary. It is applied only during calibration and adds no inference-time computation. Across three MoE language models under 4-bit and 3-bit-average weight-only quantization, MENDS-MoE achieves the best average downstream accuracy and the lowest perplexity in most evaluated model–bit settings. Ablation studies further support the contributions of both next-router supervision and boundary-aware margin preservation.

More from EMNLP 2026

All EMNLP 2026 papers →