Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Consensus Group Relative Policy Optimization for Text Generation

AbstractMany strong decoding methods for text generation follow a sample-and-rerank paradigm: they draw multiple candidates, score each under a utility (reward) function using consensus across samples, and choose the best one. Although effective, these methods incur high computational costs during inference due to repeated sampling and scoring. Prior attempts to amortize inference-time computation typically rely on gold references, teacher labels, or curated preference data, thereby increasing the effort required to construct datasets and the demand for high-fidelity reward models. We propose Consensus Group Relative Policy Optimization (C-GRPO), which distills Minimum Bayes Risk (MBR) decoding into training by formulating the consensus utility as a group-relative objective within GRPO. C-GRPO requires only a utility function and policy samples, without gold references or explicit preference labels. As supporting analysis, we study an idealized sentence-level surrogate and show that the expected update is directionally consistent with the expected-utility objective underlying MBR decoding under simplifying assumptions. Experiments on machine translation (WMT 2024), text summarization (XSum), and Japanese social-bias mitigation (JBBQ) show that C-GRPO often matches or exceeds inference-time MBR decoding while replacing multi-sample reranking with a single generation.

More from EMNLP 2026

All EMNLP 2026 papers →