Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Reinforcement Learning with Robust Rubric Rewards

AbstractWhile RLVR is effective for deterministically verifiable tasks, many vision-language tasks are only *partially verifiable*, demanding multi-criterion supervision. To bridge this gap, we propose *Reinforcement Learning with Robust Rubric Rewards (RLR3), extending RL with criterion-level rewards. To ensure faithful scoring, RLR3 routes each criterion to one of two execution paths: an LLM-as-an-extractor paired with a verifier, or an LLM-as-a-Judge for fuzzy criteria. We also introduce a minimal exposure strategy that masks verifier targets from extractors and source images from the GenRM. Furthermore, RLR3 employs decoupled normalization to mitigate score saturation and hierarchical aggregation to prioritize essential criteria. Evaluated on Qwen3-VL-30B-A3B across 15 benchmarks, RLR3 consistently outperforms RLVR, yielding a 4.7-point improvement over the base model and exceeding the official instruct-to-thinking model gap. Controlled audits show that deterministic verifier and minimal exposure strategy reduce exploitable false positives.

More from EMNLP 2026

All EMNLP 2026 papers →