Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning

AbstractRubric-based rewards are increasingly used for open-ended language model post-training, but criterion-level scores are commonly aggregated as independent utilities. This flat scalarization ignores semantic dependencies among criteria, allowing downstream credit or penalties to be counted even when the conditions that license them are absent. We call this structural failure False Credit Propagation (FCP).We propose GEAR (Graphical Event Aggregation for Rubric rewards), a dependency-aware aggregation framework for rubric rewards. GEAR represents rubric criteria as latent events in a typed dependency graph, uses monotone retention factors to infer dependency-adjusted criterion marginals, and computes a normalized expected-utility reward.This preserves compatibility with standard rubric-based reinforcement learning while adding only linear-time aggregation over criteria and edges. Experiments on HealthBench, WritingBench, and PLawBench with two policy backbones show that GEAR consistently improves over flat aggregation and hard gating, both in controlled aggregation-only settings and as a plug-in module for existing rubric-guided RL pipelines.FCP diagnostics further show that GEAR substantially reduces invalid downstream utility while avoiding the over-suppression of deterministic gating.

More from EMNLP 2026

All EMNLP 2026 papers →