Skip to content
conftrace_
2026 EMNLP EMNLP 2026

SEAR: Self-Judging Agentic Reward Models

AbstractAgentic reward models enhance large language model (LLM) evaluation by incorporating external tool-use capabilities. However, their training is heavily constrained by coarse-grained outcome supervision and the high cost of process-level annotations. To address these challenges, we propose SEAR (Self-judging agentic reward models), a novel training framework that enables agentic reward models to self-judge their own judging processes. SEAR introduces a dual-role adversarial co-evolution framework, where the model alternates between a judger and a self-judger within a single generation process: the judger is optimized to produce more rigorous reasoning trajectories that maximize self-judgment scores, while the self-judger learns to provide critical and discriminative evaluations. Moreover, SEAR further enables agentic RMs to self-evolve without relying on external human annotations. Extensive experiments demonstrate that SEAR consistently improves the reliability of agentic reward models across diverse benchmarks.

More from EMNLP 2026

All EMNLP 2026 papers →