Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Agentic Rubrics: Interfacing Agents with Heterogeneous Environments using Executable Rubrics

AbstractLanguage-model agents increasingly act through tools, APIs, databases, and other stateful environments, where task success depends on external evidence rather than final text alone. Existing reward paradigms face a trade-off: preference and rubric-based feedback are flexible but often weakly grounded, while verifier-based rewards are grounded but typically tied to narrow task-specific checkers. We argue that agentic reinforcement learning needs reward interfaces: mechanisms that explicitly connect task requirements to environment evidence, provenance, and reward aggregation. We introduce **Agentic Rubrics**, an executable reward-interface view with four components: criteria, evidence acquisition, provenance, and aggregation. This view separates what should be evaluated from how evidence is obtained, audited, and converted into reward. We also specify a minimal environment contract for auditable reward construction and give an illustrative instantiation in stateful tool-use environments. In this case study, Agentic Rubrics augment final environment feedback with process-aware rewards derived from tool traces and state records, yielding modest improvements in task performance and process reliability while revealing domain-dependent limitations. These results support the position that grounded agent learning should treat reward construction as an interface problem, not only as scalar reward modeling.

More from EMNLP 2026

All EMNLP 2026 papers →