Scoring-Location Sensitivity in Process Reward Models
Abstract
AbstractProcess reward models (PRMs) assign scores to reasoning steps, but an evaluator must also choose where to score. We ask whether that choice changes the predicted earliest error while the reasoning text stays unchanged. On 800 selected ProcessBench questions we compare a punctuation policy and a near-midpoint policy that insert equal numbers of within-step scoring locations, preserve every reasoning byte, and map decisions back to the original steps. At threshold 0.5 the two policies disagree on the predicted earliest error for 10.4%, 14.5%, and 14.4% of questions for Qwen2.5-Math-PRM, Math-Shepherd, and Skywork, and for the marker interfaces the disagreement exceeds a separator-only comparison on the same questions. The direct comparison was identified post hoc; a disjoint, prospectively frozen 400-question panel again shows 10.0–14.3% disagreement. Two mechanisms are at work: marker-based policies can change scores at the original step endpoints, whereas Skywork keeps every token score fixed and changes the predicted earliest error only through which positions are inspected. These are controlled sensitivity measurements, not prevalence estimates for naturally occurring segmenters. PRM evaluations should report marker rendering, separators, readout positions, the mapping to original steps, and the decision threshold.