Proof Identity and Divisible Correspondence in Multi-Derivation Reasoning Evaluation
Abstract
AbstractSystems that return several valid derivations require explicit choices about proof identity and proof-set correspondence. This study examines both choices on ProofWriter. An application-aware representation distinguishes grounded uses of the same source rule through canonical premise derivations. Source-ID projection collapses distinct reference proofs in 28 of 400 stratified derivable questions (7.0%). The comparison includes exact proof-set F₁, flattened-atom F₁, Optimal Subpattern Assignment with Jaccard distance (Jaccard-OSPA), and MCT-J. The label MCT-J denotes a balanced optimal-transport distance with Jaccard distance as the ground metric. On 160 fixed outputs from 32 derivable questions, source-ID projection lowers MCT-J distance by 0.0606 on average. A cluster bootstrap over the underlying ProofWriter worlds gives a 95% interval of 0.0316–0.0955. MCT-J equals Jaccard-OSPA on all 117 equal-cardinality outputs, while Jaccard-OSPA has the larger distance on 38 of 43 unequal-cardinality outputs. Of 433 generated proof members, 430 pass independent rule-level replay. Excluding the three replay failures changes the mean representation effect from 0.0606 to 0.0589. Proof identity and proof-set correspondence therefore change structural evaluation on fixed outputs.