2026
EMNLP
EMNLP 2026
SetProbe: Description-First Relation-Wise Self-Consistency for Closed-Book Knowledge Base Construction
Abstract
AbstractThe AKBC 2026 shared task asks a language model to recover complete object sets closed-book. SetProbe anchors each subject, samples several candidate sets, and aggregates them per relation, reaching 0.513 macro F1. Sample count, aggregation rule, and abstention thresholds each move the score by less than the evaluation set’s own 0.045 confidence interval; only the choice of model does. The binding constraint is elementary: closed-book prediction cannot recover an object that appears in no sample.