Skip to content
conftrace_
2026 EMNLP EMNLP 2026

SetProbe: Description-First Relation-Wise Self-Consistency for Closed-Book Knowledge Base Construction

AbstractThe AKBC 2026 shared task asks a language model to recover complete object sets closed-book. SetProbe anchors each subject, samples several candidate sets, and aggregates them per relation, reaching 0.513 macro F1. Sample count, aggregation rule, and abstention thresholds each move the score by less than the evaluation set’s own 0.045 confidence interval; only the choice of model does. The binding constraint is elementary: closed-book prediction cannot recover an object that appears in no sample.

More from EMNLP 2026

All EMNLP 2026 papers →