Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Knowledge Base Construction from Language Models: Mechanisms and Calibrations for Multi-Object Relational Extraction under Parameter Budgets

AbstractWe describe our system for the AKBC Shared Task 2026, which asks participants to build knowledge base entries from open-weight language models under a strict 32B total-parameter budget, closed-book setting, and no fine-tuning. Our system combines three complementary families of techniques: (i) diversity-based generation, which uses multi-question paraphrase prompting and temperature sampling with self-consistency voting to maximize recall; (ii) gold-caliber post-processing, a set of generic rules that align model outputs to the gold annotation caliber (e.g., collapsing regional stock exchanges, filtering contested territories, and calibrating capacity estimates by rounding patterns); and (iii) confidence-based gating, which uses log-probability signals to reject low-confidence answers and to verify candidates through independent knowledge pathways. On the validation set our best single-model configuration (Mistral-24B, 24B parameters) reaches a macro-F1 of 0.6562. We additionally analyze which strategies transfer across base models and which fail, providing a practical recipe for closed-book KB construction within tight parameter budgets.[Code is available at https://github.com/LexieeWei/akbc2026.]

More from EMNLP 2026

All EMNLP 2026 papers →