Summary of the AKBC 2026 Shared Task: Language Models for Knowledge Base Construction (LM-KBC, 5th Edition)
Abstract
AbstractPretrained language models (LMs) have significantly advanced a variety of knowledge engineering tasks and have shown promise as sources of factual knowledge. While prior work has studied factual knowledge through probing and prompting, reliably constructing complete knowledge bases from the parametric knowledge of LMs remains challenging. The AKBC 2026 Shared Task invited participants to construct knowledge bases directly from LMs, given specific subjects and relations. Unlike existing probing benchmarks, the challenge imposed no simplifying assumptions on relation cardinality, allowing a subject entity to be associated with zero, one, or multiple objects, including long answer sets. To ensure accessibility, the challenge featured a single track using open-weight LMs with a total neural parameter budget of up to 32B. Nineteen submissions were received, of which twelve were ranked, exploring a variety of approaches including relation-specific prompting, evidence aggregation, and model selection.