Singing from the Same ANTHEM: An Enriched ACL Anthology for Scholarly Comparison
Abstract
AbstractComparative analyses of NLP research, such as identifying who is shaping a subfield, how a topic is shifting geographically, where method- ological gaps lie between research communi- ties, require institutional and topical metadata at a density and granularity the ACL Anthol- ogy does not itself provide. Large-scale re- sources offer partial support: OpenAlex pro- vides high-quality institutional metadata where it can match ACL papers (32% of our gold set), but the DOI dependency limits its cov- erage on older Anthology publications; and the S2ORC bulk release contains 52,013 ACL Anthology fulltexts but populates structured author–affiliation annotations on a small fraction of them in its current state. We present ANTHEM (ANTHology with Enriched Meta- data), a complementary resource that narrows this gap for NLP publications. ANTHEM links 117,386 ACL Anthology papers to Research Or- ganization Registry (ROR)-normalized author affiliations and Computer Science Ontology (CSO) topic labels via a multi-parser, LLM- based extraction pipeline that retains evidence spans for every decision, achieving F1hier of 0.760 on a 781-paper decade-stratified gold set, where parent–child ROR identifiers are treated as equivalent (e.g., a department and its parent university). On top of the corpus we release a benchmark of 99 structured comparison ques- tions across five categories, evaluated under a programmatic gold-set construction in which ANTHEM’s own metadata defines the evalu- ation ground truth. All data, code, and infras- tructure are released under CC-BY-SA-NC.