2026
EMNLP
EMNLP 2026
Stratified Topographic Similarity
Abstract
AbstractTopographic Similarity (TopSim) is the most widely used metric for compositionality in emergent communication research. However, when evaluated on a class of languages where compositional structure is fully known (simple natural-language word concatenations), TopSim, along with other well-known metrics, under-reports compositionality. We investigate and trace TopSim’s failure to a distributional imbalance in pairwise object distances that grows with the input space size, and propose stratified-TopSim, a minimal fix that remains stable across settings and reliably evaluates languages of varying compositional structure, on both controlled benchmarks and emergent languages.