Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Length and Meaning for Language Model Maps

AbstractVarious studies have proposed methods to quantify and compare the rapidly growing number of large language models (LLMs). However, when constructing a macroscopic map of LLMs, there is a trade-off between estimation accuracy and computational cost, making the selection of evaluation texts critical. In this work, we propose Predictive Importance Sampling (PredIS), which selects informative texts by predicting likelihood variance from text-level features. Experiments on 100 LLMs across 25 domains demonstrate that our predictor generalizes robustly. Furthermore, we reveal that text importance cannot be explained by text length alone, as it is also closely tied to meaning reflected in the embedding space.

More from EMNLP 2026

All EMNLP 2026 papers →