Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Understanding Cultural Alignment in Multilingual LLMs via Natural Debate Statements

AbstractIn this work we investigate the sociocultural values learned by large language models (LLMs). We introduce CultureProbe, a novel open-access dataset constructed from natural debate statements using a multi-step methodology. The dataset is synthetically annotated with quantized sociocultural norms according to the Hofstede cultural dimensions. We verify the accuracy of the synthetic annotations using a crowdsourced human annotation experiment conducted on a representative sample. We conduct a comparative analysis between two groups of LLMs developed in different countries (U.S. and China), and use as a comparative baseline patterns observed in human measurements. Using this new dataset and the analysis above, we found that culturally-distinct LLMs largely reflect the values and norms of the countries in which they were developed, highlighting their inability to adapt to the sociocultural backgrounds of their users.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →