Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Cultural Binding Heads in Language Models

AbstractLLMs often default to equal treatment across cultural groups, even though context warrants differentiation: this is a lack of difference awareness. Using mechanistic interpretability and a factorial design on the N4 cultural appropriation benchmark from Wang et al. (2025), we identify 2–3 mid-layer attention heads per model that contribute causally to cultural binding across eight models (base and instruct versions of four architectures). Cultural binding is the process of associating a cultural item with its related identity. Knockout of the identity→item edges on these heads lowers the binding strength by 9–23%. The identified heads transfer from instruct to base models, suggesting that cultural binding is created during pre-training. An 𝛼-scaling shows a graded dose–response. Moderate amplification steering at generation (𝛼 = 2–3) increases cultural differentiation accuracy by 1–3 pp while leaving reasoning on culturally neutral questions mostly intact. A knowledge probing task shows that models know 3–6× more than they act upon, indicating that the bottleneck lies in routing and not knowledge.

More from EMNLP 2026

All EMNLP 2026 papers →