Skip to content
conftrace_
2026 AACL AACL 2026

C-VoicesCompass: Evaluating Value Alignment of LLMs with Chinese Social Values

AbstractWith the widespread deployment of Large Language Models (LLMs), safety evaluation within diverse cultural value contexts has emerged as a critical issue in AI safety research. Existing studies predominantly rely on Western value systems, such as Schwartz’s theory of basic values, lacking systematic evaluation frameworks tailored to Chinese cultural values. To address this gap, C-VoicesCompass, a value alignment evaluation framework for LLMs is proposed oriented toward Chinese Social Values (CSV). The framework aims to assess the degree of LLM alignment across 12 CSV dimensions, and constructs the first human-annotated dataset, C-Voices to cover all 12 CSV dimensions, serving as a benchmark to investigate the self-annotation capacities of LLMs. Second, a lightweight value evaluator, C-VoicesEvaluator, is trained by employing an asymmetric loss function to enhance quantitative assessment precision. Finally, a value-oriented evaluation question set is developed to induce value-related responses from LLMs, enabling a systematic alignment evaluation. Experimental results show that the proposed multi-round cross-correction annotation strategy achieves annotation quality close to human annotation. Evaluations on eight mainstream LLMs reveal significant disparities in alignment and distinct value expression tendencies across the 12 dimensions. The evaluation framework has been developed for public testing at https://cvoices.net.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from AACL 2026

All AACL 2026 papers →