Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Labels have Human Values: Value Calibration of Subjective Tasks

AbstractAlthough pluralistic societies exhibit diverse human values that lead to legitimate disagreements in subjective tasks (e.g., safety and preference judgments), NLP models trained on such subjective labels often ignore latent value structures, resulting in miscalibrated predictions over relevant value classes. We propose MultiCalibrated Subjective Task Learning (MC-STL), a framework that identifies latent value groups from annotations (via label rationale similarity, expert value taxonomies, or annotator sociocultural descriptors) and enforces value-conditional calibration through value group-specific representations. MC-STL applies to binary, ordinal, and preference learning settings, and is evaluated on multiple datasets covering toxic chatbot conversations, value reasoning, T2I-safety and preference alignment. The results demonstrate that MC-STL consistently outperforms existing baselines, achieving multicalibration across relevant value groups, while delivering gains in probabilistic prediction performance.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →