Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Not All Evidence Is Equal: Source-Sensitive Calibration in Large Language Models

AbstractLarge language models are increasingly used in settings where information must be evaluated not only for content, but also for reliability. This paper introduces a human-grounded evaluation of source-sensitive calibration in Turkish and English. Across hearer-side uptake tasks, we manipulate source reliability while holding propositional content constant, and test whether humans and LLMs adjust action-readiness judgments according to source trustworthiness. Turkish human participants show robust source sensitivity: they are substantially more willing to act on information attributed to high-trust sources than to low-trust sources, while Turkish evidential morphology itself does not independently affect uptake once the source is overtly specified. LLMs show weaker calibration. In Turkish, models partially register explicit source reliability, but the effect is much smaller than in humans and there is no reliable effect of the -DI/-mIş contrast. In English, models show small effects of both source reliability and the lexical marker apparently, but no robust interaction between them. These results suggest that current LLMs encode source cues only partially and often fail to convert them into human-like judgments about when information is reliable enough to guide action.

More from EMNLP 2026

All EMNLP 2026 papers →