Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Poly-FEVER: A Multilingual Hallucination Detection Benchmark

AbstractWe present Poly-FEVER, an open-source large-scale multilingual benchmark for hallucination detection through controlled factuality judgment in large language models. Poly-FEVER extends FEVER, Climate-FEVER, and SciFact to 77,973 Supported/Refuted claims aligned across 11 languages. We augment each claim with topic metadata from a 22-topic LDA model, enabling topic-aware analysis of multilingual factuality behavior. Translations are produced with Google Cloud Translation and validated by a targeted human audit, GEMBA, and COMETKiwi. We benchmark ChatGPT-3.5, LLaMA-2/3, Phi-3.5, and Mistral across three prompt families and evaluate robustness to model-generated rephrasing. We further analyze resource imbalance using web presence as a proxy for online claim availability, and evaluate retrieval-augmented factuality judgment with DPR over Wikipedia. Results show pronounced cross-lingual disparities under aligned claims, with stronger accuracy in high-resource languages such as English and Chinese and weaker accuracy in lower-resource languages such as Amharic and Tamil. Topic-aware prompting and retrieval provide selective gains, especially in Arabic and Amharic, while their effects vary across languages and can conflict with model priors. Poly-FEVER establishes a rigorous resource for responsible, language-adaptive hallucination detection and language-sensitive factuality evaluation. The proposed Poly-FEVER is available at: https://huggingface.co/datasets/HanzhiZhang/Poly-FEVER.

More from EMNLP 2026

All EMNLP 2026 papers →