Skip to content
conftrace_
2026 EMNLP EMNLP 2026

AyahVerse at HalluScoring-2026: Category-Aware Arabic Hallucination Detection with Multi-Layer Representations

AbstractWe present our submission to HalluScoring 2026, an ArabicNLP shared task on hallucination detection and answer recovery in Arabic LLM outputs. For Track 1, our submitted baseline fine-tunes CAMeLBERT for binary hallucination detection, achieving 0.921 and 0.763 AUC-ROC on the development and test sets of Task 1.1, and 0.749 and 0.729 on Task 1.2, respectively. We further conduct post-submission ablations with intermediate-layer supervision and category embeddings to study their effect on generalization. The combined model achieves 0.929 AUC-ROC on Task 1.1 and 0.777 on Task 1.2 development sets. For Track 2, we separate hallucination detection from answer selection, using CAMeLBERT for detection and a decoder-based, QLoRA-tuned AceGPT-v2-8B-Chat for option selection. The system achieves combined scores of 0.784 on Task 2.1 and 0.829 on Task 2.2 on the validation sets, improving substantially over the corresponding zero-shot baselines.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →