Skip to content
conftrace_
2026 EMNLP EMNLP 2026

FactNLI: Dynamic and Automated Fact-Based Augmentation of NLI Benchmarks

AbstractNatural Language Inference (NLI) is a core task for language understanding, yet existing NLI datasets are static and increasingly saturated, allowing current Large Language Models (LLMs) to perform well without fully revealing their capabilities and shortcomings. To address this problem, we propose a new data augmentation framework that automatically builds more challenging NLI datasets by iteratively fusing rich facts into the premise and hypothesis of an NLI instance. We use a strict fact filter to ensure that fused facts are non-contradictory and non-redundant. Applied to SNLI and MNLI, our augmentation substantially increases data length and complexity, and the performance of a range of LLMs on the augmented datasets drops significantly (up to 30%). Ablation experiments and human quality checks confirm the high quality of the generated data.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →