Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Beyond Static Grounding: Behavioral Contracts for Trustworthy Retrieval-Augmented Generation

AbstractCurrent RAG evaluation measures answer quality under fixed evidence but cannot detect systems that ignore evidence changes or fall back to parametric knowledge. We introduce Dynamic Evidence Contracts, a behavioral evaluation paradigm that specifies expected RAG behavior under four evidence interventions (deletion, conflict, distraction, permutation), and the Intervention Compliance Score (ICS) to quantify compliance. Evaluation on five datasets reveals that all evaluated open-source RAG pipelines achieve moderate conflict detection (CDR=0.283–0.847) via passive NLI-detected refutation signals, but lack explicit conflict acknowledgment. We further present DOVE-RAG, a minimal reference instantiation showing that explicit conflict acknowledgment yields higher CDR (0.505–0.999), up to 1.6× ICS improvement, and 2.0× citation recall gains. Ablation reveals that verification without structured output reorganization yields negligible improvement, suggesting that evidence compliance requires an integrated pipeline.

More from EMNLP 2026

All EMNLP 2026 papers →