Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Auditing RAG Necessity: A Conformal Test for Source-Independent Answers

AbstractA retrieval-augmented language model returns the correct free-cash-flow figure from Microsoft’s 10-K filing. Replace the filing with a topical distractor containing no figure, and the model returns the same number: the document was never needed. We introduce RAG-CF, a training-free black-box probe that issues a per-query verdict on whether an apparently correct RAG answer is source-independent, meaning produced without the retrieved documents. The probe perturbs document context under four counterfactual operators, regenerates the answer, and converts the aggregated mismatch into a split-conformal P-value with a finite-sample Type-I bound. Across 60 configurations spanning five benchmarks, four generators, and three retrievers, the FPR-adjusted flagged rate, (raw − α)/(1 − α), ranges from 13% to 56% (raw 21% to 60%); the instrument’s FPR is domain-dependent. A held-out no-retrieval label set confirms: 78% of flagged answers remain correct without documents, against 22% of unflagged; the gap is preserved under LLM-judge equivalence. Source-independence does not decrease with retriever quality. Accuracy alone conflates grounded and parametric success; reporting the source-independence rate alongside it would make the distinction visible on every future RAG benchmark.

More from EMNLP 2026

All EMNLP 2026 papers →